Win a copy of Kotlin in Action this week in the Kotlin forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

reading the standard out from another process  RSS feed

 
jeremy reynolds
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all, I have written a piece of C code and I want to be able to read the output from that process into my java application. All output from the C application goes to standard out. I can start the application with the Runtime.exec command, and use the Process.getInputStream() method to direct data from the running process to either java's standard in (System.setIn(inputstream)) or to a buffered reader.

The problem is not receiving of the data, but the time of receiving the data. My C application will print something, then sleep for a short time, then print something and then sleep for a short amount of time and this will happen a couple of times.My java application will read and display everything that was printed by my C application, but not with the timing it should (ie print, delay, print, delay).

By this i mean all of the data that gets printed to standard out by my C application all comes back to my java application in one big bundle of data. I want to be able to read data from this C application in "real-time", as the application diplays it.

I want to be able to let the C code print to standard out to make debugging of that code by itself easier.

Does any of that make sense? Does anyone know if this is a possibility?

Regards,
Jeremy Reynolds
 
Joe Ess
Bartender
Posts: 9429
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It's been a while since I've touched C, but I do know you can get some weird output timing if you do not flush the output after writing to it (same goes for Java).
 
Paul Clapham
Sheriff
Posts: 22502
43
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Apparently there's some kind of buffering going on. You might try simplifying your setup by having your Java program read from System.in, and just piping the C program's output to the Java program's input like this:
 
jeremy reynolds
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks, but in my case I need to start the Java app first, and the C code wont neccessarily be run.

At the moment when i start a new java process, get send the output stream straight to standard in using System.setIn(proc.getInputStream())

This may look wrong, but as the documentation says, get

"The stream obtains data piped from the standard output stream of the process represented by this Process object."

But i have a feeling all of the data is buffered in this "Process" object, and doesnt get returned until the process has completed.

As a bit of weirdness, i also collect standard error from my C application, and pipe it into an input stream as well. Interesting that all standard error information gets returned from this process object before any of the standard out.

Thanks for you help though,
Jeremy
 
jeremy reynolds
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for that, I will give it a go, but when the C application is run from the command line by itself, the prints are displayed with the correct timing so standard out seems to be flushed anyway.

I have a feeling that the data is getting buffered not in the streams, but in the "process" which doesnt return that information until the process has exited.

Regards,
Jeremy
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm pretty sure the process doesn't hold data until it's done, because I've built an interactive program that sends commands to the process and gets responses back over and over.

You've probably done this, but all I can think of is to make sure the C program flushes its outputs and avoid using buffered input in Java. As a shot in the dark, try reading the response in a separate thread just to make sure it's not waiting on something non-obvious before reading.

Oh, does your program C also write to err out? As another shot in the dark, read that in another thread, too.
 
jeremy reynolds
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks guys, turns out you need to know one small bit of obscure knowledge about how C write to output. Turns out that if something is written to a terminal window it gets written straight to it. But if you are wrting to a file (as is the case when piping outputs around), it will only write in blocks of about 4Kbytes. As I was only testing with a couple of lines of output, this would be buffered and written to the output file all in one go.

As has been mentioned, flush did the trick, but I dont want to do that everywhere. Sure enough (as is the way in the coding world) one line fixed the problem

setlinebuf(stdout);

so now it only buffers data lines at a time, which is perfect for me. I should perhaps consider doing the same with standar error, but it seems to work fine now.

Regards,
Jeremy
 
Laurent Ploix
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
When a program is started (your C program, in this case), it is either started from a terminal, either not.

For any reason that I ignore, when a program is started _not_ from a terminal, it usually does not flush its std output (and std error) stream each time it writes sg in.

Unfortunatly, when starting a process from java with Runtime.getRuntime().exec , the started programs does not 'think' that is it started from a terminal. Therefore, you have a problem when the java program waits for sg that is written to the started output. I know, I had tons of problems with my python programs that I wnated to have run from java (Yactu project).

BUT, you may find options in your started program to have them 'think' they are started by a terminal, even if it's not true. After a few tries, I know that starting a python program with the '-u -i' options works perfectly. I also know that the "cmd" under w$ works also. But I had headache with many others.

The perfect solution would be to be able to start an external program in 'terminal' mode. Someone has a solution ?

It would be perfect, beacause in most cases, you have no choice, because you can't flush the standard output of the started program.

Laurent Ploix
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!