Input redirection in Java

[Vivek Jain]

Hi,
I've a perl script called process.pl which I use to invoke through unix client as shown below:

$ process < abc.txt

Now, abc.txt contains entries per line which is processed by this perl script.

I want to invoke this process script through Java. I've captured the input which is required for this script in StringBuffer. My question is how do I pass this input which I stored in stringbuffer to this script?

I've tried below command, but it takes only 1st string within my Stringbuffer.

    
StringBuffer str= new StringBuffer();
str.append("1st line");
str.append("\n");
str.append("2nd line");
 
Process process = Runtime.getRuntime().exec("process " + str);
OutputStream outStream = process.getOutputStream();
 
int exitVal = process.waitFor();
 
if (exitVal == 0) {
     result = true;
} else {
     result = false;
}

But the above code executes & parses only 1st line in my stringbuffer. Can any one provide me the code which will take all input from this stringbuffer & pass it to process script??

Please note that I DO NOT want to create a file & then pass it to script.

Thanks in Advance!

[Freddy Wong]

Why not do something like this:

In Linux:

   
ProcessBuilder pb = new ProcessBuilder("bash", "-c", "perl", "process", "<input.txt");

In Windows:

    
ProcessBuilder pb = new ProcessBuilder("cmd", "/c", "perl", "process", "<input.txt");

[Rob Spoor]

Freddy Wong wrote:Why not do something like this:

In Linux:

   
ProcessBuilder pb = new ProcessBuilder("bash", "-c", "perl", "process", "<input.txt");

In Windows:

    
ProcessBuilder pb = new ProcessBuilder("cmd", "/c", "perl", "process", "<input.txt");

Because the <input.txt part is not part of the process - it is the shell (bash, csh, cmd.exe) that is handling the redirection.

Vivek, you need to copy the contents of the StringBuffer to outStream:

OutputStream outStream = process.getOutputStream();   
Writer writer = new OutputStreamWriter(outStream);
Reader reader = new StringReader(str.toString());
char[] buffer = new char[str.length()];
int len;
while ((len = reader.read(buffer)) != -1)
{
    writer.write(buffer, 0, len);
}
int exitVal = process.waitFor();

The similar technique can also be used to capture the output and error stream from the process.

Also, I advice you to read "When Runtime.exec() won't" article on JavaWorld.>

[Vivek Jain]

Hi Rob,
Thanks! I tried your code which is what I was expecting.....but I'm having slight issue. I tried below code both with & without "<" in the code but none of them worked. Any idea what could be wrong? The perl script "process" works fine when invoked from shell.

$ process < abc.txt

Here is the code which I tried as per your suggestion.

    
StringBuffer str= new StringBuffer();   
str.append("1st line");   
str.append("\n");   
str.append("2nd line");   
    
Process process = Runtime.getRuntime().exec("process <" + str);   

OutputStream outStream = process.getOutputStream();      
Writer writer = new OutputStreamWriter(outStream);   
Reader reader = new StringReader(str.toString());   
char[] buffer = new char[str.length()];   
int len;   
while ((len = reader.read(buffer)) != -1)   
{   
    writer.write(buffer, 0, len);   
}   
int exitVal = process.waitFor();

Thanks in Advance!

[Steve Luke]

You should read this:
When Runtime.exec()won't
It is likely to fix some of the problems (read it thoroughly and implement all parts of it).

Also:

Process process = Runtime.getRuntime().exec("process <" + str);

The "<" + str part is not part of the command, so it should be dropped. Pipe in the String like Rob suggested.

[Vivek Jain]

Hi All,
Ok...After a research I found that somehow, when I'm throwing the result to outstream, my data in stringbuffer which is added 1 entry per line is not going to the stream as it is. It is ignoring str.append("\n"); & instead adding just a space.

My perl script needs the new line & hence it is not working. The runtime command works perfectly fine.

While throwing the stringbuffer to output stream, how can I retain the new line within stringbuffer?

   
import java.io.*;

class red
{
	public static void main(String[] args) throws Exception {
		InputStream inStream = null;
		String line = null;

		StringBuffer str= new StringBuffer();      

		str.append("1st line ");
		str.append("\n");
		str.append("2nd line ");
		str.append("\n");
		str.append("3rd line ");			  

		Process process = Runtime.getRuntime().exec("process "+ str);
		OutputStream outStream = process.getOutputStream();         
		Writer writer = new OutputStreamWriter(outStream);      
		Reader reader = new StringReader(str.toString());      
		char[] buffer = new char[str.length()];      
		int len;      
		while ((len = reader.read(buffer)) != -1) {      
			writer.write(buffer, 0, len);      
		}      
		
		int exitVal = process.waitFor(); 
		if (exitVal == 0) {
			System.out.println("Success");
		} else {
			System.out.println("Error Exitval="+exitVal);
		}
	}
}

When I run this code, the perl script only reads "1st line" & then it treats "\n" as space & reads next one which "2nd line". But because of this my perl script's logic fails. Java program has to throw "\n" as new line. Can you please tell me how can it be acheived?

Thanks in Advance!

[Rob Spoor]

Try adding System.getProperty("line.separator") instead of "\n".

[Vivek Jain]

Nop...Still no luck

I tried

 str.append(System.getProperty("line.separator")); 

but no luck??

Any other idea?

Thanks!

[pete stein]

Vivek Jain wrote:Any other idea?

Yes, please look here: BeForthrightWhenCrossPostingToOtherSites

18943-input-redirection-java.html
Input redirection in Java

[Steve Luke]

<sorry ... just a repeat of what has been said before>

Have you read the article I posted earlier? Have you consumed the output? How do you know the perl script considers it one line?

[Vivek Jain]

Yes...Because I wrote the perl script. And I gave debugging output everywhere.....somehow, the the writer object does not translate new line as new line instead it converts to single space....

by the way...sorry for posting this question in other sites...but I'm going to post henceforth only at this site.....

Thanks!

[Rob Spoor]

Please Use Real Words. No more "Bcuz"

[Moojid Hamid]

Vivek Jain wrote:I gave debuging output everywhere

It seems like you have not yet read the article that Steve mentioned. In your code you are not even reading the output of the process, how do you know what is the debug output of your Perl script? Have you had read that article, you would have known that you need to consume the output of the process to keep it from deadlocking.

Runtime.exec() is not trivial to use. The above mentioned article is a must read if you want to do it right.

[Vivek Jain]

My perl script reads input from each line & on each line it expects 2 parameters separated by a space. 2nd parameter is optional. So if I don't pass 2nd parameter, then it takes only 1st parameter, do some processing with that parameter & then inserts record into database. However, if I pass 2 parameters to perl script, then it reads those 2 parameters, processes them & then put them both in database as a single record under different columns.

Now, in this example, the reason I'm saying that Runtime is being executed is because I can see the record stored in the database which is done by perl script. However, it stores as shown below:

"1st line" "2nd line"

Here, these 2 inputs go into database as a single record which means the newline character is converted to space & the perl script thinks that "2nd line" is an 2nd parameter of 1st input.

If it would have treated it as newline character, I would have seen 2 records in my database but somehow, the newline character is getting converted to space which is causing the problem....

Thanks!

[Moojid Hamid]

   Process process = Runtime.getRuntime().exec("process "+ str); 

Why "process "+ str? can you try it without "+ str" since we are outputting str later in the code and report back what you get. Does your Perl script ever output to standard out or standard error? You will need the output gobblers in that case.

[Vivek Jain]

Hi Moojid,
I tried without "str" too but then when I run the Java program, it waits for input in the console...My perl script outputs to STDERR & returns code 0 on success (exit 0;)

Thanks!

[Vivek Jain]

okay...Sorry for all confusion.....After looking into perl script thoroughly (its 1000 line code....), I found that perl script is not getting any parameter....I'm providing both my perl test program & java program...

   
import java.io.*;

class dummytest
{
	public static void main(String[] args) throws Exception {
		InputStream inStream = null;
		String line = null;

		StringBuffer str= new StringBuffer();      

		str.append("1stLine 1stOptionalParameter");
		str.append("\n");   
		str.append("2ndLine 2ndOptionalParameter");
		str.append("\n");
		str.append("3rdLine 3rdOptionalParameter");
			 
		Process process = Runtime.getRuntime().exec("/usr/local/bin/demo "+ str);

		OutputStream outStream = process.getOutputStream();         
		OutputStreamWriter writer = new OutputStreamWriter(outStream);     
		
		Reader reader = new StringReader(str.toString());      
		char[] buffer = new char[str.length()];      
		int len;      
		while ((len = reader.read(buffer)) != -1) {      
			writer.write(buffer, 0, len);      
		}

		int exitVal = process.waitFor(); 
		if (exitVal == 0) {
			System.out.println("Success");
			inStream = process.getInputStream();				
			BufferedReader bufReader = new BufferedReader(new InputStreamReader(inStream));

			while ((line = bufReader.readLine()) != null) {
				if (line != null && !line.equalsIgnoreCase("")) {
					System.out.println("line=="+line);
				}
			}
		} else {
			System.out.println("Error ="+exitVal);
		}
	}
}

My perl test program is shown below:

    
#!/usr/bin/perl -w

open OUTFILE, ">/out.txt";
	
print OUTFILE "Outside the loop\n";

while (<>) {
	print OUTFILE "Inside the loop\n";
    chop;
    print OUTFILE "this is line: \"", $_, "\"\n"; 
}

close OUTFILE;

Why its not getting any parameter? It is printing "Outside the loop" but not going to while loop at all which means its not getting any parameter.

Thanks in Advance!

[Moojid Hamid]

add a writer.flush() after the while loop.

[Steve Luke]

First, you really should read the article I posted earlier. It is a must read. Really. No really you should. I know that you have already been told 2 or 3 times, but you really need to do it.

That said: By reading the article I posted and implementing its suggestions I found an error reported in your script. The error suggested that Majood's suggestion to run without appending the "str" to the command line is necessary. The command line I ended up running was:

            Process perlRunner = Runtime.getRuntime().exec(
                            new String[] { "perl", "demo.pl" }, null,
                            new File(cmdDir));

If I didn't add "perl" to the command line, and ran it like this:

            Process perlRunner = Runtime.getRuntime().exec(
                            new String[] { "demo.pl" }, null,
                            new File(cmdDir));

The code runs without error (nothing sent to the error output stream), but it appeared no STDIN was created and not lines were appended to the file. When I add the perl executable as part of the command it worked fine. I am running in windows. Things may be different in other platforms. I also don't claim to be a Perl programmer so I could be way off... All I know is that it appears like no input ever reached Perl if I didn't use the "perl" part of the command line.

[Moojid Hamid]

Steve I have been called lots of names but never Majood, I ended up googling it

it appeared no STDIN was created and not lines were appended to the file. When I add the perl executable as part of the command it worked fine.

IMO the Java part of the stream does not write to the output stream until you flush or close it, stream has to be closed to send EOF and terminate the Perl script, otherwise it hangs waiting for input and Java hangs on process.waitFor() since the Perl is hung. The fact that the script partially runs even without passing the string "perl" suggests that is not the cause of problem.

[Rob Spoor]

You still haven't read that JavaWorld article, have you? Because it explains exactly why your Process is hanging, and also how to solve it.

[Steve Luke]

Moojid Hamid wrote:Steve I have been called lots of names but never Majood, I ended up googling it

Oh My, I am so sorry!

it appeared no STDIN was created and not lines were appended to the file. When I add the perl executable as part of the command it worked fine.

IMO the Java part of the stream does not write to the output stream until you flush or close it, stream has to be closed to send EOF and terminate the Perl script, otherwise it hangs waiting for input and Java hangs on process.waitFor() since the Perl is hung. The fact that the script partially runs even without passing the string "perl" suggests that is not the cause of problem.

Just as an FYI, I added flush and close before posting, and added output to the Perl script for when it completed reading STDIN. It appeared to instantly end reading STDIN (ie I would get 'Start Reading Input' followed directly by 'End Reading Input' which occurs after the loop). I would post code but I really want to see OP show he is paying attention.