Java And R, anyone?

[Mike London]

Hello all,

Leaving a message here asking if anyone has done any Java and R programming.

I've got a sticky problem that I would like to ask a question about regarding R and Java code.

Please reply if you've done any R with Java and I'll post my short Java program.

(R is a powerful, yet free, statistics program. https://www.r-project.org/)

Thanks in advance,

-mike

[Piet Souris]

hi Mike,

at the office we use R a lot but no java, and at home I use java a little but I have no direct usages for R sofar.
So, show us the code and your problem, since "never shot is always a miss".

[Mike London]

Piet Souris wrote:hi Mike,

at the office we use R a lot but no java, and at home I use java a little but I have no direct usages for R sofar.
So, show us the code and your problem, since "never shot is always a miss".

Hey Piet,

Thanks for your reply.

What I'm trying to do is a Box's M test from Java given the same included R "Iris" dataset that the "biotools" package (which has Box's M) uses.

The example they list in biotools uses this same data set with this example usage:

Examples

data(iris)    <<< This line is for reading an included dataset, but I'm supplying the dataset separately below.
boxM(iris[, -5], iris[, 5])  <<< This is the line my Java code creates.

=================

My steps are:

(Assuming you have a connection to RServe in the code below)

* require("Rserve")
* Rserve(args="--no-save")

----------------------------------

1. Loading the biotools library

 rConnection.eval("library('biotools')");

2. Reading the Iris sample (included R dataset) into a String variable

  String inputIris = "5.1,3.5,1.4,0.2,setosa\n" +
                    "4.9,3,1.4,0.2,setosa\n" +
                    "4.7,3.2,1.3,0.2,setosa\n" +
                    "4.6,3.1,1.5,0.2,setosa\n" +
                    "5,3.6,1.4,0.2,setosa\n" +
                    "5.4,3.9,1.7,0.4,setosa\n" +
                    "4.6,3.4,1.4,0.3,setosa\n" +
                    "5,3.4,1.5,0.2,setosa\n" +
                    "4.4,2.9,1.4,0.2,setosa\n" +
                    "4.9,3.1,1.5,0.1,setosa\n" +
                    "5.4,3.7,1.5,0.2,setosa\n" +
                    "4.8,3.4,1.6,0.2,setosa\n" +
                    "4.8,3,1.4,0.1,setosa\n" +
                    "4.3,3,1.1,0.1,setosa\n" +
                    "5.8,4,1.2,0.2,setosa\n" +
                    "5.7,4.4,1.5,0.4,setosa\n" +
                    "5.4,3.9,1.3,0.4,setosa\n" +
                    "5.1,3.5,1.4,0.3,setosa\n" +
                    "5.7,3.8,1.7,0.3,setosa\n" +
                    "5.1,3.8,1.5,0.3,setosa\n" +
                    "5.4,3.4,1.7,0.2,setosa\n" +
                    "5.1,3.7,1.5,0.4,setosa\n" +
                    "4.6,3.6,1,0.2,setosa\n" +
                    "5.1,3.3,1.7,0.5,setosa\n" +
                    "4.8,3.4,1.9,0.2,setosa\n" +
                    "5,3,1.6,0.2,setosa\n" +
                    "5,3.4,1.6,0.4,setosa\n" +
                    "5.2,3.5,1.5,0.2,setosa\n" +
                    "5.2,3.4,1.4,0.2,setosa\n" +
                    "4.7,3.2,1.6,0.2,setosa\n" +
                    "4.8,3.1,1.6,0.2,setosa\n" +
                    "5.4,3.4,1.5,0.4,setosa\n" +
                    "5.2,4.1,1.5,0.1,setosa\n" +
                    "5.5,4.2,1.4,0.2,setosa\n" +
                    "4.9,3.1,1.5,0.2,setosa\n" +
                    "5,3.2,1.2,0.2,setosa\n" +
                    "5.5,3.5,1.3,0.2,setosa\n" +
                    "4.9,3.6,1.4,0.1,setosa\n" +
                    "4.4,3,1.3,0.2,setosa\n" +
                    "5.1,3.4,1.5,0.2,setosa\n" +
                    "5,3.5,1.3,0.3,setosa\n" +
                    "4.5,2.3,1.3,0.3,setosa\n" +
                    "4.4,3.2,1.3,0.2,setosa\n" +
                    "5,3.5,1.6,0.6,setosa\n" +
                    "5.1,3.8,1.9,0.4,setosa\n" +
                    "4.8,3,1.4,0.3,setosa\n" +
                    "5.1,3.8,1.6,0.2,setosa\n" +
                    "4.6,3.2,1.4,0.2,setosa\n" +
                    "5.3,3.7,1.5,0.2,setosa\n" +
                    "5,3.3,1.4,0.2,setosa\n" +
                    "7,3.2,4.7,1.4,versicolor\n" +
                    "6.4,3.2,4.5,1.5,versicolor\n" +
                    "6.9,3.1,4.9,1.5,versicolor\n" +
                    "5.5,2.3,4,1.3,versicolor\n" +
                    "6.5,2.8,4.6,1.5,versicolor\n" +
                    "5.7,2.8,4.5,1.3,versicolor\n" +
                    "6.3,3.3,4.7,1.6,versicolor\n" +
                    "4.9,2.4,3.3,1,versicolor\n" +
                    "6.6,2.9,4.6,1.3,versicolor\n" +
                    "5.2,2.7,3.9,1.4,versicolor\n" +
                    "5,2,3.5,1,versicolor\n" +
                    "5.9,3,4.2,1.5,versicolor\n" +
                    "6,2.2,4,1,versicolor\n" +
                    "6.1,2.9,4.7,1.4,versicolor\n" +
                    "5.6,2.9,3.6,1.3,versicolor\n" +
                    "6.7,3.1,4.4,1.4,versicolor\n" +
                    "5.6,3,4.5,1.5,versicolor\n" +
                    "5.8,2.7,4.1,1,versicolor\n" +
                    "6.2,2.2,4.5,1.5,versicolor\n" +
                    "5.6,2.5,3.9,1.1,versicolor\n" +
                    "5.9,3.2,4.8,1.8,versicolor\n" +
                    "6.1,2.8,4,1.3,versicolor\n" +
                    "6.3,2.5,4.9,1.5,versicolor\n" +
                    "6.1,2.8,4.7,1.2,versicolor\n" +
                    "6.4,2.9,4.3,1.3,versicolor\n" +
                    "6.6,3,4.4,1.4,versicolor\n" +
                    "6.8,2.8,4.8,1.4,versicolor\n" +
                    "6.7,3,5,1.7,versicolor\n" +
                    "6,2.9,4.5,1.5,versicolor\n" +
                    "5.7,2.6,3.5,1,versicolor\n" +
                    "5.5,2.4,3.8,1.1,versicolor\n" +
                    "5.5,2.4,3.7,1,versicolor\n" +
                    "5.8,2.7,3.9,1.2,versicolor\n" +
                    "6,2.7,5.1,1.6,versicolor\n" +
                    "5.4,3,4.5,1.5,versicolor\n" +
                    "6,3.4,4.5,1.6,versicolor\n" +
                    "6.7,3.1,4.7,1.5,versicolor\n" +
                    "6.3,2.3,4.4,1.3,versicolor\n" +
                    "5.6,3,4.1,1.3,versicolor\n" +
                    "5.5,2.5,4,1.3,versicolor\n" +
                    "5.5,2.6,4.4,1.2,versicolor\n" +
                    "6.1,3,4.6,1.4,versicolor\n" +
                    "5.8,2.6,4,1.2,versicolor\n" +
                    "5,2.3,3.3,1,versicolor\n" +
                    "5.6,2.7,4.2,1.3,versicolor\n" +
                    "5.7,3,4.2,1.2,versicolor\n" +
                    "5.7,2.9,4.2,1.3,versicolor\n" +
                    "6.2,2.9,4.3,1.3,versicolor\n" +
                    "5.1,2.5,3,1.1,versicolor\n" +
                    "5.7,2.8,4.1,1.3,versicolor\n" +
                    "6.3,3.3,6,2.5,virginica\n" +
                    "5.8,2.7,5.1,1.9,virginica\n" +
                    "7.1,3,5.9,2.1,virginica\n" +
                    "6.3,2.9,5.6,1.8,virginica\n" +
                    "6.5,3,5.8,2.2,virginica\n" +
                    "7.6,3,6.6,2.1,virginica\n" +
                    "4.9,2.5,4.5,1.7,virginica\n" +
                    "7.3,2.9,6.3,1.8,virginica\n" +
                    "6.7,2.5,5.8,1.8,virginica\n" +
                    "7.2,3.6,6.1,2.5,virginica\n" +
                    "6.5,3.2,5.1,2,virginica\n" +
                    "6.4,2.7,5.3,1.9,virginica\n" +
                    "6.8,3,5.5,2.1,virginica\n" +
                    "5.7,2.5,5,2,virginica\n" +
                    "5.8,2.8,5.1,2.4,virginica\n" +
                    "6.4,3.2,5.3,2.3,virginica\n" +
                    "6.5,3,5.5,1.8,virginica\n" +
                    "7.7,3.8,6.7,2.2,virginica\n" +
                    "7.7,2.6,6.9,2.3,virginica\n" +
                    "6,2.2,5,1.5,virginica\n" +
                    "6.9,3.2,5.7,2.3,virginica\n" +
                    "5.6,2.8,4.9,2,virginica\n" +
                    "7.7,2.8,6.7,2,virginica\n" +
                    "6.3,2.7,4.9,1.8,virginica\n" +
                    "6.7,3.3,5.7,2.1,virginica\n" +
                    "7.2,3.2,6,1.8,virginica\n" +
                    "6.2,2.8,4.8,1.8,virginica\n" +
                    "6.1,3,4.9,1.8,virginica\n" +
                    "6.4,2.8,5.6,2.1,virginica\n" +
                    "7.2,3,5.8,1.6,virginica\n" +
                    "7.4,2.8,6.1,1.9,virginica\n" +
                    "7.9,3.8,6.4,2,virginica\n" +
                    "6.4,2.8,5.6,2.2,virginica\n" +
                    "6.3,2.8,5.1,1.5,virginica\n" +
                    "6.1,2.6,5.6,1.4,virginica\n" +
                    "7.7,3,6.1,2.3,virginica\n" +
                    "6.3,3.4,5.6,2.4,virginica\n" +
                    "6.4,3.1,5.5,1.8,virginica\n" +
                    "6,3,4.8,1.8,virginica\n" +
                    "6.9,3.1,5.4,2.1,virginica\n" +
                    "6.7,3.1,5.6,2.4,virginica\n" +
                    "6.9,3.1,5.1,2.3,virginica\n" +
                    "5.8,2.7,5.1,1.9,virginica\n" +
                    "6.8,3.2,5.9,2.3,virginica\n" +
                    "6.7,3.3,5.7,2.5,virginica\n" +
                    "6.7,3,5.2,2.3,virginica\n" +
                    "6.3,2.5,5,1.9,virginica\n" +
                    "6.5,3,5.2,2,virginica\n" +
                    "6.2,3.4,5.4,2.3,virginica\n" +
                    "5.9,3,5.1,1.8,virginica\n";

3. Read the CSV file data from the String variable above, using TextConnection and "csv.read" to create a List

 List tableRead = rConnection.eval(
                    "read.csv(textConnection(\"" + inputIris + "\"), header = FALSE)").asList();

            double[] d1 = ((REXPVector) ((RList) tableRead).get(0)).asDoubles();
            double[] d2 = ((REXPVector) ((RList) tableRead).get(1)).asDoubles();
            double[] d3 = ((REXPVector) ((RList) tableRead).get(2)).asDoubles();
            double[] d4 = ((REXPVector) ((RList) tableRead).get(3)).asDoubles();
            String[] d5 = ((REXPVector) ((RList) tableRead).get(4)).asStrings();

4. Create an R "data.frame" using those objects

 

REXP myDf = REXP.createDataFrame(new RList(
                    new REXP[]
                            {
                                    new REXPDouble(d1),
                                    new REXPDouble(d2),
                                    new REXPDouble(d3),
                                    new REXPDouble(d4),
                                    new REXPString(d5)
                            }));

5. Assign the data.frame variable to an R variable

   

 rConnection.assign("boxMVariable", myDf);

6. Create a string variable with the entire command for R to evaluate.

  String boxVariable = "boxM(boxMVariable[, -5], boxMVariable[, 5])";

7. Use R's special debugging command to catch the "actual" R error thrown.

 

 REXP rResponseObject = rConnection.parseAndEval("try(eval(" + boxVariable+ "),silent=TRUE)");
            if (rResponseObject.inherits("try-error"))
            {
                System.out.println("R Serve Eval Exception : " + rResponseObject.asString());
            }

In this case, that error is:  Error in `[.data.frame`(boxMVariable, , -5) : undefined columns selected

Any suggestions??

Can you duplicate this issue using the same data and maybe see how to fix this?

Thanks in advance,

- mike

[Piet Souris]

Sorry for responding so late!

A few questions about some things that are not clear to me (note that I have never done anything with R and Java together, at the office we have no java unfortunately):

1) does your java program write a complete R-program about Box M test, up and ready to be executed in R?
2) or does your java program simply (well, simply...) do the equivalent of R's Box M?
3) What are these 'rConnection' and 'TextConnection'?

Tomorrow att the office I'll try to load the 'biotools' package, to practise a bit. Last time I used a Box M test... ehh, can't remember! But that won't be a problem.

I'll do my best!

[Mike London]

Piet Souris wrote:Sorry for responding so late!

A few questions about some things that are not clear to me (note that I have never done anything with R and Java together, at the office we have no java unfortunately):

1) does your java program write a complete R-program about Box M test, up and ready to be executed in R?
2) or does your java program simply (well, simply...) do the equivalent of R's Box M?
3) What are these 'rConnection' and 'TextConnection'?

Tomorrow att the office I'll try to load the 'biotools' package, to practise a bit. Last time I used a Box M test... ehh, can't remember! But that won't be a problem.

I'll do my best!

Hi Piet,

Thanks for your reply.

Answers to questions:

1. The Java program is trying to call R using the connection (I'll answer that in 3) and pass the structures created to R. I can get R to work with simpler things like T Tests and descriptive statistics, but this data.frame is not working. So, I'm creating R structures and then using the R connection using the "eval" method to have R do it. You can see where I'm doing an "eval" of the "boxM". That's where I'm trying to tell R: "Hey R, you have everything you need, so just compute the boxM already...". R isn't happy yet.

2. That's an interesting idea, but I would imagine writing the boxM from scratch would be incredibly difficult with all the co-variance matrices computations. Perhaps Apache Math library could help with the intermediate calcs, but doing boxM from scratch would be a bear.

3. First, TextConnection. I just learned about that one after having an increbily hard time reading a CSV.

https://stat.ethz.ch/R-manual/R-devel/library/base/html/textconnections.html

RConnection:

You need to download a couple JARs for your Java project.

See: https://www.rforge.net/Rserve/files/

My imports on the Java program are:

import org.rosuda.REngine.*;
import org.rosuda.REngine.Rserve.RConnection;
import org.rosuda.REngine.Rserve.RserveException;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;

Once you have those imports you can create a connection to R.

In R or RStudio, you need to first start RServe, like this:

require("Rserve")
Rserve(args="--no-save")

---

Once you have RServe started, with the imports in your Java program (JARS part of the project), you can then:

RConnection rConnection = new RConnection();

Then you have a connection to R in your Java program via "Rserve"!

(https://www.rforge.net/Rserve/)

----

Hope this info is useful.

Please write back with any questions or setup problems, OK?

Look forward to hearing back.

Thanks,

- mike

[Piet Souris]

hi Mike,

It was not much use doing it at the office, for lack of java, so tried it at home this evening.

Bit of a setback, I'm afraid.

Loaded everything in R (rJava, biotools and Rserve), oke
created a project in NetBeans, and copied rTest.java into it (is part of the rJava package in R)
added JRI.jar, JRIEngine.jar and REngine.jar to the library of that project
added the %RHOME%\bin and %R_HOME%\library\rJava\jri to the PATH in w10

Then tried to run rtest.java, but I got the following errors:

run:
Cannot find JRI native library!
Please make sure that the JRI native library is in a directory listed in java.library.path.

java.lang.UnsatisfiedLinkError: D:\R-3.1.2\library\rJava\jri\jri.dll: Can't load IA 32-bit .dll on a AMD 64-bit platform
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1857)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at org.rosuda.JRI.Rengine.<clinit>(Rengine.java:19)
at rjava.rtest.main(rtest.java:63)

It drives me nuts! Any idea what is happening here?

Edit: I was following this (rather old) website: http://www.studytrails.com/r/r-and-java-jri-using-netbeans/

[Mike London]

Piet Souris wrote:hi Mike,

It was not much use doing it at the office, for lack of java, so tried it at home this evening.

Bit of a setback, I'm afraid.

Loaded everything in R (rJava, biotools and Rserve), oke
created a project in NetBeans, and copied rTest.java into it (is part of the rJava package in R)
added JRI.jar, JRIEngine.jar and REngine.jar to the library of that project
added the %RHOME%\bin and %R_HOME%\library\rJava\jri to the PATH in w10

Then tried to run rtest.java, but I got the following errors:

run:
Cannot find JRI native library!
Please make sure that the JRI native library is in a directory listed in java.library.path.

java.lang.UnsatisfiedLinkError: D:\R-3.1.2\library\rJava\jri\jri.dll: Can't load IA 32-bit .dll on a AMD 64-bit platform
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1857)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at org.rosuda.JRI.Rengine.<clinit>(Rengine.java:19)
at rjava.rtest.main(rtest.java:63)

It drives me nuts! Any idea what is happening here?

Edit: I was following this (rather old) website: http://www.studytrails.com/r/r-and-java-jri-using-netbeans/

Hi Piet,

I totally get your frustration. I'm assuming your code compiles and that the problem is a run time issue?

However, being on a Mac, I never have any "DLL" ("Hell") type issues.

In any case, I saw postings on StackOverflow, like this one:

https://stackoverflow.com/questions/19100875/java-r-interface-jri-setup

which may help you solve your issue.

Configuration is a royal pain with R. Seemingly very brittle.

---

Let me know if the posting above or one like it helps solve your issue.

Thanks,

- mike

[Piet Souris]

hi Mike,

I'll certainly give it a try! Being able to call R from within java sounds fantastic. The boxM test that you used works a treat in R, so that's not the problem. Well, hopefully I can get it to work and try to reproduce your problem. That'll be this evening.

[Mike London]

Piet Souris wrote:hi Mike,

I'll certainly give it a try! Being able to call R from within java sounds fantastic. The boxM test that you used works a treat in R, so that's not the problem. Well, hopefully I can get it to work and try to reproduce your problem. That'll be this evening.

Please don't get the idea that my relationship with "R" has been, well, easy.

I ran into a nasty problem where R couldn't find a Virtual Machine and threw a ridiculous error message. After about a full day banging my head on the desk (maybe I should stop doing that? LOL), the "solution" I had to use was to re-install (even though I had Java 8, 151 installed) .... Java ....... 6.  Nothing else I tried (JAVA_HOME, ....) worked. Nothing.

Then, I ran into another problem after upgrading R to 3.4.2 hoping it would fix a problem when I then this error:

Error : .onLoad failed in loadNamespace() for 'rJava', details:
 call: dyn.load(file, DLLpath = DLLpath, ...)
 error: unable to load shared object '/Library/Frameworks/R.framework/Versions/3.3/Resources/library/rJava/libs/rJava.so':
 dlopen(/Library/Frameworks/R.framework/Versions/3.3/Resources/library/rJava/libs/rJava.so, 6): Library not loaded: @rpath/libjvm.dylib
 Referenced from: /Library/Frameworks/R.framework/Versions/3.3/Resources/library/rJava/libs/rJava.so
 Reason: image not found

After another hour or so of searching, the solution was:

$ sudo ln -f -s $(/usr/libexec/java_home)/jre/lib/server/libjvm.dylib /usr/local/lib

---

All in all, R is challenging!

You sound like you know R better than I do (been a long time since I've done statistics daily) so once you get it working with Java, that will be great.

---

Remember that you can also run Java ... FROM R.

Look forward to hearing back.

-- mike

[Piet Souris]

hi Mike,

I tried following your StackOverflow link. Problem is: I don't really understand what these guys are saying, I seem to be too big a dummy for that. But I copied dll's all over the place, added some in my Java/bin map, editted PATH, JAVA_HOME, R_HOME and the lot, gosh I really didn't know what I was doing. But the JRI native error disappeared, so did the IA 32 dll misery, only to be replaced by some dependency error. Hmm, I would not be surprised if my Java stopped working...

Anyway, I just noticed that in that StackOverflow topic there was, at the end, a detailed instruction that just looks I might be able to understand. I'll give that a try tonight!

If I don't get it up and runing tomorrow, then try StackOverflow otherwise you'll be retired before I get somewhere.

But meanwhile: the error you report says that 'boxMvariable[,-5]' has undefined columns selected. Can you mimic the simple expression 'c <- dim(boxMVariable)' and see how many columns there are? (c should give 150 by 5). Or mimic 'c <- boxMVariable[,-5]'? Or simple 'c <- boxMvariable[,5]'? See if you get any errors with that.

Well, I'm not giving up.

[Mike London]

Piet Souris wrote:hi Mike,

I tried following your StackOverflow link. Problem is: I don't really understand what these guys are saying, I seem to be too big a dummy for that. But I copied dll's all over the place, added some in my Java/bin map, editted PATH, JAVA_HOME, R_HOME and the lot, gosh I really didn't know what I was doing. But the JRI native error disappeared, so did the IA 32 dll misery, only to be replaced by some dependency error. Hmm, I would not be surprised if my Java stopped working...

Anyway, I just noticed that in that StackOverflow topic there was, at the end, a detailed instruction that just looks I might be able to understand. I'll give that a try tonight!

If I don't get it up and runing tomorrow, then try StackOverflow otherwise you'll be retired before I get somewhere.

But meanwhile: the error you report says that 'boxMvariable[,-5]' has undefined columns selected. Can you mimic the simple expression 'c <- dim(boxMVariable)' and see how many columns there are? (c should give 150 by 5). Or mimic 'c <- boxMVariable[,-5]'? Or simple 'c <- boxMvariable[,5]'? See if you get any errors with that.

Well, I'm not giving up.

Yes, please do not give up. We can figure this out.

I added the line of code to my test program:

REXP resultBV = rConnection.eval("dim(boxMVariable)");

Results attached.

---

StackOverflow may not be the best resource.

You might also consider joining the R-Help mailing list and asking questions there, too.

https://stat.ethz.ch/mailman/listinfo/r-help

Look forward to hearing how it's going. I know it's frustrating, been there, to have all these errors just to get to the part where there's a problem...

Thanks for hanging in there!

- mike

[Piet Souris]

hi Mike,

free afternoon, so I had time to continue this "continuing story of Peyton Place" with Mike & Piet starring as Rodney and Leslie Harrington.

Well, I still can't believe it, but I got the stuff up and running! I followed your SO-link, and in there the part that came along with some nice pictures! Thank you, whoever that was. It was not exactly applicable to what I had, but this time I understood the intention. Now, it seems that you and I are the only ones working with R, but if there is someone out there who would like to have a description of what I did, drop here a line.

Now, for the bad news:

1) a lot is fired at me! It'll take me some time to find my way with JRI, REngine and REXP.
2) My first impression of your problem was that maybe an expression like "iris[,-5]" was not supported. Well, it works a treat.
3) As a first exercise I added this code to the "RTest.java" program:

System.out.println("****************************************");
System.out.println("trying Mike's problem");
System.out.println(x = re.eval("names(iris)"));

with the output:

****************************************
trying Mike's problem
[STRING* ("Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width", "Species")]

Well, not bad for a complete beginner.
4) But then the dissappointing part:

System.out.println(x = re.eval("boxM(iris[, -5], iris[,5])"));

gave a disappointing

Error: could not find function "boxM"

5) Then I tried this code:

x = re.eval("iris");
System.out.println(("x = re.eval(iris) = " + x));
System.out.println("trying ");
System.out.println(y = re.eval("class(iris)"));
RVector v = x.asVector();
if (v == null) {
    System.out.println("in 'RVector v = x.asVector' v is NULL!!!!!");
    System.exit(1);
}
if (v.getNames() != null) {
    System.out.println("has names:");
    for (Enumeration e = v.getNames().elements(); e.hasMoreElements();) {
        System.out.println(e.nextElement());
    }
}

And, as if by magic, no NPE (which I did get at first, just like you had). Output:

trying 
[STRING "data.frame"]
has names:
Sepal.Length
Sepal.Width
Petal.Length
Petal.Width
Species
and once again from the list:  // et cetera

Hmmm... seems perfect, no NPE, no boxM, and I have a lot to learn. Maybe it was jut the case of putting 'x = re.eval(iris)' outside the println command?

What do you think?

[Mike London]

Cool!

It sounds like you're getting close to where I couldn't evaluate the boxM call, too.

In my case, it found the boxM, but it just couldn't evaluate it.

I also tried double brackets, but that just gave a different error.

--

My current "workaround" is to read the R Script and execute it line by line from Java - instead of trying to create my own data structures and then communicate them with R. That puts the onus on the R side and keeps me from trying to reproduce complex things in Java which can be done easily in R.

I also may be moving to Python for this project as it seems to have much richer Statistics libraries for this kind of work.

(I couldn't find a single serious Java statistics library, paid or free. Apache Math has some relatively basic stuff, but nothing like Box's M or Logistic Regression or Discriminant Analysis.)

R is scary in other ways, too. It's single threaded and on a Devoxx presentation I watched, apparently R can take down the JVM and any container, too.

Yikes!!!

Look forward to your next update.

- mike

[Piet Souris]

hi Mike,

I experimented a little further, my code looks very clumsy at this stage, but it is not hard to get the results of that boxM.

First of all, I added "re.eval("require(biotools)");", omitting that gave me that error in my previous post.
Then I executed this code:

System.out.println("****************************************");
System.out.println("trying Mike's problem");
System.out.println(re.eval("names(iris)"));
System.out.println("trying out the boxM....");
re.eval("z <- boxM(iris[, -5], iris[,5])");
RVector z = re.eval("z").asVector();
System.out.println("and printing it");
System.out.println("chi_sqr: " + z.at(0));
System.out.println("degrees of freedom: " + z.at(1));
System.out.println("p value: " + z.at(2));
System.out.println("*********************************");

And it gave me this outcome:

****************************************
trying Mike's problem
[STRING* ("Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width", "Species")]
trying out the boxM....
and printing it
chi_sqr: [REAL* (140.94304992349774)]
degrees of freedom: [REAL* (20.0)]
p value: [REAL* (3.352034178317223E-20)]
*********************************

Now, I don't know yet what these Vector values are, and if you can use the numeric value in it, but that'll be the next thing to investigate.

But have a look at the documentation of the 'boxM' method in R. The value that method returns is a List of class "htest" containing the values statistic, parameter, p.value, cov et cetera. Turning that into an RVector makes all the parts available with "vector.at(0)", et cetera.

[Mike London]

Nice, but my issue is getting the CSV to play nicely.

Using your idea....

           String boxVariable = "z<-boxM(boxMVariable[, -5], boxMVariable[, 5])";
           REXP res = rConnection.eval(boxVariable);

Just gives the same error for me.

Were you able to get my code working with your method?

I'm sure I have something silly like a minor syntax error, but I haven't found it yet...

- mike

[Piet Souris]

Well, I'm sorry but at the moment my knowledge is simply insufficient to understand instructions like REXP(List(REXP(d[0] et cetera. This method does seem pretty awkward, though.

For now, the easiest that I could think of was: saving iris to a file (normally you would have your data in some sort of file anyway), and then applying the code I already gave:

re.eval("iris_file <- read.csv('d:/iris.csv')");
System.out.println("trying out the boxM....");
re.eval("z <- boxM(iris_file[, -5], iris_file[,5])");
RVector z = re.eval("z").asVector();
System.out.println("and printing it");
System.out.println("chi_sqr: " + z.at(0));
System.out.println("degrees of freedom: " + z.at(1));
System.out.println("p value: " + z.at(2));
System.out.println("*********************************");

Edit: maybe you could supply your code again? That would make it a little easier for me to follow.

[Piet Souris]

I just tried to copy your code line for line into a java class. But I run into problems with this:

String str = "read.csv(textConnection(\"" + inputIris + "\"), header = FALSE)";
System.out.println(str);
List tableRead = re.eval(str).asList();   // re is an instance of Rengine

I get an error "RList cannot be converted to List"
I must change it to

org.rosuda.JRI.RList tableRead = re.eval(str).asList();

before the error disappeared. But then I got an error at these lines:

double[] d1 = ((REXPVector) ((RList) tableRead).get(0)).asDoubles();
double[] d2 = ((REXPVector) ((RList) tableRead).get(1)).asDoubles();
double[] d3 = ((REXPVector) ((RList) tableRead).get(2)).asDoubles();
double[] d4 = ((REXPVector) ((RList) tableRead).get(3)).asDoubles();
String[] d5 = ((REXPVector) ((RList) tableRead).get(4)).asStrings();

The error being that JRI.Rlist cannot be converted to REngine.RList.

Do I miss something?

[Mike London]

Piet Souris wrote:I just tried to copy your code line for line into a java class. But I run into problems with this:

String str = "read.csv(textConnection(\"" + inputIris + "\"), header = FALSE)";
System.out.println(str);
List tableRead = re.eval(str).asList();   // re is an instance of Rengine

I get an error "RList cannot be converted to List"
I must change it to

org.rosuda.JRI.RList tableRead = re.eval(str).asList();

before the error disappeared. But then I got an error at these lines:

double[] d1 = ((REXPVector) ((RList) tableRead).get(0)).asDoubles();
double[] d2 = ((REXPVector) ((RList) tableRead).get(1)).asDoubles();
double[] d3 = ((REXPVector) ((RList) tableRead).get(2)).asDoubles();
double[] d4 = ((REXPVector) ((RList) tableRead).get(3)).asDoubles();
String[] d5 = ((REXPVector) ((RList) tableRead).get(4)).asStrings();

The error being that JRI.Rlist cannot be converted to REngine.RList.

Do I miss something?

Getting the data types in Java when reading them from R proved to be too complicated for our project.

Instead, I'm just now reading the R file itself and doing an "eval()" on each line.

The problem I'm now faced with, however, is that the client wants to send the "CSV" data file to R via the web service, and I can't figure out how to put that variable into the R Script context.

I tried this (csvData is the String of CSV that R needs to read):

rConnection.assign("csv", csvData);
rConnection.eval("variableIWantToUseInRScript" <- csv");

But, that variable is not getting set in the R environment from Java.

I also tried (in the second line of code above):

rConnection.eval("variableIWantToUseInRScript" <- read.csv(textConnection(csv), header=TRUE)");

But, nope.

What do you think?

Thanks,

- mike

[Mike London]

ANSWER

I didn't hear back from you this time, but here's how you do it..

It's actually super simple:

1. Use Rserve command: "assign" to assign the csv file to some R data item.

rConnection.assign("csv", theCSVData);

then, realizing the CSV is a table format, use this command:

2. rConnection("variableYouWantVisibleinRScript <- read.csv(textConnection(csv)");

Then, your script should have this csv data in it's address space and can use it.

Thus, you CAN send parameters to an R Script, programmatically, with Rserve.

Rserve is an amazing utility.

--mike

[Piet Souris]

hi Mike,

indeed with re.assign(....) it doesn't work for me too. You are using the instance 'rConsole'. Where do you get this instance from?
If I do 'RConsole rc = ---' I get the error tha 'RConsole' cannot be found.

[Mike London]

Piet Souris wrote:hi Mike,

indeed with re.assign(....) it doesn't work for me too. You are using the instance 'rConsole'. Where do you get this instance from?
If I do 'RConsole rc = ---' I get the error tha 'RConsole' cannot be found.

Hi Piet,

Please re-scan our earlier conversation above about how to create that instance.

More examples here: https://www.rforge.net/Rserve/example.html

Post your code and I'll look at it tomorrow.

Thanks,

-- mike

[Piet Souris]

Hmmm, bad luck here.

First of all: I do not have the package '...REngine.Rserve'. Don't know how come.

I had a look at RForge and I saw Rserve in the jar-file 'RserveEngine.jar' that contained the Rserve package. I downloaded it and added it to my library.
So far so good.

But if I run this code:

package rjava;

import org.rosuda.REngine.Rserve.*;

/**
 *
 * @author Piet
 */
public class Piet {
    RConnection rc;
    
    public static void main(String... args) throws Exception {
        new Piet(args).go();
    }
    
    public Piet(String... args) throws Exception {
        rc = new RConnection();
    }
    
    private void go() throws Exception {
        String f = "C:/Users/Piet/Downloads/xxx.csv";
        rc.assign("f", f);
    }
    
}

I keep getting the error:
Exception in thread "main" org.rosuda.REngine.Rserve.RserveException: Cannot connect: Connection refused: connect
at org.rosuda.REngine.Rserve.RConnection.<init>(RConnection.java:88)
at org.rosuda.REngine.Rserve.RConnection.<init>(RConnection.java:60)
at org.rosuda.REngine.Rserve.RConnection.<init>(RConnection.java:44)
at rjava.Piet.<init>(Piet.java:26)  // that's line 17 in the code above
at rjava.Piet.main(Piet.java:22)
I have R running with the package Rserve.
I have googled for the error, but so far no clear causes.

But my use is not setting variables in R from Java, but for instance to get samples from distributions back in Java, and that part works beautifully.

Piet

[Mike London]

Piet Souris wrote:Hmmm, bad luck here.

First of all: I do not have the package '...REngine.Rserve'. Don't know how come.

I had a look at RForge and I saw Rserve in the jar-file 'RserveEngine.jar' that contained the Rserve package. I downloaded it and added it to my library.
So far so good.

But if I run this code:

package rjava;

import org.rosuda.REngine.Rserve.*;

/**
 *
 * @author Piet
 */
public class Piet {
    RConnection rc;
    
    public static void main(String... args) throws Exception {
        new Piet(args).go();
    }
    
    public Piet(String... args) throws Exception {
        rc = new RConnection();
    }
    
    private void go() throws Exception {
        String f = "C:/Users/Piet/Downloads/xxx.csv";
        rc.assign("f", f);
    }
    
}

I keep getting the error:
Exception in thread "main" org.rosuda.REngine.Rserve.RserveException: Cannot connect: Connection refused: connect
at org.rosuda.REngine.Rserve.RConnection.<init>(RConnection.java:88)
at org.rosuda.REngine.Rserve.RConnection.<init>(RConnection.java:60)
at org.rosuda.REngine.Rserve.RConnection.<init>(RConnection.java:44)
at rjava.Piet.<init>(Piet.java:26)  // that's line 17 in the code above
at rjava.Piet.main(Piet.java:22)
I have R running with the package Rserve.
I have googled for the error, but so far no clear causes.

But my use is not setting variables in R from Java, but for instance to get samples from distributions back in Java, and that part works beautifully.

Piet

It sounds like you don't have the "Rserve" library installed in R?

Could that be the problem?

- mike

[Piet Souris]

hi Mike,

It could be, but I really don't know.

Today I completely removed R-3.1.2 and RStudio. I downloaded it all again, installed it, and now when I run this piece of code from the 'rTest' program:

public static void main(String[] args) {
    // just making sure we have the right version of everything
    if (!Rengine.versionCheck()) {
        System.err.println("** Version mismatch - Java files don't match library version.");
        System.exit(1);
    }
    System.out.println("Creating Rengine (with arguments)");
    // 1) we pass the arguments from the command line
    // 2) we won't use the main loop at first, we'll start it later
    //    (that's the "false" as second argument)
    // 3) the callbacks are implemented by the TextConsole class above
    System.out.println("args when calling Rengine:");
    System.out.println(Arrays.toString(args));
    System.out.println("**********************************************");
    System.out.println("**********************************************");
    Rengine re = new Rengine(args, false, new TextConsole());
    System.out.println("Rengine created, waiting for R");
    // the engine creates R is a new thread, so we should wait until it's ready
    if (!re.waitForR()) {
        System.out.println("Cannot load R");
        return;
    }

    re.eval("require(biotools)");

I get this result:

run:
just loaded library 'jri'
**********************************************

Creating Rengine (with arguments)
args when calling Rengine:
[]
**********************************************
**********************************************
Warning message:
In local({ : bytecode version mismatch; using eval
Error in nchar(.Library.site) : 
  4 arguments passed to .Internal(nchar) which requires 3
Error in .Internal(getOption(x)) : 
  there is no .Internal function 'getOption'

R version 3.1.2 (2014-10-31) -- "Pumpkin Helmet"
Copyright (C) 2014 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)  // and a lot more

It sounds like there might be a 32 bit-64 bit problem somewhere, still trying finding the problem.

But this last report, about R version 3.1.2, is the limit: where on earth is thát coming from?

As soon as I make some progress in this horror story, I let you know.

[Mike London]

Wow, those errors looks nasty.

I had lots of little problems initially getting things to work.

Hang in there...post back...

- mike