Confusing maven error: Cannot run program

[Mike London]

Hello,

I have a JAR file I had to install manually into my local maven repo. The JAR file works correctly in standalone mode, but not in a Maven project.

Here's the mvn command I used to install it:

mvn install:install-file -Dfile=spssjavaplugin.jar -DgroupId=com.ibm  -DartifactId=spssplugin -Dversion=23 -Dpackaging=jar


Here is the pom.xml referencing that same file (in intellij, the entries are all black (OK).  The code referencing the JAR compiles)
<dependency>
<groupId>com.ibm</groupId>
<artifactId>spssplugin</artifactId>
<version>23</version>
</dependency>

Here's the error at RUNTIME trying to access this JAR in code. I don't understand this error since Intellj should have brought this JAR file into the project, right? Also, since the pom.xml looks fine (text black) and the code all compiles, I'm baffled about this error below.

"java.io.IOException: Cannot run program "/Users/mike/.m2/repository/com/ibm/spssplugi/MacOS/StartXD.sh": error=2, No such file or directory

---

Thanks very much in advance for suggestions how to get around this issue.

- mike

[Knute Snortum]

I'm confused.  What's the command you issued that caused the error?

[Mike London]

Knute Snortum wrote:I'm confused.  What's the command you issued that caused the error?

The line of code was StatsUtil.start();

That's a static method in the JAR file.

In the Java code inside the maven project, all the code compiles fine with no error.

Yet, when the first line of code that references that JAR file runs, I get the "error=2, No such file or directory" as indicated above.

The Maven directory itself looks correct also.

Would appreciate any ideas.

Thanks,

- mike

[Paul Clapham]

That looks like an error message saying that it can't find a certain file. But you're talking about a JAR file in a way that makes me think that there's something inside the JAR which is supposed to be used. And things inside JAR files are resources, not files, so perhaps the code throwing the error message is looking for a file and not a resource?

[Mike London]

Paul Clapham wrote:That looks like an error message saying that it can't find a certain file. But you're talking about a JAR file in a way that makes me think that there's something inside the JAR which is supposed to be used. And things inside JAR files are resources, not files, so perhaps the code throwing the error message is looking for a file and not a resource?

What's strange is that even the path in the error message doesn't match what's in the repo.

Could this be some kind of Maven bug?

This particular JAR file isn't available in a Maven repo that I could find so I had to first (sigh) manually add it to my local repo and then reference it. Previous suggestions about using other methods to access the local JAR file in Maven don't work as that local JAR file isn't in the final output JAR. Thus, the only way to add a local JAR that I've found that actually (not in this case, sadly so far) works is to first add the JAR to your local repo, then reference that JAR as a dependency in your pom.xml.

The error has this: ".m2/repository/com/ibm/spssplugi"

But the actual path (see attached screenshot) is this: .m2/repository/com/ibm/spssplugin/23/spssplugin-23.jar

In the "spss-maven-jar.png" below, I have the actual JAR contents showing the spss plug-in in the JAR file.

---

This only appears to be a Maven problem. If I build a regular Java application and include the spss jar file, then it works fine.

The dependency in Maven in Intellij looks correct (all black, no "red" errors).

In Intellij, the code compiles and builds.

In Intellij Project Manager, under Libraries, the "Classes" path to the maven JAR file is correct.

---

What do you do about these maddening configuration problems?

I've now spent an entire day on this and it's just .... configuration.

I really appreciate your help and hopefully, this extra info clarifies what you were asking.

Look forward to hearing back.

Thanks!!!

- mike

[Knute Snortum]

Is this an exact copy of (part of) the error message?

java.io.IOException: Cannot run program "/Users/mike/.m2/repository/com/ibm/spssplugi/MacOS/StartXD.sh": error=2, No such file or directory  

Notice that it references "spssplugi", not spssplugin.  Try searching your POM to see if "spssplugi" is anywhere and add the n.

[Mike London]

Knute Snortum wrote:Is this an exact copy of (part of) the error message?

java.io.IOException: Cannot run program "/Users/mike/.m2/repository/com/ibm/spssplugi/MacOS/StartXD.sh": error=2, No such file or directory  

Notice that it references "spssplugi", not spssplugin.  Try searching your POM to see if "spssplugi" is anywhere and add the n.

Right, I pointed that error turncation problem out in the postings above.

Since I left the last message, I've created two simple maven projects (non-proprietary) which have the exact same problem.

By limiting the version number to just "1.0", I now don't get the "i" in "spssjavaplugin" chopped off, but I'm still getting the same basic error message:

java.io.IOException: Cannot run program "/Users/mike/.m2/repository/com/ibm/statistics/spssjavaplugin/MacOS/StartXD.sh": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at com.ibm.statistics.plugin.XDAPI.start(Unknown Source)
at com.ibm.statistics.plugin.StatsUtil.start(Unknown Source)
at spsssimple.Test.main(Test.java:10)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
... 3 more

I have no idea why the error appends "/MacOS/StartXD.sh" to the error string. That is definitely NOT part of the path to the JAR File.

So, in summary (now in two IDEs and three project tests):

1. Code all complies in IDE, no errors. Imports all resolve OK.

2. Maven pom.xml file happy. That is, the dependencies are black. No red missing or other problems.

3. Generated JAR file has SPSS jar file included.

4. Paths referenced check OK per screenshots in messages above.

5. Standalone (non-maven) project using the same JAR file works fine.

-----

I could send you the test project to an email account there at JR, but you'd need to have SPSS installed (at least the free 14-day eval version).

I have to get this working so if I need to pay someone to help with this issue I will definitely consider that.

Thanks,

- mike

P.S. I've attached the sample code in the IDE simple test project I just created (the code comes from the IBM site), which compiles fine as you can see, but throws the error indicated above at runtime due to this confusing maven issue.   (Note: This code works perfectly in a standalone Java program where the lib is imported into the local project)

[Knute Snortum]

[Edit: Mike London's post where he does many of the things I describe below posted while I was writing this response.]

Here's a note about how to ask questions so that you get a good answer.  Describe the problem fully.  Describe exactly what you were doing when you got the error.  Copy and paste the full error message or stack trace.

So in this case, I'm guessing this is what happened:

* You put a 3rd party jar into the Maven local directory
* You build the project with the command mvn install or the like (exact command unknown)
* You execute the application from IntelliJ
* While running the application you encounter an error (stack trace unknown)
* You build the project some other way (unknown)
* When you execute the application, you don't get an error

Is that about right?  The problem is, I have to guess at a lot of what you did.  I'm not even sure I'm describing the problem correctly -- but that's your job.  I shouldn't have to guess.

Also (if the above is correct) your conclusion is that there's a bug in Maven.  But isn't it more likely that there's a problem with your POM?

So if you can't find any problem in your POM, go through my starred assumptions, filling in the unknown information and correcting what's wrong.  This is what we call TellTheDetails (that's a link).

[Mike London]

Knute Snortum wrote:Here's a note about how to ask questions so that you get a good answer.  Describe the problem fully.  Describe exactly what you were doing when you got the error.  Copy and paste the full error message or stack trace.

So in this case, I'm guessing this is what happened:

* You put a 3rd party jar into the Maven local directory
* You build the project with the command mvn install or the like (exact command unknown)
* You execute the application from IntelliJ
* While running the application you encounter an error (stack trace unknown)
* You build the project some other way (unknown)
* When you execute the application, you don't get an error

Is that about right?  The problem is, I have to guess at a lot of what you did.  I'm not even sure I'm describing the problem correctly -- but that's your job.  I shouldn't have to guess.

Also (if the above is correct) your conclusion is that there's a bug in Maven.  But isn't it more likely that there's a problem with your POM?

So if you can't find any problem in your POM, go through my starred assumptions, filling in the unknown information and correcting what's wrong.  This is what we call TellTheDetails (that's a link).

Sorry, I thought I was pretty thorough.

OK, here goes with your starred items:

* You put a 3rd party jar into the Maven local directory

Yes, as I indicated above, that's what I did. I also included above the exact maven install command I used.

* You build the project with the command mvn install or the like (exact command unknown)

For the SpringBoot version, yes, Maven install. However, for the separate test project(s), which has the exact same problem, as you can see with the screen-shot I posted above, this was a non SpringBoot example with a simple Java main method.

* You execute the application from IntelliJ

Yes, but also tested with MyExclipse and using the output JAR generated from Maven "Install" goal.

* While running the application you encounter an error (stack trace unknown)

I included the stack trace above in my last posting for the test project I created.

java.io.IOException: Cannot run program "/Users/mike/.m2/repository/com/ibm/statistics/spssjavaplugin/MacOS/StartXD.sh": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at com.ibm.statistics.plugin.XDAPI.start(Unknown Source)
at com.ibm.statistics.plugin.StatsUtil.start(Unknown Source)
at spsssimple.Test.main(Test.java:10)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
... 3 more


* You build the project some other way (unknown)

As mentioned in postings problem happens with simple Java application. And, in the separate production project using SpringBoot with built jars.

* When you execute the application, you don't get an error

In the main method of the sample project (screenshot in last posting), the error happens on the first line of code that references the JAR file (StatusUtil.start()).

If I run the separate SpringBoot project, then the project loads and then crashes when the incoming URL hits that same line of code.

So, really the same behavior.

---

Sorry if my previous postings were unclear.

Look forward to any ideas you might have.

Thanks again.

- mike

[Knute Snortum]

Thanks for your post with a more detailed explanation.  If you are pretty sure it's not a POM configuration problem, tell me what you do exactly to build the project some other way.

Another note: Please UseCodeTags (that's a link) when you post code; don't use screenshots.  It's a lot easier to copy the code if needed when it's pasted in with code tags surrounding it.

[Mike London]

Knute Snortum wrote:Thanks for your post with a more detailed explanation.  If you are pretty sure it's not a POM configuration problem, tell me what you do exactly to build the project some other way.

Another note: Please UseCodeTags (that's a link) when you post code; don't use screenshots.  It's a lot easier to copy the code if needed when it's pasted in with code tags surrounding it.

Steps:

1. I'm creating a maven project.

2. I add the dependency to the spss JAR file added to the local repo (no issues showing in pom file).

3. Create a simple main method which runs code that references the classes in the JAR file.

4. See error crash stacktrace shown above.


------

But, you raise an excellent point. I'm assuming the POM is OK since nothing is showing up red (will go red if I change anything), but can you tell me if this looks correct?

Here's the POM file from my simple test project:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>com.testmavensimple</groupId>
<artifactId>testsimple</artifactId>
<version>1.0-SNAPSHOT</version>

<properties>
   <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
   <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
   <java.version>1.8</java.version>
</properties>

<dependencies>

   <dependency>
       <groupId>com.ibm.statistics</groupId>
       <artifactId>spssjavaplugin</artifactId>
       <version>1.0</version>
   </dependency>

</dependencies>

</project>

-------

And I've attached a screenshot of the repository (beneath "com").

Do the Maven definition and attached screenshot look correct?

That could very well be the issue.

Thanks,

- mike

[Knute Snortum]

Mike London wrote:Sorry, I thought I was pretty thorough.

You were.  I was writing my post when you posted yours, so I hadn't seen it

* You put a 3rd party jar into the Maven local directory

Yes, as I indicated above, that's what I did. I also included above the exact maven install command I used.

Thank you.

* You build the project with the command mvn install or the like (exact command unknown)
For the SpringBoot version, yes, Maven install. However, for the separate test project(s), which has the exact same problem, as you can see with the screen-shot I posted above, this was a non SpringBoot example with a simple Java main method.

This is the first I've heard of SpringBoot.  So this is a Spring application?  My understanding is that SpringBoot helps configure a Spring app, correct?  So you execute SimpleTest without Spring and you still get the error?  I don't think it's Maven that's the problem.

* You execute the application from IntelliJ

Yes, but also tested with MyExclipse and using the output JAR generated from Maven "Install" goal.

Thanks.

* While running the application you encounter an error (stack trace unknown)

I included the stack trace above in my last posting for the test project I created.

Yes you did, thanks.  Your post posted while I was writing my reply.  We're playing "post tag."

* You build the project some other way (unknown)

As mentioned in postings problem happens with simple Java application. And, in the separate production project using SpringBoot with built jars.

Okay, then I don't think it's a Maven problem -- although if all projects used the same POM, maybe so.  Why don't you post your POM, surrounded with XML tags.

* When you execute the application, you don't get an error

In the main method of the sample project (screenshot in last posting), the error happens on the first line of code that references the JAR file (StatusUtil.start()).

If I run the separate SpringBoot project, then the project loads and then crashes when the incoming URL hits that same line of code.

So, really the same behavior.

Do they all use the same POM?  

[Mike London]

I posted the complete pom.xml above for the sample standalone project and, in the screenshot, the entire main method code, too.

In my last posting, I also included a screenshot for what my maven repo looks like relative to the pom.xml file.

-------

IAC, the basic issue is that when I use a maven project with the pom.xml shown above (simple Java test SPSS application, NOT SpringBoot in this case), it does not work. It throws the exceptions noted above in several postings.

If I instead import the spss jar file into a standalone (separate) Java project (no maven), then it works fine.  Hurray >>> Output!!

---

OK, as you said, the problem might not be maven, but then what could it be? Maven is the only functional difference I can see.

** Same code with either standalone Java application (no maven) or with simple maven project with main method.

** No errors in code in either case.

** One works (standalone Java, no maven), one doesn't (standalone Java, with maven).

---

As I said, I can send you the test project, but you'd have to install the SPSS JAR yourself to see if you can re-create the issue.

I'm totally baffled.

What do you think I should try next?

Thanks,

- mike

[Knute Snortum]

I'm pretty baffled too.

If I instead import the spss jar file into a standalone (separate) Java project (no maven), then it works fine.  

Could you expand on this?  I'm very interested in a project that works correctly.  What exactly do you mean by "import the ssps jar"?

I may need to try this out myself (downloading the SPSS jar and running the simple program) to get a better idea of what's going on.

[Mike London]

Knute Snortum wrote:I'm pretty baffled too.

If I instead import the spss jar file into a standalone (separate) Java project (no maven), then it works fine.  

Could you expand on this?  I'm very interested in a project that works correctly.  What exactly do you mean by "import the ssps jar"?

Sure, all I meant was that instead of a maven project, using a "regular" Java application, I just (in Intellij) add the library from "Java" (local path) to the project. So, no pom.xml in this case.

Knute Snortum wrote:I may need to try this out myself (downloading the SPSS jar and running the simple program) to get a better idea of what's going on.

You'll actually need to download the SPSS Trial from the IBM site (what I did), which installs the JAR as part of the install. It's free for 14 days.

That would be great if you could help. The source code for the IBM sample program I posted above is here:

import com.ibm.statistics.plugin.*;

public class demo {public static void main(String[] args) {

   try {
        StatsUtil.start();
        String[] command={"OMS",
                "/DESTINATION FORMAT=HTML OUTFILE='/output/demo.html'.",
                "DATA LIST FREE /salary (F).",
                "BEGIN DATA",
                "21450",
                "30000",
                "57000",
                "END DATA.",
                "DESCRIPTIVES salary.",
                "OMSEND."};
       StatsUtil.submit(command);
       StatsUtil.stop();
   } catch (StatsException e) {
       e.printStackTrace();
   }
}

}

IBM link for Java code for SPSS example

I can send you my Intellij project, too, but if you add the "spssjavaplugin.jar" file to your own maven repo, it might be better not to do anything I've done thus far given my issues.

If you search your SPSS installation directory, you can find the spssjavaplugin.jar file so you can add it to your maven repo.

Let me know what I can do to help clarify, OK?

Really appreciate all your replies and offers to help.

Thanks very much!!

- mike

[Dave Tolls]

In the installation directory for SPSS does it have that sh script?

I have seen other errors like this in a simple google, but for the exe (which I presume is the WIndows installation), for example here.
Reading that and it seems that it needs to be linked to in situ...as in, the classpath points to the jar file in the installation directory.
Now, I can only assume that means that the missing file is actually somewhere in that area, and so won't actually be available if you stick it in Maven.

If that's the case then this isn't a Maven issue, but an IBM one.

[Mike London]

Dave Tolls wrote:In the installation directory for SPSS does it have that sh script?

I have seen other errors like this in a simple google, but for the exe (which I presume is the WIndows installation), for example here.
Reading that and it seems that it needs to be linked to in situ...as in, the classpath points to the jar file in the installation directory.
Now, I can only assume that means that the missing file is actually somewhere in that area, and so won't actually be available if you stick it in Maven.

If that's the case then this isn't a Maven issue, but an IBM one.

As I've mentioned above in several replies, adding the IBM spssjavaplugin.jar to a regular Java project (non maven) works fine. You get output. No errors. Where the IBM SPSS installer puts the JAR file depends on the OS you're using (see the IBM documentation on this). Regardless, you just need to find that JAR and add it to (depending on project type) your project libs or your maven repo.

But, in a maven project, even though the pom.xml looks fine (no red error XML text, etc.) and the code all compiles (code complete also works for methods inside the JAR!), you get the runtime error I posted above. Thus, the JAR file is clearly not "missing" since everything compiles and builds.

If you double check my posting above, you'll see this is a Mac installation -- see the confusing "MacOS" that's getting appended to the FileNotFound Exception.

So, everything is the same in all my test projects except for maven vs. not maven. Now, agreed, it may be another issue that's causing the runtime error. I have no idea or any way to proceed at this moment.

I did lots of searches, too, but nothing remotely was helpful.

I'm hopeful that @Knute will install the SPSS trial and confirm my findings or figure out what's going on.

Thanks for your reply.

- mike

[Dave Tolls]

I'm not saying the jar file is missing.
I'm saying that, from that IBM forum thread, you should be pointing your classpath at the jar file in the installation directory, because it looks like it uses that location to find the relative path to this shell script.

How are you adding the jar file to the regular Java project?
If you are using the IDE to point at the jar file in the installation directory then that would explain why it works.

But have you actually looked for that file?

[Mike London]

Dave Tolls wrote:I'm not saying the jar file is missing.
I'm saying that, from that IBM forum thread, you should be pointing your classpath at the jar file in the installation directory, because it looks like it uses that location to find the relative path to this shell script.

How are you adding the jar file to the regular Java project?
If you are using the IDE to point at the jar file in the installation directory then that would explain why it works.

But have you actually looked for that file?

I think you figured out the issue. Though I don't know how to solve it yet.

YES. That's it!!! The reason the standalone Java version worked was because I added the JAR file from the IBM installation directory without moving it.

And, I now believe that the reason the maven version does not work is that that referenced JAR is now in the maven repo and not being referenced in the IBM install folder.

Is there a way to solve this "JAR must be in the installation directory" if you're using Maven?

Thanks very much for your posting. I think you nailed it, the problem anyway.

Look forward to hearing back how I might get around this directory requirement and still be able to use maven.

Thanks!!!

- mike

P.S. Using this:

   <dependency>
       <groupId>com.ibm.statistics</groupId>
       <artifactId>spssjavaplugin</artifactId>
       <version>1.0</version>
       <scope>system</scope>
       <systemPath>/Applications/IBMSPSSStatistics/Subscription/SPSSStatisticsSubscription.app/Contents/bin/spssjavaplugin.jar</systemPath>
   </dependency>
   
DOES WORK FOR STANDALONE WITHIN THE IDE ITSELF

>>> BUT <<<

The resulting JAR file does not have the JAR file included.

[Mike London]

I'm now wondering how Maven can work for this project given that the SPSS JAR file MUST be left in its directory, not included in a local maven repo.

For example, I tried this in the pom.xml file:

       <properties>
    <spss.lib.folder.macserver>/Applications/IBMSPSSStatistics/Subscription/SPSSStatisticsSubscription.app/Contents/bin/spssjavaplugin.jar</spss.lib.folder.macserver>
</properties>

<dependency>
<groupId>com.ibm.statistics</groupId>
<artifactId>spssjavaplugin</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>${spss.lib.folder.macserver}</systemPath>
</dependency>


And, YES, this works, but ONLY FOR IDE use.

Once I create the JAR and try to run it, I get nothing but compile errors where maven can't resolve the class files in the SPSS JAR file.

---

What's the solution to using Maven (with a build JAR file) but leaving some JARs alone in their directories?

Thanks in advance.

- mike

[Knute Snortum]

I came to the same conclusion as Dave Tolls: you can't move the spssplugin.jar file from the IBM installation directory.  This link talks about how to distribute an app using spssplugin.jar.  It sounds like you need the end user site to have IBM SPSS installed and then you use either an ini file or a utility to locate the installation.

[Mike London]

Knute Snortum wrote:I came to the same conclusion as Dave Tolls: you can't move the spssplugin.jar file from the IBM installation directory.  This link talks about how to distribute an app using spssplugin.jar.  It sounds like you need the end user site to have IBM SPSS installed and then you use either an ini file or a utility to locate the installation.

Yes, the end-user would have SPSS installed on the server. That's a given.

This guide is fine, and I've seen it, but but would do I use Maven with an "ini" file?

As I posted above, <systemPath> does not work when building the JAR file.

This configuration seems nightmarish.

Thanks,

[Knute Snortum]

My take on the web page is that you can't use Maven to find the spssplugin.jar, unless you used the bat file and rewrote the POM -- but don't do that.  Programmatically load the jar file using the utilities mentioned in the web page.  My guess is that it would go like this:

* Download the SPSS developer utilities here
* Use those utilities in your app to get the location of the local IBM SPSS installation
* Use the Java Class class to load the jar file: Class.forClass("location/of/spssplugin.jar");

There may be another way to programmatically load an external jar file in Java, but Class.forName() is the one I know.

[Mike London]

Knute Snortum wrote:My take on the web page is that you can't use Maven to find the spssplugin.jar, unless you used the bat file and rewrote the POM -- but don't do that.  Programmatically load the jar file using the utilities mentioned in the web page.  My guess is that it would go like this:

* Download the SPSS developer utilities here
* Use those utilities in your app to get the location of the local IBM SPSS installation
* Use the Java Class class to load the jar file: Class.forClass("location/of/spssplugin.jar");

There may be another way to programmatically load an external jar file in Java, but Class.forName() is the one I know.

I could just store the known location of the installation in a java properties file and avoid steps 1 and 2 above.

However, if I'm using Maven for the rest of the SpringBoot project, how do I first load the spssplugin.jar file (assuming you meant Class.forName()) so the maven project can resolve dependencies?

I may be missing part of your point.

- mike

[Knute Snortum]

Well, if you don't mind changing the POM for every installation, you can specify a local repository and point it at your local jar file (I think, I haven't tested it).  But this sort of defeats the idea of a POM file, which you should be able to just download, run mvn clean install, and be done with it.

Rather, I'm suggesting (or IBM is) that you use the utilities to find the path to the local installation, then load the jar in Java programatically.  With this approach, Maven doesn't know about the SPSS jar file at all; it's all handled in Java.

I've tested the code below and it works for me.  What I don't know for sure is whether you have do a Class.forName() for every class you need.  If so, then the Maven local repository is the way to go.

package net.snortum.play;

import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;

public class LoadJarPlay {
	
	// Get this string from the IBM utilities
	private String pathToSpssJar = "file:///C:/Program%20Files/IBM/SPSS/Statistics/Subscription/spssjavaplugin.jar";
	
	// Pick any class in spssjavaplugin.jar
	private String spssClassName = "com.ibm.statistics.plugin.Alignment";

	public static void main(String[] args) {
		new LoadJarPlay().run();
	}

	private void run() {
		URL url;
		try {
			url = new URL(pathToSpssJar);
		} catch (MalformedURLException e) {
			e.printStackTrace();
			return;
		}
		URL[] urls = new URL[] { url };
		ClassLoader parent = getClass().getClassLoader();
		URLClassLoader loader = new URLClassLoader(urls, parent);
		try {
			Class.forName(spssClassName, true, loader);
		} catch (ClassNotFoundException e) {
			e.printStackTrace();
			return;
		}
		System.out.println("Success");
	}

}

[Mike London]

Knute Snortum wrote:Well, if you don't mind changing the POM for every installation, you can specify a local repository and point it at your local jar file (I think, I haven't tested it).  But this sort of defeats the idea of a POM file, which you should be able to just download, run mvn clean install, and be done with it.

Rather, I'm suggesting (or IBM is) that you use the utilities to find the path to the local installation, then load the jar in Java programatically.  With this approach, Maven doesn't know about the SPSS jar file at all; it's all handled in Java.

I've tested the code below and it works for me.  What I don't know for sure is whether you have do a Class.forName() for every class you need.  If so, then the Maven local repository is the way to go.

package net.snortum.play;

import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;

public class LoadJarPlay {
	
	// Get this string from the IBM utilities
	private String pathToSpssJar = "file:///C:/Program%20Files/IBM/SPSS/Statistics/Subscription/spssjavaplugin.jar";
	
	// Pick any class in spssjavaplugin.jar
	private String spssClassName = "com.ibm.statistics.plugin.Alignment";

	public static void main(String[] args) {
		new LoadJarPlay().run();
	}

	private void run() {
		URL url;
		try {
			url = new URL(pathToSpssJar);
		} catch (MalformedURLException e) {
			e.printStackTrace();
			return;
		}
		URL[] urls = new URL[] { url };
		ClassLoader parent = getClass().getClassLoader();
		URLClassLoader loader = new URLClassLoader(urls, parent);
		try {
			Class.forName(spssClassName, true, loader);
		} catch (ClassNotFoundException e) {
			e.printStackTrace();
			return;
		}
		System.out.println("Success");
	}

}

Nice. Appreciate these ideas!

Two questions:

1. How would this code actually be used with a Maven SpringBoot project? I'm not sure what the architecture would look like. Not sure how the maven project would do a build when the JAR file isn't present or how this code would integrate with a maven build.

2. I don't think you can have a local repository because then the JAR file would not be in the IBM directory and thus would not work, but perhaps I misunderstood your point above?

Thanks in advance,

[Dave Tolls]

I think this would be a case of having the jar file in the repo for compilation only, so it doesn't get included in any distribution.
At runtime the code (or something similar) that Knute posted would handle loading the actual jar file from the installation directory.

[Liutauras Vilda]

Mike London wrote:...

However, if I'm using Maven for the rest of the SpringBoot project, how do I first load the spssplugin.jar file (assuming you meant Class.forName()) so the maven project can resolve dependencies?

I may be missing part of your point.

- mike

I didn't understand properly, do you have this jar somewhere in repository?

In case yes, can't you use maven assembly plugin to pack this jar into resulting jar? i.e.:
<plugin> <artifactId>maven-assembly-plugin</artifactId> <version>3.0.0</version> <configuration> <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef> </descriptorRefs> </configuration> <executions> <execution> <id>make-assembly</id> <phase>package</phase> <goals> <goal>single</goal> </goals> </execution> </executions> </plugin>

[Dave Tolls]

Can't do that as it appears the IBM code is written to rely on the jar being in the installation directory, as it needs to access some launch script/code from the proper SPSS application.
It would make little sense to include the jar file in the distribution (it may not even be legal to do so) as it doesn't work by itself.

It's a bit like the servlet-api.jar.  You don't deploy it with your web application.

[Mike London]

Good stuff Knute (thanks for that cool code), Dave (thanks for your continued help and insights).

OK, I'm getting there...but still no joy.

In the main part of the SPSS SpringBoot application, I now have this:

public static void main(String[] args)
{
   loadSpssJarInfo();
   SpringApplication.run(SpssdemoApplication.class);
}

.
.
.
Then, below in that class:

    // -----------------------------
	       private static void loadSpssJar()
	       //-----------------------------
	       {
	           // Get this string from the IBM utilities or a properties file...
	           String pathToSpssJar = "file:///Applications/IBMSPSSStatistics/Subscription/SPSSStatisticsSubscription.app/Contents/bin/spssjavaplugin.jar";

	           // Load Needed Classes from SPSS JAR
	           String spssStatsUtilClass = "com.ibm.statistics.plugin.StatsUtil";
	           String spssStatsExceptionClass = "com.ibm.statistics.plugin.StatsException";

	           URL url;
	           try
	           {
	               url = new URL(pathToSpssJar);
	           }
	           catch (MalformedURLException e)
	           {
	               e.printStackTrace();
	               return;
	           }
	           URL[] urls = new URL[]{url};
	           ClassLoader parent = ClassLoader.getSystemClassLoader();
	           URLClassLoader loader = new URLClassLoader(urls, parent);

	           // use the code below if we need to load a particular class within the JAR.
	           try
	           {
	               Class.forName(spssStatsUtilClass, true, loader);
	               Class.forName(spssStatsExceptionClass, true, loader);
	           }
	           catch (ClassNotFoundException e)
	           {
	               log.debug("Class Not Found Exception: " + e.getLocalizedMessage());
	               return;
	           }

	           System.out.println("\nDynamically Loaded SPSS Classes from SPSS JAR...");
	       }

 

This code seems to work OK. I can break and view classes in the Evaluation tool in the debugger before the Spring app starts.

----

To get the code to run at all, I seem to still need this in pom.xml:

<dependency>
<groupId>com.ibm.statistics</groupId>
<artifactId>spssjavaplugin</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>${spss.lib.folder.macserver}</systemPath>
</dependency>


So, everything works, as before in the IDE. No changes there.

-----

But, still, trying to build and run the JAR file gives me this error stack which references StatsException not found even though I dynamically loaded it. Perhaps this is a Spring thing or I don't have the initialization code in the right place?:

  Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
  ""2017-10-11 07:03:32 - Application startup failed
  "org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'mainController': Unsatisfied dependency expressed through field 'service'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'spssService': Failed to introspect bean class [com.spssdemo.service.SpssService] for lookup method metadata: could not find class that it depends on; nested exception is java.lang.NoClassDefFoundError: com/ibm/statistics/plugin/StatsException
  at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:588)
  at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
  at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1264)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
  at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
  at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
  at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
  at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
  at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
  at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
  at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
  at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
  at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
  at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
  at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118)
  at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107)
  at com.spssdemo.SpssdemoApplication.main(SpssdemoApplication.java:22)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
  at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
  at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
  at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
  Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'spssService': Failed to introspect bean class [com.spssdemo.service.SpssService] for lookup method metadata: could not find class that it depends on; nested exception is java.lang.NoClassDefFoundError: com/ibm/statistics/plugin/StatsException
  at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.determineCandidateConstructors(AutowiredAnnotationBeanPostProcessor.java:269)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.determineConstructorsFromBeanPostProcessors(AbstractAutowireCapableBeanFactory.java:1118)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1091)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
  at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
  at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
  at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
  at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
  at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1138)
  at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1066)
  at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:585)
  ... 27 common frames omitted
  Caused by: java.lang.NoClassDefFoundError: com/ibm/statistics/plugin/StatsException
  at java.lang.Class.getDeclaredMethods0(Native Method)
  at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
  at java.lang.Class.getDeclaredMethods(Class.java:1975)
  at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:613)
  at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:524)
  at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:510)
  at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.determineCandidateConstructors(AutowiredAnnotationBeanPostProcessor.java:247)
  ... 39 common frames omitted
  Caused by: java.lang.ClassNotFoundException: com.ibm.statistics.plugin.StatsException
  at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:94)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  ... 46 common frames omitted
 
-----

I think I'm close, but am missing something I need to do.

Suggestions?

Thanks!

- mike

[Dave Tolls]

How do you run it in the IDE?  What is the configuration used?

How do you run the jar file?

ETA:  Also, what do your logs show?
Ensure they are setup to show debug level, and add some more debug logging into the loadSpssJar method.

[Mike London]

Dave Tolls wrote:
How do you run it in the IDE?  What is the configuration used?

Good questions.

The configuration in the iDE, which works is this pom.xml portion below.

Note that I am merely referencing the server SPSS JAR file, not including it in the JAR.

<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.7.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<spss.lib.folder.macserver>/Applications/IBMSPSSStatistics/Subscription/SPSSStatisticsSubscription.app/Contents/bin/spssjavaplugin.jar</spss.lib.folder.macserver>
</properties>

<dependencies>

<!--
Can't do this since you can't have the IBM SPSS JAR locally --> WON'T WORK
<dependency>
<groupId>com.ibm.statistics</groupId>
<artifactId>spssjavaplugin</artifactId>
<version>1.0</version>
</dependency>
-->
<dependency>
<groupId>com.ibm.statistics</groupId>
<artifactId>spssjavaplugin</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>${spss.lib.folder.macserver}</systemPath>
</dependency>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>


<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>

Dave Tolls wrote:[b]How do you run the jar file?

The JAR file is the same project built by the maven install goal.

I then run it at the command line like this:

$java -jar spss.jar

Dave Tolls wrote: ETA:  Also, what do your logs show?
Ensure they are setup to show debug level, and add some more debug logging into the loadSpssJar method.

The logs show the same Exceptions as posted.

As you suggested, I added more logging to the loadSpssJarInfo() method, and, indeed, the classes are being dynamically loaded:

 

  try
        {
            Class.forName(spssStatsUtilClass, true, loader);
            log.debug("Loaded "+ spssStatsUtilClass);
            Class.forName(spssStatsExceptionClass, true, loader);
            log.debug("Loaded "+ spssStatsExceptionClass);
        }

Gives...

07:45:09.999 [main] DEBUG com.spssdemo.SpssdemoApplication - Loaded com.ibm.statistics.plugin.StatsUtil
07:45:10.005 [main] DEBUG com.spssdemo.SpssdemoApplication - Loaded com.ibm.statistics.plugin.StatsException

Dynamically Loaded SPSS Classes from SPSS JAR...

-----

Does this additional info help?

Thanks,

- mike

[Dave Tolls]

Well, when I run code in my IDE I use the run button, so I was checking to see how you were doing it.
What are you selecting in order to run the code?

(Actually, might have missed it, but what IDE is it?)

Also you say you run the jar file, so how do you do that?

(And your quote boxes are a bit out of synch).

[Mike London]

Dave Tolls wrote:Well, when I run code in my IDE I use the run button, so I was checking to see how you were doing it.
What are you selecting in order to run the code?

(Actually, might have missed it, but what IDE is it?)

Also you say you run the jar file, so how do you do that?

(And your quote boxes are a bit out of synch).

LOL.

Sorry, here's a bit better info.

Yes, when I use the IDE for the SpringBoot application, I use the run button. Then Spring starts up in the console window below successfully and is listening for Requests. And, at that point, I can open a terminal on the server box or another computer and do a CURL command to the @RequestMapping in the Controller. It all works fine...there.

-----

After building the JAR file from the Maven install goal, I run it by the Java -jar command, like this:

$java -jar spss.jar

(the error stack happens here after the notice that I successfully loaded the to classes from the IBM SPSS JAR file.)

----

I've tried adding other logging, too, and even extracting out the "loadSpssJarInfo()" method to it's own static method in a separate class and then calling it from different places to see if that would make any difference. Nope. I had thought that perhaps after Spring initialized the application, the environment would get overwritten or something like that.

Knute didn't yet reply on exactly how he envisioned the class loading working, but so far it hasn't based on what I'm doing.

Does this help?

Thanks,

- mike

[Dave Tolls]

OK, so the IDE is launching from the main() definitely.

What do the logs show for the version run from the command line?
What's the MANIFEST for your jar file hold?


Edit: Ah, forget about the logs bit, I just saw that you are seeing the output for loading the classes!

[Mike London]

Dave Tolls wrote:OK, so the IDE is launching from the main() definitely.

What do the logs show for the version run from the command line?
What's the MANIFEST for your jar file hold?


Edit: Ah, forget about the logs bit, I just saw that you are seeing the output for loading the classes!

Forgot to answer your other question. IDE is Intelij IDEA - version: 2017.2.5

Below's the built JAR's Manifest:

Manifest-Version: 1.0
Implementation-Title: spssdemo
Implementation-Version: 0.0.1-SNAPSHOT
Archiver-Version: Plexus Archiver
Built-By: mike
Implementation-Vendor-Id: com.company
Spring-Boot-Version: 1.5.7.RELEASE
Implementation-Vendor: Pivotal Software, Inc.
Main-Class: org.springframework.boot.loader.JarLauncher
Start-Class: com.company.spssdemo.SpssdemoApplication
Spring-Boot-Classes: BOOT-INF/classes/
Spring-Boot-Lib: BOOT-INF/lib/
Created-By: Apache Maven 3.3.9
Build-Jdk: 1.8.0_144
Implementation-URL: http://projects.spring.io/spring-boot/spssdemo/

=====

Thanks,

- mike

[Dave Tolls]

There you go:

Main-Class: org.springframework.boot.loader.JarLauncher
Start-Class: com.company.spssdemo.SpssdemoApplication


I thought that's how SpringBoot works.
It doesn't use your main() method as an entry point, it uses its own (in JarLauncher).
So not sure how the above code can be appearing in your logs?

[Dave Tolls]

Right, I think (and this is only a "think"!) that you might get around this by adding Loader-Path into your MANIFEST, which can contain the path to the jar file you need.
See here (near the bottom).

I think...

[Knute Snortum]

My conclusion from playing with the POM file in Maven is that you can't make a dependency to the IBM SPSS plugin jar because Maven requires a version and the path to the plugin doesn't include a version.  Basically, you can't make SPSS's folder structure fit Maven's requirement.  At least, I didn't find a way.

[Mike London]

Knute Snortum wrote:My conclusion from playing with the POM file in Maven is that you can't make a dependency to the IBM SPSS plugin jar because Maven requires a version and the path to the plugin doesn't include a version.  Basically, you can't make SPSS's folder structure fit Maven's requirement.  At least, I didn't find a way.

So, is the general conclusion here that there's basically no way to do a SpringBoot application with this arbitrary requirement that the IBM SPSS JAR file needs to be in the installation folder?

I'm not sure how to build/create a SpringBoot without Maven and, using Gradle, I'm afraid the same basic issues would be present.

I need to update the client and I don't see any workarounds.

I totally appreciate all the hard work and help here!!

Thanks,

- mike