• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Running python (.py) script from Java code

 
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello Code Ranchers,

I'm trying to call a python script from within my java program. Jython is not an option for me. I'm running MacOS Catalina 10.15.2 with NetBeans.

I've tried this:

This java compiles and runs fine, but doesn't launch the python. No exceptions. No nothing.

I think this may be a python problem. When I run that same command in terminal:

I get:


I've tried updating python with homebrew. I've also tried pointing to my python directory:

This gives me the same error.

Also - my final java implementation will be on Raspberry Pi (Buster), so I'll need a solution which is platform agnostic. Any feedback on this subject is welcomed.

Thanks,
 
Bartender
Posts: 9626
16
Mac OS X Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Simon McNamara wrote:
This java compiles and runs fine, but doesn't launch the python. No exceptions. No nothing.



If the problem is with the Python script, as it appears to be, you would not get an exception on the Java side.  Errors would be written by Python to Standard Out or Standard Err and you would need to read from those streams in order to capture them (and you should read from them.  They are buffers of limited side, and if they get full, your application stops).  
Have a look at this article for some of the pitfalls with using Runtime.exec().
Have you installed PyQt4?  I do not think it is included with the standard Python distribution.
 
Rancher
Posts: 144
9
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Maybe one could you provide with some useful reply if can tell us what it is you're tryin to run in python and why you run it from Java. Maybe there is an alternate way to achieve your goal with less overhead.
 
Sheriff
Posts: 22783
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Simon McNamara wrote:Jython is not an option for me.


Can you tell us why not?
 
Simon McNamara
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Joe -

I didn't occur to me that this package wasn't included with my distribution. I found on the interwebs ( https://stackoverflow.com/questions/39821177/python-pyqt-on-macos-sierra ) that I can install this with:



Unfortunately, now I get:



I don't know python at all, so now I'm probably in over my head. This is a GNURadio script.

Rob -

A feature of my program is that users can change the GNURadio script (.py) to suit their signal processing needs. My java program then calls a GNURadio script to import data via TCP. Users shouldn't need to edit the java, so embedding python inside the java wouldn't work well for me.

Kristina -

It would have been far wiser to start the entire project in python. Alas, I chose java early on (because it's what I knew), and now I'm too far along to change. The basic workflow is that a user can start the python from within the java GUI. Then, they can select to pipe up to three signals from the GNURadio .py script.

Many thanks for your replies. I am especially interested in solutions that might allow me to send commands to python (I hope to be able to change some variable values in the python on startup - not sure if this is at all possible). The Runtime.getRuntime().exec(command + param ) option might work for this, where "param" is an optional parameter.

 
Marshal
Posts: 28193
95
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Joe Ess pointed you to the classic article about how to deal with Runtime.exec(), which is still a must-read. However since then the ProcessBuilder class has been added to Java, and it simplifies a lot of the annoying details that have to be dealt with. In particular you're going to have to read the process's output stream and its error stream so you can know about what's happening in the process. It's possible to send commands to the process via its input stream, but I'm sure there are still annoying details which have to be dealt with.
 
Simon McNamara
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello Code Ranchers,

For the life of me I cannot get my PyQt4 problem resolved. The shipped version of python for MacOS is 2.7.x . The most helpful thread about getting the PyQt module running is here:

https://stackoverflow.com/questions/36615952/how-can-i-install-pyqt4-for-python-2-7

People seem to have some success installing Python 2.7.x fresh from here:

https://www.python.org/download/releases/2.7/

Then using the PyQt4 binaries package from here:

https://sourceforge.net/projects/pyqtx/ .

However, that PyQt4 package doesn't seem to work with MacOS Catalina (or perhaps other earlier versions which I cannot confirm).

MacPorts and HomeBrew don't seem to have any version of PyQt4 for Python 2.7.x unfortunately.

So I will try the ProcessBuilder approach tomorrow when I have access to my RaspberryPi. I'll report back, in the event that others need this thread in the future. Thanks.
 
Joe Ess
Bartender
Posts: 9626
16
Mac OS X Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Simon McNamara wrote:

For the life of me I cannot get my PyQt4 problem resolved.


According to this, PyQt4 is no longer actively developed or supported.  I wonder if PyQt5 would play better with Catalina.


Simon McNamara wrote:
The shipped version of python for MacOS is 2.7.x .



I have a two-week old install of Catalina and I have Python 3.7.3 as well as 2.7.16.  Open an terminal and enter "python3".  If you haven't installed it, it will prompt you to install the Command Line Developer Tools, which includes Python 3.
 
Simon McNamara
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hey Joe,

I also have python3 since before I updated to Catalina, so I thought maybe I just had that from my prior installation.

One thing I also did try was to point "python absolute/path/to/script.py" to python3, but I either failed to point correctly, or GNURadio simply requires PyQt4 (I received the same error). I would not be surprised if GNURadio requires the older module PyQt4.

How would I find the correct path for python3? The path to python2.7 seems totally different (e.g., something like /usr/lib/python2.7).

Even if pointing to python3 worked for now, establishing the absolute path to python3 would still be a problem when I deploy to RaspberryPi.

Thanks for your help.

 
Joe Ess
Bartender
Posts: 9626
16
Mac OS X Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Simon McNamara wrote:GNURadio simply requires PyQt4 (I received the same error). I would not be surprised if GNURadio requires the older module PyQt4.



Keep in mind that I don't know gnuradio from a hole in the ground but this seems to indicate that gnuradio depends on PyQt5

Simon McNamara wrote:
How would I find the correct path for python3? The path to python2.7 seems totally different (e.g., something like /usr/lib/python2.7).



The *nix command is "which".  On my Mac:


My Raspberry Pi's aren't plugged in just now, but those are the same paths as on the Linux Mint computer I have hooked up to my TV.  It's based on Debian as is Raspbian (if that's what you are using on your Pi).
 
Simon McNamara
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Many thanks for the help. I plan to try some of this on the Pi as soon as my USB-c cable arrives in the mail. I'll check back then.
 
Ranch Hand
Posts: 44
1
Python VI Editor Postgres Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Simon,

Python has a concept of a "virtual environment" for pulling together all the requirements needed for running a nontrivial program.  This is roughly similar to a java jar file being used to hold all the libraries needed for a nontrivial Java program.  Explaining how to set up a Python virtual environment is beyond what can be explained here so please search for "Python virtual environment", "virtualenv", and "Python activate" for that information.

Once the virtual environment is set up and tested, Java will probably need to use a shell script to invoke it.  The script will probably have components that will change directories to the base of the Python project, run the virtual environment's activate command, then run your program with the "python <program.py>" command.  The shell script will also make it easier to pass parameters.

BTW, Python 2 was EOL'ed as of the end of last year.  For reasons known only to Apple, Catalina still uses Python 2.  Thus /usr/bin/python is still version 2.7.  Using Homebrew to install a modern version of python is definitely a good way to get it.  It can install a modern version of (QT5) as well as the Python bindings (pyqt), although since I have not used QT, there may be more needed.

Since this project is targeted to go on a Rasberry PI, it might be a good idea to use Homebrew to install a copy of micropython to use for this project.  That may reveal any code that may have trouble running in the Pi environment.

Apparently Java can be installed on a PI.  However, Python is native on the Pi.  Perhaps it would be worthwhile to take another look at recasting all of the Java code to Python.  It may be much easier than it first looks even preparing the GUI.

There are Python IDE's that will make all this a bit easier, such as Pycharm or Wing.  Both have "community" or "personal" editions that are free.

This may have become a bigger project than you suspected at the beginning, but I hope you will continue this to completion.    
Travis
 
Saloon Keeper
Posts: 27764
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Java is available on the Raspberry Pi, but it's a little slow on the Pi 3. I think it's pre-installed on Raspbian, in fact, since the Minecraft app for Pi was written in Java. Python 2 did not formally go end-of-life until January 2020 and unless the latest Raspbian has changed things, python2 is still the pre-installed and default python on that platform. I was just doing new development and had to manually install python3, in fact.

Python2 is still the default python on most Linux distros, although I think Fedora 31 flipped over to Python 3.

A bigger question however is whether the python part is even needed here. Gnuradio itself is a self-contained engine, and while it has a Python API to allow certain functions, I think you might be able to run a canned flow without it. There's also supposed to be an XMLRPC interface, but it's poorly documented. Very poorly.

A Raspberry Pi can do just about anything a larger PC can do, if not always as fast or with as much RAM. However, while I mostly prefer Java on desktop and server systems, I usually use Python on the Pi. It's the "Visual Basic" for Linux and generally has better interfaces to os-specific functions and my Pi projects are more likely to be dealing with those.
 
Travis Risner
Ranch Hand
Posts: 44
1
Python VI Editor Postgres Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Tim for the clarification of which Python is available on the Raspberry Pi.  Would you provide a link to the instructions you used for installing Python 3?  Rumor has it that CentOS is also converting to Python 3 as well.

it would be great if Simon can run gnuradio directly from Java like he originally tried.  I didn't look deeply enough into gnuradio to find alternate ways of interacting with it.

Yes the Python 2 End-Of-Life announcement by the Python Software Foundation was confusing.  This blog post about it was updated January 8.  The following quote was added.

Clarification Update January 8, 2020
Effective January 1, 2020, no new bug reports, fixes, or changes will be made to Python 2.

After Python 2.7.17 (October 19, 2019) was released, some additional changes accumulated before the end of 2019 when the core development team froze the 2.7 branch. As a final service to the community, python-dev will bundle those fixes -- and only those fixes -- into a final 2.7.18 release. The release date for 2.7.18 will be in April 2020 because that allows time for the release managers to complete a release candidate and final release.



 
Tim Holloway
Saloon Keeper
Posts: 27764
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
To install python3 on any Debian-style system (including Raspbian and Ubuntu), the command is simply "apt-get install python3". Or, I think "apt install" works also for it these days.

Note that the python3 packages are distinct from the python2 packages, so instead of using "pip" to install them, you use "pip3".

CentOS 7 has had python3 support for a long time, even though things like Anaconda are still python 2 (corrected). Here again, you use pip3 to install python packages. And install python3 via the rpm/yum.
 
Simon McNamara
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello all,

Many thanks for your feedback. I got my USB-C cable in the mail, but I over-estimated the current supplied by my monitor's usb ports. I can't seem to power the Pi with any of my wall worts or usb ports on hand. So I'm now waiting on a 3A 5V supply to go with my USB-C cable. Dang.

A bigger question however is whether the python part is even needed here. Gnuradio itself is a self-contained engine, and while it has a Python API to allow certain functions, I think you might be able to run a canned flow without it.



Tim - can you elaborate? I see that my GNURadio script generates a .py . That's what I've been trying to run. Is this a good approach?

 
Tim Holloway
Saloon Keeper
Posts: 27764
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hmmm. I hope someday to be better informed about gnuradio, but I'm still a novice myself.

I was thinking that gnuradio was like most such systems where there would be a central engine running a program. On checking, it appears that gnuradio is really just a set of library modules and the gnuradio-companion is generating scripts to use them.

Going all the way back to your original question, I'll give you what would have been my original answer except that other elements of the discussion made it seem irrelevant.

If you're going to use Runtime.exec() or one of the other external program execution resources, then you should actually use the absolute program path. That is "/usr/bin/python", not just "python". Don't depend on the JVM's shell path. You may also need to intercept the stdio paths for the python interpreter.

If you just want to run a preset gnuradio from Java, that's probably it. If, on the other hand, you're running gnuradio as an indepedent process and hoping to use the Python control API to modify its operation, that would be messier. And you should really think long and hard about how badly you want Java complicating this mix.

Some other possibilities might be to generate the C++ gnuradio flow instead and wrap it as a JNI class, which is also pretty horrible, but might be simpler in some cases.

Or, you could set something up that launches everything, create an MQTT agent in Python to invoke the Python API and use Java MQTT code to push data to the Python agent.
 
Simon McNamara
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

If you're going to use Runtime.exec() or one of the other external program execution resources, then you should actually use the absolute program path. That is "/usr/bin/python", not just "python". Don't depend on the JVM's shell path. You may also need to intercept the stdio paths for the python interpreter.

If you just want to run a preset gnuradio from Java, that's probably it...



This is my hope, Tim.

I fear that this post is straying away from a general question of running python from java, however, and now getting into gnuradio territory.

I tried running the python on the pi. In terminal, I cd to the correct directory, then:



To my surprise, I get the exact same message:



Fortunately, getting PyQt4 is not so difficult for Raspian. I can get the package with:



But now I get this message when I try to run the python:



I found a relevant thread on git. Some users are having the same problem:
https://github.com/ptrkrysik/gr-gsm/issues/355

The respondent's feedback was, in part:

It seems that the gr-gsm-0.41.2-1 package is incompatible with gnuradio-3.7.11-6 and there is no easy way to fix it. The gnuradio package in Debian Buster has been heavily modified and all distributions that are directly or indirectly based on Debian Buster are affected - Ubuntu 18.04, Kali Rolling, Parrot Os, .... This error is not because of missing qt library but because of incompatibility between grgsm_livemon and the latest gnuradio versions from Debian/Ubuntu/Kali repositories. The gnuradio in debian has been migrated from qt4 to qt5 (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=874893) so all QT flowgraphs have to be regenerated.



And to solve this problem, I should comment out some lines in the .py . This just isn't practical and I find it hard to believe that this problem persists a year after the most recent post on that thread. If anyone has some feedback or general guidance on this, that would be great, if not, I may be creating a gnuradio-specific thread next!

Thanks,

 
Tim Holloway
Saloon Keeper
Posts: 27764
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It can be really irritating when you have a neat application that's kind of specialized and external changes break it and no one rushes to fix it. And most of us aren't motivated enough to fix or even hack around it. Although I did make some mods to a PCB layout program not long ago (it helped that it was written in Java ).

There are some possible work-arounds. One of which is to launch a VM that runs a friendlier environment. Another is to do likewise but in a container. You might find a different base OS is sufficient. Or just run an older OS release from before the problem. As a secondary machine, old security exposures are likely to be less of an issue than if the machine was more directly connected to the outside world.
 
reply
    Bookmark Topic Watch Topic
  • New Topic