• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Java Process PID

 
Ranch Hand
Posts: 79
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Hi everybody,

One of our systems is still the AS400 with Websphere 6.0.2 running (JDK 1.4.2).
We are anyway in the process the migrating those apps onto Websphere 8.5.5.

These days the Websphere on AS400 is crashing because of very high % ASP.
I prevent the operating system from crashing by stopping and starting Websphere which means there is some run away thread or deadlock happening.
Creating thread dump or javacore on new Websphere is quite easy but on old Websphere, the only way I found was to use wsadmin which is failing with " java.lang.NoClassDefFoundError: java.lang.NoClassDefFoundError: com/ibm/jvm/Dump". It would be great if anybody could help me on why this is failing?
Maybe the JVM is too old to have anything like javacore dump?

I also thought of going into QSHELL and do the kill -3 <PID> but how do I get the pid for Java process ?
Any help would be highly appreciated.
Thanks again
 
author
Posts: 23951
142
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

prasad agarwal wrote:
I also thought of going into QSHELL and do the kill -3 <PID> but how do I get the pid for Java process ?



For Unix based OSes (Linux, Solaris, etc.), you can use the "ps" command. For Windows, you can use the "tasklist" command.

Henry
 
prasad agarwal
Ranch Hand
Posts: 79
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Henry, Thanks.
I did try that yesterday with " ps -ef | grep java " but unfortunately that did not give any result ?
I gave this command at the root directory. I believe it does not really matter, right?
 
Ranch Hand
Posts: 135
5
Eclipse IDE Postgres Database Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
This would help you.
 
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Um, the most important word here is "AS/400".

The IBM AS/400, now referred to as the iSeries is a small mainframe. It has a unique hardware architecture (the Power series chips are descended from it) and a totally unique OS, unrelated even to other IBM OS's. It's also about the only OS I know of that has its own built-in DBMS.

The QSHELL provides a POSIX-style shell, but the underlying OS is NOT a Unix-style OS, and I don't know how deep the similarities go - originally the iSeries OS was pretty opaque - not even assembly language was available outside of IBM's own internal development groups.

It can be hard to locate the PID even on native Unix-style OS's, since the program name for the process is usually simply "java". Usually I use the "ps -ax" command and grep for server-specific data on the command line (this may require extra "ps" options to avoid a truncated display. For Tomcat, for example, the giveway for me is the word "catalina".

For Websphere, I don't remember, but if you can get QSHELL to display the process command lines and you can't find "websphere" in the command, look for similar terms such as "http".

If that doesn't work, probably have to hit the IBM docs.
 
prasad agarwal
Ranch Hand
Posts: 79
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Thanks Jude, I will try that.

On the other hand, I am trying to find 'core.jar' on the AS400/iSeries machine so that I can do something like 'com.ibm.jvm.Dump.JavaDump()' but I am not finding any as of now. I guess it should be there somewhere?

If anybody knows where I can find that? nothing under Java400 folder ! I also did not find under WebSphere folder either...
We are running - IBM WebSphere Application Server - Express, 6.0.2.11

We are in the process of migrating to WAS 8.5.5. but it might take a couple of months or so.

Thanks again
 
prasad agarwal
Ranch Hand
Posts: 79
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Hi Tim,

Thanks for your response.
I was trying different ways to get the javacore/dump and it appears like the wsadmin/Jacl command worked.
I mean I set the jvm and then ran '$AdminControl invoke $jvm dumpThreads' it worked without throwing any error and took me back to wsadmin command line.
It didn't tell me where the javacore is ! Now my problem is finding where it is... damn !

The below is from my wsadmin and you see after I run the command, it just takes me for entry of next command.

> set jvm [$AdminControl completeObjectName type=JVM,process=server1,*]
WASX7029I: For help, enter: "$Help help"
wsadmin>WebSphere:name=JVM,process=server1,platform=dynamicproxy,node=okjuvju
s,j2eeType=JVM,J2EEServer=server1,version=6.0.2.11,type=JVM,mbeanIdentifier=J
VM,cell=okjuvjus
wsadmin>
> $AdminControl invoke $jvm dumpThreads

wsadmin>

 
prasad agarwal
Ranch Hand
Posts: 79
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

I used the UNIX find (find . -mtime 1) and finally found the Javacore file - javacore.268136.QEJBSVR.SERVER1.20151223104626986.txt
I next supplied this javacore to the IBM Thread and monitor Dump Analyzer and I come across another problem !!

The Dump analyzer says - "Cannot find any thread dumps in javacore.268136.QEJBSVR.SERVER1.20151223104626986.txt"

I don't know why this is happening ?? Any help or suggestion of any other javacore/dump analyzer is appreciated.
Thanks
 
prasad agarwal
Ranch Hand
Posts: 79
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I found the process ID and gave the kill -3 (or QUIT) <pid>.

From documentation, this is supposed to generate the threadDump but it just killed the job.... I mean killed the Websphere server !

I need to quickly go and restart Websphere to get things back to normal.

Is this really supposed to happen this way? or something went wrong?

Thanks for your help. I appreciate
 
Bartender
Posts: 1357
39
IBM DB2 Netbeans IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
As Tim noted, IBM As400 has its own unique architecture and OS, despite the fact that Qshell gives you a taste of a Linux like shell, I would try to analyze the issue using specific AS400 OS tools and commands.Websphere on As400 is quite a different beast with respect to Websphere on other platforms, generally speaking you should read the specific documentation for iSeries version and follow specific features for such platform. Sounds honestly strange that a program or process dies on an AS400 without any OS trace and log - as400 usually is very detailed with logging and trace. You may try to investigate the issue having a look at job logs; when WAS is running, try to monitor it with wrkactjob commands and similar ( I am not an As400 expert, but I used to deal with AS400 as DBMS so I learnt a bunch of basic commands). If I am not wrong, WAS on as400 runs in its own subsystem (QEJB ?) you should ask some As400 guy in your staff to help you in sizing and monitoring it. Sorry, but my knowledge on as400 ends here...
 
prasad agarwal
Ranch Hand
Posts: 79
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Hi Claude, Thanks for your reply.
Yes WAS on as400 runs on its own subsystem (Qwas6 for WAS 6 and Qwas855 for WAS 8.5.5).
I actually got the process ID using the subsystem name like " ps -s Qwas6 " gave me the PID. I thought I was very close in generating javacore or heapdump, but it ended killing the process without any trace as far as I could see.
I quickly restarted WAS 6 and everything was back to normal.

I will still research to see what happened when I get time.... since we are anyway in the process of moving to WAS 8.5.5 on new AS400 which has a very easy way of generating javacore directly from IBM Admin Console.
 
Claude Moore
Bartender
Posts: 1357
39
IBM DB2 Netbeans IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I had a quick look at kill documentation for iSeries googling a bit, not sure if it is updated. It seems that to send a signal to a process (in your case, you want to send the signal '3') you should use 'kill -n 3 PID', not just 'kill -3 PID' . Did you try this syntax ? Maybe there are differences between 'regular' Linux commands and their qshell counterparts.
 
prasad agarwal
Ranch Hand
Posts: 79
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Hi Claude, Thanks again
I never tried 'kill -n 3 PID'. I am definitely going to give it a shot and see if that does the job. I really appreciate your help.
 
Claude Moore
Bartender
Posts: 1357
39
IBM DB2 Netbeans IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hope it helped. Let's know if it works (if you want, of course )
 
prasad agarwal
Ranch Hand
Posts: 79
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Claude, I have not yet tried it.
When the problem occurs again, I will give it a shot.
I believe sometimes I need javacore because of a deadlock or runaway job !

Good thing is ,it has not happened recently.
When it happens, I will try this command.
reply
    Bookmark Topic Watch Topic
  • New Topic