Too many opne files

[BalaiahRaju ChamarthiRaju]

Hi,

I am getting "Too many files open " error every 36 hours time span, once i restarted daemons then it will work upto 36 hours.. after the same,..


my code :
@Override
protected void execute() {
DaemonsDao daemonsDao=null;
try{
logger.fine("#### Start of execute()");
daemonsDao=DataFactoryItf.INSTANCE.createDaemonsDao(app);
//Collect all active users for which import needs to be done.
ActiveUsers users = new ActiveUsers();
users=setStartEmp(users,dmn);
logger.fine("#### After setting the starting employee to start");
DmnGsyncdDao dmnGsyncdDao =DataFactoryItf.INSTANCE.createDmnGsyncdDao(app);
logger.fine("#### After creating dmn DAO for Gsyncd");
/* Run the init and pre PL/SQL scripts for the gsycnd cycle. */
//Fix team positions.
daemonsDao.UpdateGacTeamPos(app);
logger.fine("#### After calling UpdateGacTeamPos()");
//Run the other scripts.
dmnGsyncdDao.executeInitAndPre(app);
logger.fine("#### After calling executeInitAndPre()");
/* The old daemon ran gdatatrf at this point, this should now become part of
* the processes in the processTable.
*/
List<ProcessDto> procs=daemonsDao.getProcessesInfo(dmn.getName(), app);
logger.fine("After getting the list of processes which are part of Gsyncd cycle. Processes count:"+ procs.size());
label_users:for(User user:users){
logger.fine("--*** "+user+" ***--");
logger.fine("#### dmn.isBeforeEndTime():"+ dmn.isBeforeEndTime());
for(int i=0;i<procs.size()&&!isCanceled()&&dmn.isBeforeEndTime();i++){
ActiveUsers activeUser=new ActiveUsers();
activeUser.FillForOneEmp(app, user.getEmpId());
MakeDbFlags flag=activeUser.next().getMakeDbFlag();
if(flag==MakeDbFlags.MAKEDB_BUSSY || flag==MakeDbFlags.FULL_MAKEDB_REQUIRED) continue label_users;
logger.fine("#### After verifying whether MakeDB is running OR Full MakeDB is required for this user");
activeUser.clear();
if(!logStep(procs.get(i))){
logger.fine("#### Before verifying the status of other daemons");
yield(user);
logger.fine("#### After verifying the status of other daemons");
Step step=new Step(procs.get(i),dmn,activeUser, app);
logger.fine("--*** "+procs.get(i).getPath()+" ***--");
app.logProcStat("sync",user.getJobId(), "Employee name: "+user.getEmpName()+", EmpId = " +user.getEmpId(),
users.getUsers().indexOf(user), users.getUsers().size());
step.run();
}
}
daemonsDao.changeDmnInfoText(dmn, Integer.toString(user.getJobId()),app);
//Added the below code to reduce the load of the system
Thread.sleep(_SLEEP_TIME_BETWEEN * 1000);
//End of below changes
}

} catch (SQLException e) {
app.logProcError(e,"global.format","Database failure on performing GSYNCD step.");
app.setStop(true);
} catch (SecurityException e) {
app.logProcError(e,"global.format","SECURITY failure on performing GSYNCD step.");
app.setStop(true);
} catch (NoSuchMethodException e) {
app.logProcError(e,"global.format","Illigal method failure on performing GSYNCD step.");
app.setStop(true);
} catch (ClassNotFoundException e) {
app.logProcError(e,"global.format","Class not found failure on performing GSYNCD step.");
app.setStop(true);
}catch(InterruptedException e){//Added by HCL on 16-Jan-08
app.logProcError(e, "global.format", "Class not found failure on performing GSYNCD step.");
app.setStop(true);
}finally{
}
}


Errors I am getting :
Error1:
!/gsys/stage001/GENNOR/temp/lastuser.tmp (Too many open files (errno:24))!
java.io.FileNotFoundException: /gsys/stage001/GENNOR/temp/lastuser.tmp (Too many open files (errno:24))
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
at com.merck.genesys.backoffice.model.bo.GeneSysFile.initReader(GeneSysFile.java:260)
at com.merck.genesys.backoffice.model.bo.GeneSysFile.readLine(GeneSysFile.java:162)
at com.merck.genesys.backoffice.applic.gdatatrf.GDataTrfApp.readLastUser(GDataTrfApp.java:1554)
at com.merck.genesys.backoffice.applic.gdatatrf.GDataTrfApp.readLastProcessedUser(GDataTrfApp.java:1519)
at com.merck.genesys.backoffice.applic.gdatatrf.GDataTrfApp.run(GDataTrfApp.java:206)
at com.merck.genesys.backoffice.applic.dmn.Step.run(Step.java:88)
at com.merck.genesys.backoffice.applic.dmn.GsyncdTimerTask.execute(GsyncdTimerTask.java:81)
at com.merck.genesys.backoffice.model.bo.GeneSysTimerTask.run(GeneSysTimerTask.java:110)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)


please anyone check the code and help me out..



Error2:

!Failed to make the connection to server: FTPGENNOR!
java.net.SocketException: Failed to make the connection to server: FTPGENNOR
at com.merck.genesys.backoffice.applic.gdatatrf.GDataTrfApp.connectFtp(GDataTrfApp.java:407)
at com.merck.genesys.backoffice.applic.gdatatrf.GDataTrfApp.iterateThroughUserList(GDataTrfApp.java:282)
at com.merck.genesys.backoffice.applic.gdatatrf.GDataTrfApp.run(GDataTrfApp.java:211)
at com.merck.genesys.backoffice.applic.dmn.Step.run(Step.java:88)
at com.merck.genesys.backoffice.applic.dmn.GsyncdTimerTask.execute(GsyncdTimerTask.java:81)
at com.merck.genesys.backoffice.model.bo.GeneSysTimerTask.run(GeneSysTimerTask.java:110)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)


please help me out..
thanks in advance..

Balaiah

[Rob Spoor]

Do you close() your FileInputStreams when you're done with them? Probably not.

[Joe Ess]

That code is all very good, but you leave out a lot of details. You share your stack trace but you don't tell us what class we are looking at, which line the exception occurs or show us the code where the exception actually takes place (from the stack trace I presume we are looking at GsyncdTimerTask, but the exception is taking originating in GeneSysFile).
Have a look at our FAQ, How To Ask Questions On JavaRanch. It has very helpful advice like Tell the Details and Use Code Tags

[BalaiahRaju ChamarthiRaju]

Hi Joe Ess,
Thank you very much..

please tell me one thing ..
whne I need to call flush() method..
is it before or after close() in my bufferedwriter..

which can give me good results

Thanks in advance

Bala

[Joe Ess]

If you look at the Java API documentation, you'll see that BufferedWriter.close() follows the contract specified by Writer.close() which flushes the stream before closing. A separate call to flush() is unnecessary.

[Joe Ess]

I forgot to mention a great tool for tracking down these problems. If you are on a Unix variant, you can use lsof to display the open files for a process. This will let you know exactly what resources are not getting closed properly. Here is a good introduction to using lsof.
You can do the same thing on Windows with Process Explorer.

[BalaiahRaju ChamarthiRaju]

hi,

the above programe is running continuously, In my code We are calling close() method after completion of file append and write things. the process is continuously running and doing file open and close operations continuously. after cetain time of period it is throwing error "Too many open files".

as per above suggestion I tried lsof command in Unix for running processes.. but it is restricted to use this command in my unix box. I dint have sufficient previleges to run the lsof command..

why the opened files are not closing immediatly once we are calling the close() method.. is there any alternative in java..
please suggest and give me some java sample codes for this..

Thans in advance
Bala

[Joe Ess]

Get your system administrator to run lsof.
As I mentioned in your previous post, you must close ALL file-like resources with an explicit close(). This includes sockets, database connections, pipes and so on. Since you STILL haven't told us what your code does or shown us a reasonable sample that can replicate the symptoms you are seeing, it is REALLY hard to speculate as to where you are not closing resources. However, since you are seeing a socket error:

    
java.net.SocketException: Failed to make the connection to server: FTPGENNOR

I would hazard a guess that while you are closing files, you may not be closing sockets. Again, lsof is probably your best bet to detecting your problem.

[BalaiahRaju ChamarthiRaju]

HiJoe Ess,

why my application is getting error "Too many files open".
it is occuring every 36 hours approximatly.
in my code level I have gone thru and checked all files,sockets.. everything closed after using. is there any settings in unix box.
please tell me how it is getting Error in a specific time span.

I am not able to look into Unix... the same time not that much familier with unix.

please give me some suggesiong..


Thanks in advance

Bala

[Ulf Dittmer]

I am not able to look into Unix...

That's why Joe suggested to get your system administrator to run lsof, if you can't get at the machine yourself.