• Post Reply Bookmark Topic Watch Topic
  • New Topic

FTP Client MultiThread

 
Rodrigo Bossini
Ranch Hand
Posts: 113
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

I'm using the apache FTP lib to build a ftp client application.

Basically, my application accesses N FTP servers simultaneosly, to make downloads and uploads, as indicated in the database. Obviously it's a mulithreaded application.

My problem is that after some time it's running, the netstat command in windows shows me thousands and thousands of ports in TIME_WAIT state.

I have already debugged my application and made sure that all of the threads where calling the disconnect method.

After researching the TIME_WAIT stated, I found out that it was normal.

My application does indeed open lots of connections and closes then, but after that it's normal that all of the connections will hang in TIME-WAIT state for some period.

The problem is that it gets to the limit of the ports available and after some time I can't even use my browser to view a web page, as there are no ports available.

I tried to set the reUsseADDR thing (there's a method for doing this) which should indicate that even ports in TIME-WAIT state should be used for new connections.

However it does not seem to work.

I'm thinking that I should set a standard range of ports that my application could use, and make sure that all of its threads share these ports, and no other ports will be allowed to be used.

Do you guys have any other suggestions?
 
Rodrigo Bossini
Ranch Hand
Posts: 113
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

I've been further checking this issue.

Basically, I took filezilla client to make some tests.

For my surprise, it behaves pretty much the same as my application does, regarding connections in TIME_WAIT state.

As per my tests, it was enough to transfer 180 small(about 2k) files to 4 different remote directories all belonging to the same ftp server to get about 200 ports in TIME_WAIT state.

I guess my question would be more general them.

How does one deal with multiple TCP connections in one single application focusing on avoiding the whole system going into collapse because of lots of TCP ports in TIME_WAIT state?
 
Rudradutt Joshi
Ranch Hand
Posts: 45
Fedora Google Web Toolkit Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Correct me if I am wrong about the way you are using the FTP Connection.
As far as I can guess, you are getting connected to the FTP server, performs transactions,disconnects and then again cycle repeats(with the same Object of FTPClient).

In such a case API for the Client says
Because the handling of sockets on different platforms can differ significantly, the FTPClient automatically issues a new PORT command prior to every transfer requiring that the server connect to the client's data port.


As FTP uses TCP connection to perform the operations, the above statement may create lots of trouble.


CHEERS,
Rudradutt Joshi.
 
Rodrigo Bossini
Ranch Hand
Posts: 113
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
rudradutt joshi wrote:Correct me if I am wrong about the way you are using the FTP Connection.
As far as I can guess, you are getting connected to the FTP server, performs transactions,disconnects and then again cycle repeats(with the same Object of FTPClient).

In such a case API for the Client says
Because the handling of sockets on different platforms can differ significantly, the FTPClient automatically issues a new PORT command prior to every transfer requiring that the server connect to the client's data port.


As FTP uses TCP connection to perform the operations, the above statement may create lots of trouble.


CHEERS,
Rudradutt Joshi.


Yes, that's it. However when the cycle repeats, a new connection is made using a new instance of the same class (FTPClient). I'm not sure if that's what you asked, correct if I'm wrong please.

Do you have some suggestions?
 
Rudradutt Joshi
Ranch Hand
Posts: 45
Fedora Google Web Toolkit Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What comes to my mind is
what's wrong in using same FTPClient object???
more to that limit the number of instances of the object to say 100 Instances. That will limit your number of active TCP connections, which will provide more controlled environment.
Or may I see the piece of code ??
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!