I am working on a solution that will provide multiple simultaneous file upload functionality. In order to facilitate this, I created a few classes using the FTPClient provided in the Apache Commons Net library as the underlying socket client. Note that I built my own abstraction called, "FTPClient" and composed it with the org.apache.commons.net.ftp.FTPClient class.
First is the "FileUploader" class which is used to invoke the upload process via the 'upload()' method (note: internal package names are shortened for brevity).
Second is the FileUploaderThread which implements Runnable and is responsible for polling files from a thread-safe queue and uploading them via calls to an FTPClient instance. Each FileUploaderThread instance gets a single FTPClient instance.
For a small number of files (e.g. 100), I can invoke the code via a JUnit test without any problems. If I use a larger file set which more closely reflects what will occur in our production environment (e.g. 5000+ files), the JVM will deadlock after a certain amount of time. Further, the same FTPClient upload code is being used in single threaded process without any problems.
I've tried a variety of things in an attempt to resolve my problem, but at this point I'm starting to run out of new ideas. My suspicion is that since the files I'm uploading are very small (less than 5 KB), there are two many socket connections occurring within a very short period of time.
I don't have much experience with multi-threading as it applies to socket connections. As such, any criticisms, recommendations, or suggestions are all welcomed!