Processing input files with Servlets

[Mark Howard]

Hi all you Servlet gurus!
I am hosting a website which runs purely on Servlet, JSP and JDBC technology, but I've little experience with Servlets accepting and processing input files from the user. Ok. I have no experience of it
I would appreciate any suggestions.
Essentially, I need to accept a simple text file from the client and use it to update their database records. The questions I have are as follows:
1) How best to accept the input file by the simplest means. Can I open an open-file type popup that allows the user to browse and select a text file?
2) How does the Servlet read the file? ie. which API should I use, etc
3) The file simply has two columns per record which require validation. Is a delimiter the best (or at least simplest) way for me to distinguish between the 2 columns?
4) This is my most important question. I'm expecting about 100 to 2000 records per input file, and I can receive many simultaneously from various clients. What is the best way to process this? Each file will hit different records in the database, but I'm still concerned about the performance issue. It is not imperative for the client to wait for a completed-processing response, but how do I get the Servlet to process the file in the background and allow the client to carry on with other tasks. (I can always update a process-completed flag which will notify the user that their file has been processed the next time they poll their records)
5) Should I be concerned with any synchronization issues?

Thanks a mil for taking the time to read this missive. Any suggestions are much appreciated

[Kyle Brown]

Originally posted by Mark Howard:
Hi all you Servlet gurus!
I am hosting a website which runs purely on Servlet, JSP and JDBC technology, but I've little experience with Servlets accepting and processing input files from the user. Ok. I have no experience of it
I would appreciate any suggestions.
Essentially, I need to accept a simple text file from the client and use it to update their database records. The questions I have are as follows:
1) How best to accept the input file by the simplest means. Can I open an open-file type popup that allows the user to browse and select a text file?
2) How does the Servlet read the file? ie. which API should I use, etc

If you do a Google Search for "Servlet File Upload" you will find the very first hit is this new package for the Apache Commons that will do it for you. Another example can be found here. I personally have successfully used the O'Reilly Upload classes before.

3) The file simply has two columns per record which require validation. Is a delimiter the best (or at least simplest) way for me to distinguish between the 2 columns?

Generally yes. Use one delimiter (like a comma) to delimit columns and another (like CR/LF) for rows.

4) This is my most important question. I'm expecting about 100 to 2000 records per input file, and I can receive many simultaneously from various clients. What is the best way to process this? Each file will hit different records in the database, but I'm still concerned about the performance issue. It is not imperative for the client to wait for a completed-processing response, but how do I get the Servlet to process the file in the background and allow the client to carry on with other tasks. (I can always update a process-completed flag which will notify the user that their file has been processed the next time they poll their records)
5) Should I be concerned with any synchronization issues?

Take a look at the article I wrote for the March Javaranch Journal on this very subject. This requires a full J2EE implementation (EJBs) but it's by far the best approach.

Thanks a mil for taking the time to read this missive. Any suggestions are much appreciated

[Mark Howard]

Thanks for this information, Kyle.
Very useful indeed.

[Mark Howard]

Further to the above...
If it is not imperative that the user waits for a response from the asynchronous process they triggered with their request, is there any way in which the asynchronous process could carry on crunching away while the user is freed to continue with other requests.
For example, user submits request A, servlet uses request A to trigger asynchronous process B, servlet returns control to the user who can then continue with other requests. When complete, process B could, say, update a database flag to indicate that the users data has been processed. When the user next issues a request that hits that particular data, the servlet servicing that request could query the flag and inform the user whether their data is still being processed or not.
Or is this the bit about having to spawn another thread to do Process B, which I believe is not considered good practice?
Kyle's solution in the Javaranch Journal looks quite sophisticated (and impressive), but I don't know enough about the full J2EE picture to make that kind of change. I guess I'm just looking for the quickest and simplest solution for now. Also, using Lunarpages as a host (and a good one at that), I don't know whether their Resin server does all the good things required by the J2EE spec?