• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Web interface to delete folder

 
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
I am planning to develop a web interface using JSF to delete folder recursively of size around 800G size.

I have my web front asking the user to select the folder up and I am thinking of two approach to do deletion. Please let me know which one is more appropriate.

1. As soon as the user selects the folder and then clicks on delete command, I can delete the folder using java io api. Is this advisable because the folder size is around 800G and whwn I do it through command line on my linux server, it takes more than 40 minutes to complete the task. What happens when the user closes the web browser as soon as the user clicks the command button to delete the folder. Will it kill the deletion process when the user closes the browser before the folder is deleted. I am not sure about this and hence planning for the second approach as below.

2. Second approach is I am allowing the user to select the folder by the user and then storing the data in a database. Then I invoke a perl script to delete the folder using forking preocess.

Please let me know which approach is appropriate.

My only concern is what happens when the user closes the browser when the process of deletion is going on in my first approach. Is there anyway I can overcome this problem? Then I don't have to use second approach.
Thanks.
 
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
A web request isn't like some sort of a timesharing session where if the user "disconnects" the activity gets cancelled. In HTTP, you send a request, the server processes it, and sends the response back to the requester. If the requester isn't listening for it any more, the response just gets dropped.

Actually, browsers used to time out if they didn't get a response within a certain time. I haven't seen that lately - either because someone changed things, the timeout interval was increased, or web developers learned not to take so long processing requests.

Most commonly for a long-running process, it's advisable to run what I call a "null servlet". The null servlet is just a servlet that doesn't handle requests - unless it wants to - but in its init() method it spawns one or more "engine" threads. You can then set up a disatching queue to give work to the engine.

In you case, you could accept the delete request, pass it to the engine and return a response.The engine would then run the delete offline. You could email a "delete completed successfully" message back to the user if you like.
 
Ranch Hand
Posts: 598
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Can we see any example working on the logic given above as many terms are not clear to me like disatching queue, "engine" threads.

please explain or give some example code if possible.
 
Gopu Akraju
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Tim for your response. A small example would be easy for me follow up. As you had suggested, once the deletion gets completed, I am planning to send mail to the users saying that deletion gets comlpeted. Most of the things you have explained, I really couldn't follow. Please let me know if you could give me a small example. Thanks.

I have finished using my second approach. For that I need the combination of java web front with perl script to carry on 'rm -rf' command. But your suggestion is interesting and want to learn more about it. Please let me know. Thanks.

Regards
 
Tim Holloway
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
OK. I'm going to do this in the abstract, since, while it's not that much to code, it's more than I can code correctly in a hurry.

First, you need to understand that HTTP is a request/response protocol, not a continuous-session (client/server) operation. So when you code a servlet, JSP, or whatever, you're obligated to take the request, process it, and return the response as fast as possible. This is the main reason why the normal request handlers are forbidden to spawn threads. If you return the response, but you've spawned a thread, it's tied to its parent and that can muck up the internal resource management of the server's request dispatcher.

So any threading you do has to be done somewhere safe. I used to to this with vendor-specific code, but it's more portable when you do it using regular webapp resources. A safe place to to this is in a servlet's init() method. Unlike the doGet/doPut and related routines, init() is invoked only once - when the servlet is constructed, and it's running under the same thread that controls the servlet as a whole, so no problem. Or at least none I know of!

You want to spawn an "engine" thread here, because the servlet won't be placed into service until init() returns. An engine is just a process that takes work in and does something, then returns to idle until the next unit of work comes in. Unlike HTTP, however, where the handlers react to external requests, the engine obtains requests on its own.

The link between the HTTP and the engine is a Queue, most commonly a FIFO. There are a number of Java collection classes that can be used for this purpose. Vector is good, since it's thread-safe. The more general collection classes normally aren't, but they can be told to be thread-safe.

Thread-safe is important, since the HTTP and engine threads are 2 different threads and therefore a non-thread-safe structure can become corrupter if they both attempt to modify it at the same time.

There's one other item you need - a semaphore. This is a mechanism that causes a process to wait until there's data in the work queue. Otherwise, you'd sit and spin, polling and wasting CPU time. Or you could run a timer and poll periodically, which is more polite, but not as responsive as when you get a wake-up call as soon as something's posted to the work queue.

There are also some Java classes that provide both queue and semaphore as one - which is known as blocking. So if you haven't studied up on Java collections, this is a good excuse to learn them!

Here's an example, in brief:

Define a work queue. Let's say it's an array of Strings, each one being the name of a directory to delete. At init time, you start the engine thread. The engine goes to pull work from the queue, but there is no work, so the engine goes to sleep.

Now an HTTP request comes in. The request handler adds an entry to the engine's work queue and returns a "Work has been scheduled" message to the user.The queue manager wakes up the engine and it pulls the request off the queue and executes it.When the request finished, the engine attempts to pull another request and repeat the process. If other requests have come in, the dequeuing operation will return immediately with the next request string. If the queue is empty, the engine goes back to sleep until more work arrives.

In REALLY busy environments, the engine might not do the work directly. Instead it might construct multiple sub-threads and pass each request to a sub-thread. Usually you set an upper limit on the number of sub-threads, and you might even pool them. You can get very elaborate if you need to.
 
Gopu Akraju
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Tims for your eloborate reply. I could understand but am not in a position to code as you had mentioned as I am also new to JSF and java. It is still interesting and want to do as I might learn a lot of new things in java.

I have a folder structure like below: It contains images which contains 8 subfolders from test1 till test8 and each test contains 8 folder as 1 2 ..8.

Images
test1
1
2
... till 8
test2
1
... till 8
test3
1
2
... till8
test4
8 folders as above
....
test8

When the user clicks once, I have to delete the entire Images folder and finally send a mail to the user. It would be really helpful if you could give an abstract skeleton on how to do. I don't know how to satrt the engine or thread etc... as you had mentioned and could not proceed further.
I am using JSF for web front and kindly give a small sketch on a fuction which will be called on command button. Thanks.

Regards
[ September 27, 2008: Message edited by: Gopu Akraju ]
 
Gopu Akraju
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

After doing some google search, I thought I can still call a perl script which can do the forking and deleting the folders and sends an email to the user. I am calling my perl script from java as below:



Now I have a doubt.
1. As the deletion takes place for more than 40 minutes, my above program will still work?
2. The user might click or start and then close the browser when the deletion ia taking place. What will happen to the process? How to handle it?
3. My intention is to start the perl script and my java control can exit as perl script takes of everything. Is it possible to do it this way?


Please let me know about your suggestions. Thanks.
 
Himanshu Gupta
Ranch Hand
Posts: 598
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yes, your programe will still work after closing the browser window. It is described in the above post also.

The problem is that the programme will be in the stack upto the completion of the process. To solve this we can give this task in a differtnt thread and can detach it from its parent. I am not sure that this will solve the problem It is just a try.
 
Gopu Akraju
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Himanshu,

Yes, I did check and even after closing the browser, the perl script continues its work to complete deletion till sending mail to the user. And I could see System.out.println("CommandSuccessful"); after the invoking the perl script.

I am not seeing errors or any such report in the tomcat logs after that. That means the java exits gracefully or am I missing something? Experts please advice. Thanks.
 
Himanshu Gupta
Ranch Hand
Posts: 598
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ya that's right.

The point to be considered here is that when our programme is being executed the servlet may block some of the resources depending upon where we write the code like may be in init() or in doPost(). Well i am not clear about the absolute life cycle and don't have the information that what are the resources it takes and when. If we are going to implement this way our resources will be waiting till the end of the process. So if there are a number of request then it will make our server slow.

As discussed above implementation of engine will remove any such possibility. The thread and queue pattern (as discussed above) seems to be apt for the particular case.
[ October 01, 2008: Message edited by: Himanshu Gupta ]
 
Gopu Akraju
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Himanshu for your input. Can you please give me an example? I am not able to proceed further. Thanks.
 
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

I am not able to proceed further.


What do you have now, and what stops you from making progress?
 
Gopu Akraju
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
I am invoking a perl script which runs for about 1 hour time and you may see my above code how I start my perl code etc.. The java part after invoking the perl script comes out of the control ( A web front for the user to click) and then the user can close the browser. As far as I am concerened I am not seeing any error message in my tomcat logs and it looks like a smooth exit. As Himanshu and Tom explained, do I have to invoke this invoking (perl script) in a separate thread to be in the safer place?

Do my class have to implement runnabale interface and put the aboce code inside run method? Or am I missing something here?

Sorry for the confusion. Thanks.
 
Ulf Dittmer
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It's certainly preferable not to have servlet code run for an hour, so, yes, running that code in its own thread would be better. You can set some flag when it's done in the user's session, and display that to him some time later if required.
 
Gopu Akraju
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Ulf,
Thanks for your response. I don't want to show anything back to the user when the deletion gets completed as my perl script will send mail to the user regarding the completion. And hence the user can just close the browser. I am justusing java to invoke perl script and if it is not advisable I can always run perl script as cron job. Please let me know which one is preferable?
 
Ulf Dittmer
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
How would using a cron job work if the action is to be started from a web interface?
 
Gopu Akraju
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi
I will be dumping all the info from the web front into a DB and then perl script looks for the job from DB and invokes.
 
Tim Holloway
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hmmm. It looks like I missed some traffic on this subject.

1. You CAN exec a Perl script to delete files, but plain old Java is normally sufficient to delete a file or a directory tree. Saves some overhead and makes the app less complex. Look at the Javadocs for java.io.File.

2. A browser/webserver connection is NOT a client-server connection. I should make that display in 26-point purple flashing type. The only other false impression that's that commonplace is the idea that a web server is the same thing as a file server (it's not).

A client-server session over the Internet is a conversational mechanism. The user attaches to the server program and holds an open network linkage from the start of online communications until they are done.

HTTP is different, and HTTP is the protocol used for the web, regardless of whether the web app is cgi, PHP, .Net or Java and regardless of what framework you're using - raw JSP/servlet, Struts, JSF or whatever. HTTP is the foundation or skeleton upon which all these are built, and it shapes and constrains how web communications can work.

The most important thing about HTTP is that it's transactional. Everything in HTTP is in terms of a request followed by a response. A request comes in, the server sends a response back. End of transaction. Each request/response cycle stands alone. There's no ongoing communication. Web applications that want to give the illusion of an ongoing communication have to fake it, usually by creating some sort of server-side session object whose key is passed back and forth in the request/response cycle. Generally that's done by URL rewriting or by passing the session ID in a cookie, but it doesn't have to be.

What this means it it doesn't mean NOTHING that the user closes the browser window. There's no long-term connection to break, so the response simply ends up being thrown away if the browser stops listening for a response. For most purposes, you can consider the response as being entirely generated before anything is sent back to the user/browser. Only once this is done would it even matter whether a network link still exists. If it does, the response is transmitted.

Once the client/browser has received the response, the cycle is complete and all network connections between the browser and web server are closed. A whole new set of connections have to be created if and when another request/response cycle is made.
 
Gopu Akraju
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Tim,

Thanks for your response. I use existing perl script to do all the manupulations and hence am not using java io api.

In your explanation you have mentioned as below.

Once the client/browser has received the response, the cycle is complete and all network connections between the browser and web server are closed. A whole new set of connections have to be created if and when another request/response cycle is made.



When the browser is not ready to hear a response from ther server (when the user closes the browser), then am I doing any damage or am I not handling the response properly? What might be the effect?
 
Tim Holloway
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You don't have to close a browser for it to stop listening for a response. After a while, the browser may simply time out and display a timeout message.

I haven't seen timeouts recently, and I don't know if it's because newer browsers don't timeout (unlikely), the default interval is now longer or people just got better at returning responses before the interval expired.

When you initiate a request/response cycle, two socket connections are established. One is to the port on the server (by default, 80 for HTTP). The other is a "random" port on the client. That's the port the server will respond to.

If the response port stops listening, no matter whether it's because the browser timed out the cycle and released resources (closed the port), or because the browser was terminated, or for any other reason, the server will not be able to initiate transmission of the response. In that case, the server simply discards the response.

If the port closes while it's receiving a response, the server will get errors in response to its attempts to write the response. Since these are non-recoverable errors, the server will simply discard the remaining part of the response.

Ultimately, in HTTP, it's the client's responsibility to receive and process a response. If the client won't or can't, the server isn't required to do anything about it.
 
Gopu Akraju
Ranch Hand
Posts: 242
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Tim,
Thank you so much for you response and I did learn a lot and of course I am also reading proper JSF and many things are new to me. Thanks.
Let me summarise.
1. I use my web front to select the folder to be deleted and invoke a perl script.
2. perl script should coninue to delete the folder which usually takes more than 50 minutes.
3. I did the testing in deleting the folder which took about 7 or 8 minutes (mini folder and not the actual folder) and it works fine.
4. I see the tomcat configuration web.xml as below (30 minutes)

5. Since I haven't tried deleting the real larger folder, I would like to know whether perl script will continue to remove the folder or will it stop after 30 minutes due to timeout.
6. In my main perl script, I have forked a process and child process will remove the folder.

Please let me know. If this fails, I have to follow the suggestion you had mentioned earlier, opening a new thread to keep the session alive and as well java io api to delete the folder.
 
Himanshu Gupta
Ranch Hand
Posts: 598
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Perl script will keep on executing after 30 min or 60 min as it doesnt depends on the session timeout. Please refer again all the discussion and I am sure that this will clear your confusion.

Your process has to do nothing with the session time out.
 
Ranch Hand
Posts: 94
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Gopu,
I think you are really getting confused with all these concepts.

1)From WebUI you select a folder to delete
2)Internally it executes perlscript
3)Now you WebUI's job is done, the deleting part is done by perl script now.
It is the responsibility of perl script to delete now, even if your WebUi
Crashes or browser crashes it doesn't matter because "perl" script will
delete the folder
4)Now the big question here is what is going to happen the user who
used WebUI to select folder for deletion? the browser would be blank
and the user would be frustrated seeing that he is not getting any
response for long time say "50minutes" in your case.This is really bad
idea..

5)You can easily tackle this!!
Before you execute the perl script or after you execute the perl script
send an immediate resonse to the user saying"Started to delete folder,
it may take few minutes to delete folder, check your mail after some time"

In this way user gets the response that he has selected the folder for deletion and need not wait for long time till the process gets deleted
 
aleem khan
Ranch Hand
Posts: 94
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Gopu,
I think you are really getting confused with all these concepts.

1)From WebUI you select a folder to delete
2)Internally it executes perlscript
3)Now you WebUI's job is done, the deleting part is done by perl script now.
It is the responsibility of perl script to delete now, even if your WebUi
Crashes or browser crashes it doesn't matter because "perl" script will
delete the folder
4)Now the big question here is what is going to happen the user who
used WebUI to select folder for deletion? the browser would be blank
and the user would be frustrated seeing that he is not getting any
response for long time say "50minutes" in your case.This is really bad
idea..

5)You can easily tackle this!!
Before you execute the perl script or after you execute the perl script
send an immediate resonse to the user saying"Started to delete folder,
it may take few minutes to delete folder, check your mail after some time"

In this way user gets the response that he has selected the folder for deletion and need not wait for long time till the folder gets deleted
 
Ranch Hand
Posts: 114
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Just a little off topic, but I am curious to know how Gopu is selecting a folder on the browser - is there a JSF component? My application had a requirement to select a folder using a 'Browse' button, but all we have in regular Html is to select a file not a folder. Thanks in advance.
reply
    Bookmark Topic Watch Topic
  • New Topic