• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • paul wheaton
  • Jeanne Boyarsky
  • Ron McLeod
Sheriffs:
  • Paul Clapham
  • Liutauras Vilda
  • Devaka Cooray
Saloon Keepers:
  • Tim Holloway
  • Roland Mueller
Bartenders:

Upload large files via HTTP

 
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello,

I need to upload large files to a web server wit the GWT.

All across the internet, I found dozens of examples that pretend like the main problem lies with coding the server, while I have the biggest problem with the client.

I am not using RPC, but simply the request builder on the client and a PHP backend on the server (no servlets).

I know you can use the FileUpload widget. I also know the HTML-Stuff (how you need a form element, too, and how the GWT compiler will create an "input-type:file" element). Since JavaScript does not allow you to access your file system (read/write) directly, I assume that the submit() method of the Form automatically instructs the browser to read the file from disk and create the approriate HTTP header and body to send to the server. I know that will incorporate the file into the body with the proper encoding.
The way I understand it, there is no need to implement any of the onSubmit() or onSubmitSucceeded() methods (sorry, don't know their proper names now) of the Form widget. Unless you want to control input fields or throw messages etc.

I wonder what will happen if that file is bigger than the prescribed HTTP packet size? Will the rest of the file simply be cut off?

But the big question is, how do I upload files larger than 10MB over HTTP using the GWT? I don't want to set up an FTP server (no extra ports by my employer).
Also, since you cannot use JavaScript to split the video into pieces on client side (in the background, without the user noticing), I cannot think of a way of doing this.

And yet, you can do this in Youtube. How? Does Youtube use FTP? Seems unlikely, since the client might have decided to firewall FTP traffic (which Youtube should be prepared for).
Or does Youtube use ActiveX?

Please, I really need to know how to do this.


Thanks in advance.
 
Bartender
Posts: 11497
19
Android Google Web Toolkit Mac Eclipse IDE Ubuntu Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Welcome to the Ranch.

Is your question about uploading large files using the file upload widget or about how the underlying technology works. I reread a few times and am still a bit confused.
 
Marchi Bu
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Maneesh Godbole wrote:Welcome to the Ranch.

Is your question about uploading large files using the file upload widget or about how the underlying technology works. I reread a few times and am still a bit confused.



Well, primarily, it is about uploading large files using the GWT. Since the file upload widget seems to be the easiest potential way of doing this, it is also about using the file upload widget for this purpose.

Moreover, since, in theory, the whole thing seems to be impossible, some questions arouse how this technology actually works for places like youtube. I cannot explain how youtube even accomplishes its large file uploads, given the limited HTML package size and no way of reading and writing the file system with Java Script.

Can you help me with this?
 
Sheriff
Posts: 28371
99
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You refer to something called "HTTP packet size" or "HTTP package size". Can you explain what you think this term (or these terms, if they are different) mean?
 
Marchi Bu
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Paul Clapham wrote:You refer to something called "HTTP packet size" or "HTTP package size". Can you explain what you think this term (or these terms, if they are different) mean?



Well, there usually is a limit of how big each HTTP packet is allowed to be. If there is no such official limitation in the according RFC, there surely is one in PHP (on server side). You can configure it to be 10MB at the maximum from what I've read. This means that any file larger than that has to be split up into smaller parts so they can be placed into multiple HTTP packets.
 
Paul Clapham
Sheriff
Posts: 28371
99
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Okay. But that's an internal detail of the computer's HTTP stack and users shouldn't need to concern themselves with it. Or have you found in practice that your file uploads are being truncated at a size which you believe is this number, or found some documentation which suggests you have to fiddle with this number to be allowed to upload large files?
 
Marchi Bu
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Paul Clapham wrote:Okay. But that's an internal detail of the computer's HTTP stack and users shouldn't need to concern themselves with it. Or have you found in practice that your file uploads are being truncated at a size which you believe is this number, or found some documentation which suggests you have to fiddle with this number to be allowed to upload large files?



Hmm...looking at the PHP and the HTTP documentations, it seems there is really no limit on this. Maybe what I had read before referred to old versions of PHP, but neither the php.ini's UPLOAD_MAX_FILESIZE nor the HTTP (content-length header field have an explicitly stated limit to be used.

So in theory, this should work (provided you have enough memory on server to manage simultaneously several uploads of videos sized several hundreds of megabytes each).
I'll see how this works out in practice, and at worst, implement some kind of waiting queue if the memory is too busy.

Thanks so far!
 
Paul Clapham
Sheriff
Posts: 28371
99
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yes, servers can be written to limit the size of file uploads (otherwise you could have a denial-of-service attack by somebody uploading a file which was ridiculously enormous). But that's not a limit which comes from HTTP, it's a limit which the servers themselves can choose to enforce. So yeah, give it a try and see how it goes.
 
Do not set lab on fire. Or this tiny ad:
Smokeless wood heat with a rocket mass heater
https://woodheat.net
reply
    Bookmark Topic Watch Topic
  • New Topic