• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Tim Cooke
  • Junilu Lacar
Sheriffs:
  • Rob Spoor
  • Devaka Cooray
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
  • Tim Holloway
Bartenders:
  • Jj Roberts
  • Al Hobbs
  • Piet Souris

commons file upload - how big file can it handle?

 
Ranch Hand
Posts: 365
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Friends,

Is commons file upload suitable for uploading files>10MB in size?

What we have observed that commons file upload load a file in memory. So for files > 10MB the deply in response is too much.

Is this right? Is there something we are missing?
Is there any other good variation of commons file upload if we have to use it for uploads upto 30MB?

Regards,
Tina
 
Ranch Hand
Posts: 64
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yes it is time consuming for bigger size files,
for this you need to implement multiThreading so that
You can have background thread for uploading CSV files & validation of it.
 
Sheriff
Posts: 13411
Firefox Browser VI Editor Redhat
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Commons upload can write to disk as well.


FileUpload creates new file items using a FileItemFactory. This is what gives FileUpload most of its flexibility. The factory has ultimate control over how each item is created. The factory implementation that currently ships with FileUpload stores the item's data in memory or on disk, depending on the size of the item (i.e. bytes of data). However, this behavior can be customized to suit your application.


Source:
http://jakarta.apache.org/commons/fileupload/using.html
 
Tina Desai
Ranch Hand
Posts: 365
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yes, we just tried that option too Ben.
There is some gain for small files but still no help for large files.

For large files, we are getting Items list as 0.
So there is nothing to iterate on and check if it is a file.

We trapped this scenario and added code to show some customized error when Items are 0. But then it takes agaes to display the error message on screen when from the logs we can make out that the control did go till the point of displaying the error message to screen.

Do not know why this behaviour for large files [>40MB]. It can be good to know the some threshold at which such error occurs and if it is a known problem with commons fileupload.

Regards,
Tina
 
Ben Souther
Sheriff
Posts: 13411
Firefox Browser VI Editor Redhat
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I don't.
Since this conversation is becoming more about commons fileupload, I'm going to move it to the Other Open Source APIs forum where this topic is usually discussed. (A copy will remain here).
 
Tina Desai
Ranch Hand
Posts: 365
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Ben! Regrets for the inconvenience.

Regards,
 
Tina Desai
Ranch Hand
Posts: 365
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Anyone?

Regards,
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic