• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

JSF file upload IE8 over 2GB

 
K. Tsang
Bartender
Pie
Posts: 3222
12
Android Java
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello ranchers

I'm using primefaces 3.5 and JSF 2 to do file upload. The client's browser support are IE8 and FF. I've done some research and understand the IE8 internal file upload size is 2GB and IE8 doesn't support HTML5 File API and stuff like that, yet FF does. Also when trying to upload large files say over 1GB, the browser seems to hang not knowing if the server is doing anything.

So I managed to do the upload through "iframe" approach. Now once I do that, I can't get the JSF validation message to come out in the parent page (outside iframe). Also if I manually call the JSF validator, the FacesContext is null. Attached is the screen shot when upload over 2GB file in IE8. This screen will occur inside the iframe which is acceptable by me. Yet I want some error message saying it failed. I have tried throwing exception in my UploadServlet which will display a HTTP500 page with the stack trace inside the iframe.

In the below code if I do not specify the "action" in the form tag, the entire page (form) will display inside the iframe with the error messages.







So my question are
1) How do get error messages to display outside iframe?
2) Is it possible to get a instance of FacesContext inside the upload validator or before hand (say in the managed bean and store in some scope) and get this inside the validator?

I think able to solve either one will get something going.

Thanks in advance.
pic.png
IE8_over_2gb
[Thumbnail for pic.png]
 
Jayesh A Lalwani
Rancher
Pie
Posts: 2756
32
Eclipse IDE Spring Tomcat Server
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
IIRC, if you try uploading a file greater than 2g in IE, the page will crap out immediately. It won't even give your code any control.
 
Ulf Dittmer
Rancher
Pie
Posts: 42966
73
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
A file of that size is perhaps not well suited for HTTP form file upload, no matter what browser. Have you considered an alternative method, like FTP?
 
K. Tsang
Bartender
Pie
Posts: 3222
12
Android Java
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
the client wont allow ftp. The purpose of http upload is also filter out first round of invalid files eg incorrect headers or footers. Another thing is they cant overwrite the same file name which ftp will sure do.
 
Jayesh A Lalwani
Rancher
Pie
Posts: 2756
32
Eclipse IDE Spring Tomcat Server
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
K. Tsang wrote:the client wont allow ftp.


Yeah we go through this dance with our clients. It multi matey comes down to that The client doesn't know what they want. Usually, it's a business guy who doesn't understand how crappy it is to do a 1g http upload. A/he will ask for http upload because it sounds convenient for him/her. On the other side, there is a tech guy at the client who is grumbling why products don't allow FTP upload of 1g files.

Generally 1g files are created by some sort of servicing systems, and/or extracted from databases. Users do not maintain 1g files on their local machines. Business users are used to having their tech guys give them extracts of tidbits of data from the servicing system, and they don't realize that when they get a BI product that can process their entire data, it's better to transfer data from their servicing system to yours directly. Chances are if you get 2 tech guys together they will figure out a way to get FTP interaction working.


Here's what you should do if you can. Don't build http upload for 1g files. Instead, tell your client that your tech guys can work with their tech guys to get the data into your system.
 
K. Tsang
Bartender
Pie
Posts: 3222
12
Android Java
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Jayesh.

We propose the FTP thing some time before but nope they didn't listen. To complicate things even more, they have the app server (JBoss) and database server (Oracle) on 2 separate machines. Meaning even the http upload is ok these files are on the JBoss server which later when they run batch processes they will need to manually copy such file to the db server.

Another not so good suggestion is to lower the http upload size limit from 2GB to 1GB or even lower.
 
Tim Holloway
Saloon Keeper
Pie
Posts: 17616
39
Android Eclipse IDE Linux
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Actually, depending on the environment, the 2GB limit might not even be coming from the webapp directly. It might be a limitation based on the memory size or filesystem of the target machine. Or JVM.

FTP can very definitely be used to transfer files with screening and anti-duplication measures. Some years back I used to send files to an external service. They used what I call the "magic coin bank" approach. It's rather like certain novelty items where you put a coin in a holder, press the "start button" and this lid flips up, a hand comes out, and the coin is dragged out of sight into the bank. At that point, what happens to the coin is up to the bank. Often a small dummy file is posted to the ftp account so that the client knows when/whether the operation succeeded or not (the actual "hand" mechanism runs on a cron task).

Other options that can be even more effective are scp and rsync. In some cases, you can get rid of the requirement for a user to manually initiate transfer altogether. In fact, this is part of the 4am server backup processing that I do daily on my own systems.
 
Ulf Dittmer
Rancher
Pie
Posts: 42966
73
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You could embed an FTP server like Mina, which -via the concept of FtpLets- lets you accomplish the missing functionalities. It also supports FTP over SSL if you want to make the transport more secure.
 
Jayesh A Lalwani
Rancher
Pie
Posts: 2756
32
Eclipse IDE Spring Tomcat Server
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Is this an installed application, or a hosted SaaS application? By that I mean are you installing your application into their servers, or are you hosting the application in your servers. In the Saas model, FTP can become an issue because they have to make sure they open FTP ports, and they make have some company policies against that. If it's installed, it should be easier since it should be easier for them to open ports internally (or not depending on how paranoid they are)

ANother thing that we did is in situations where our clients do have to physically upload the file through HTTP, we asked them to zip the file. This reduces the need to send large files. Or alternatively, ask your client to split up the file into multiple uploads. If they are going to do things in stupid ways, why make it easy for them? :p
 
Tim Holloway
Saloon Keeper
Pie
Posts: 17616
39
Android Eclipse IDE Linux
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jayesh A Lalwani wrote:Is this an installed application, or a hosted SaaS application? By that I mean are you installing your application into their servers, or are you hosting the application in your servers. In the Saas model, FTP can become an issue because they have to make sure they open FTP ports, and they make have some company policies against that. If it's installed, it should be easier since it should be easier for them to open ports internally (or not depending on how paranoid they are)

ANother thing that we did is in situations where our clients do have to physically upload the file through HTTP, we asked them to zip the file. This reduces the need to send large files. Or alternatively, ask your client to split up the file into multiple uploads. If they are going to do things in stupid ways, why make it easy for them? :p


Hmmm. I think on that score that both Dropbox and Amazon S3 are using HTTP for large-file uploads. Dropbox using S3, in fact. But of course, Amazon built their infrastructure specifically for that kind of stuff.

ZIPping uploads might be counter-productive, depending on how client/server compression options are set, since a compressed upload isn't necessarily going to optimally handle a compressed file.
 
Jayesh A Lalwani
Rancher
Pie
Posts: 2756
32
Eclipse IDE Spring Tomcat Server
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Well, it just gets around the limitation of IE. ALso, what we do is store ths compressed file in database, and uncompress it only when we need it. This reduces the IO not only during upload but also during the backend when we process the large file.
 
K. Tsang
Bartender
Pie
Posts: 3222
12
Android Java
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Zipping the file may be a way to go because it reduces the size. Yet I'm thinking if the browser (HTTP) can't even communicate with the server for x GB the server can't do anything.
 
K. Tsang
Bartender
Pie
Posts: 3222
12
Android Java
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Good news. The 2GB limit will be lifted to a smaller, manageable size limit :) Now in place of this a timer will be added to indicate the server is processing the upload.
 
Jorge Castro
Greenhorn
Posts: 1
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi there:

As an alternate solution, you can install a flash component for upload files.
 
Tim Holloway
Saloon Keeper
Pie
Posts: 17616
39
Android Eclipse IDE Linux
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jorge Castro wrote:Hi there:

As an alternate solution, you can install a flash component for upload files.


Please don't!
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic