Win a copy of Functional Reactive Programming this week in the Other Languages forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Downloading large files

 
Saurabh Pillai
Ranch Hand
Posts: 524
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We are saving large files (typically around 50-100 MB) in our database. I want to know what kind of web service framework we should be using? Web service client (consumer) would be in C# .NET.
 
Ulf Dittmer
Rancher
Posts: 42968
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
None. WS are not made to handle large chunks of data. There are, on the other hand, a number of options that are better suited to that task,like FTP, SFTP, or even straight HTTP. You can still use a WS for communicating the details needed to access the file, though.

Saving large files in the DB is also something that you should think hard about - it might be better to store just the metadata in the DB, and keep the files in a file system.
 
Abhay Agarwal
Ranch Hand
Posts: 1376
Eclipse IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Saurabh

Agreed with Ulf.
Follwoing Ulf advicve can help in easier management of these large files later on

thanks
 
Saurabh Pillai
Ranch Hand
Posts: 524
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ulf Dittmer wrote:None. WS are not made to handle large chunks of data. There are, on the other hand, a number of options that are better suited to that task,like FTP, SFTP, or even straight HTTP. You can still use a WS for communicating the details needed to access the file, though.


So after conversation with our network admin, we decided to use HTTPS. Now I have few more issues. We are going to have Tomcat clustering (i.e. multiple tomcat instance). Now typical URL to access the file would be https://www.mycompany.com/appname/.../video/<filename>. Obviously, video folder would be under webapps folder under tomcat. Now consider this scenario,

- First Request comes to server on tomcat instance 1 to download xyz.mov file. Server pulls it from database and saves it under webapps/.../video/ folder. And send OK response to caller.

- Caller sends another request (https://www.mycompany.com/appname/.../video/xyz.mov) to download the file which goes to say tomcat instance 2 because of load balancing. and it won't find the xyz.mov file there.

How do I resolve this issue? Please let me know if I am not enough clear.
 
Ulf Dittmer
Rancher
Posts: 42968
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You could use a shared file system where files reside in one place, but are accessed from several web apps. It would be better not have those directories underneath Tomcat's web app directory hierarchy anyway. If the Tomcats are fronted by an Apache you could have the Apache serve those files out of one of its directories - it would be better suited for that task.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic