Win a copy of Cross-Platform Desktop Applications: Using Node, Electron, and NW.js this week in the JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Form data exceeding 10MB gradually which is not being accepted by server.  RSS feed

 
Jack Numen
Ranch Hand
Posts: 42
Hibernate Java Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am currently working on a module where huge amount of data needs to be sent to the weblogic server.
The limit on Weblogic-> Server - > Protocols is 10MB.

The huge data is coming when i try to add more than 20 rows to the Ajax request.

There are possible two solutions for this

1) Increase the limit on Weblogic server to 1Gb which might lead to server crash when two or more users are trying to add 100 records at the same time.
2) Send memory in the form of chunks of (length of a row) everytime i add a row. Stor this in session and upon form submission retrive from session. This cannot work if there are more than 20 users doing load testing as increase in session data decreses performance gradually.

Please let me know any solution . I need that data passed to my server without any performance or errors.Currently its showing weblogic.socket.MaxMesssageSizeExceededException and RequestURI too long errors.
 
E Armitage
Rancher
Posts: 989
9
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Test the solution where you stage the session data to a database instead and see if its performance is within the accepted SLA.
 
Tim Holloway
Bartender
Posts: 18661
71
Android Eclipse IDE Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I would definitely "chunk" the data if there's that much. A smaller transmission unit size makes it easier to recover in the case of problems.

However, the web (HTTP) is not the protocol that I would consider first when you're talking that much data. There are others that are more suitable for bulk data transmissions.

However, regardless of transmission channels, E Armitage has the right idea. You don't want to keep that much data in RAM. Either stream it to a staging area (database or flat file) or, if the target is a database, consider using an extended transaction so that the data loads into the database but doesn't commit until the last of the data has been received and stored.
 
Jack Numen
Ranch Hand
Posts: 42
Hibernate Java Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is one of the solution i thought of - staging the session data to DB or a temporary text file. I would give it a try. In order to avoid maximum DB hits i need to send 5 records / rows at a stretch instead of one at a time.

Its a feasible solution and only thing i need to worry is that if the user changes any data in the previous records i need to track those changes and make necessary ammendments to the strategy thereby.

Thanks Tim and Armitage.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!