• Post Reply Bookmark Topic Watch Topic
  • New Topic

Question about URLConnection  RSS feed

 
George Lin
Ranch Hand
Posts: 125
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello everyone,


I am using HTTP protocol to transfer large amount of data across two machines. And I have found out that some packages will be lost when using the following source codes. I am wondering whether there are anything wrong or potential issues in them.

Data request



Data response



I am wondering whether the package losing is caused by the reason that, data response is too fast than data receiving so data receiving application will miss some chances to read packages.

I am also wondering what is the effective approach of transfering data across two machines using HTTP protocol without losing any packages. Are there any online samples?


Thanks in advance,
George
 
Joe Ess
Bartender
Posts: 9406
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Packets don't "get lost". TCP/IP is constructed to ensure delivery if at all possible. Either you have a physical network problem or bad code. In your case, you are probably trying to send more than 2k and there is a limitation to the amount of data sent in an HTTP GET request. You also have to encode the data as some of the data may be considered reserved characters in a URL. Try sending a proper POST request like this example from the Java Almanac.
 
Ilja Preuss
author
Sheriff
Posts: 14112
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Moving to Sockets and Internet Protocols...
 
George Lin
Ranch Hand
Posts: 125
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you Ilja, I think this forum is more suitable for my post.


Originally posted by Ilja Preuss:
Moving to Sockets and Internet Protocols...



regards,
George
 
George Lin
Ranch Hand
Posts: 125
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Joe,


Originally posted by Joe Ess:
Packets don't "get lost". TCP/IP is constructed to ensure delivery if at all possible. Either you have a physical network problem or bad code. In your case, you are probably trying to send more than 2k and there is a limitation to the amount of data sent in an HTTP GET request. You also have to encode the data as some of the data may be considered reserved characters in a URL. Try sending a proper POST request like this example from the Java Almanac.


Your reply is very helpful. I am wondering if sender application sends data much faster than receiver application receives and process data, how can HTTP protocol ensure that no data is losing. Does there exist a buffer to contain data which is sent out (write) and not yet received (read) by receiver application?


regards,
George
 
Joe Ess
Bartender
Posts: 9406
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by George Lin:
Does there exist a buffer to contain data which is sent out (write) and not yet received (read) by receiver application?


Yes. TCP/IP and the Internet were developed by the US government's Defense Advanced Research Project Agency to survive nuclear attacks so it is pretty robust. Packets are numbered and are transmitted and received in order. If a packet is received out of order or damaged in transit, the network will let the sender know so it can resend the missing/damaged packets (UDP, in contrast, does not transmit data in order or insure delivery). Buffering takes place at many layers to keep data flowing.
Read about TCP at Wikipedia
It is extrememly unlikely that you are overwhelming your server.
 
George Lin
Ranch Hand
Posts: 125
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Joe,


Originally posted by Joe Ess:


Yes. TCP/IP and the Internet were developed by the US government's Defense Advanced Research Project Agency to survive nuclear attacks so it is pretty robust. Packets are numbered and are transmitted and received in order. If a packet is received out of order or damaged in transit, the network will let the sender know so it can resend the missing/damaged packets (UDP, in contrast, does not transmit data in order or insure delivery). Buffering takes place at many layers to keep data flowing.
Read about TCP at Wikipedia
It is extrememly unlikely that you are overwhelming your server.


Your reply is very helpful. I think you mean my issue is not caused by network layer or HTTP stack but the specific logics of my application. Am I correct?


regards,
George
 
Joe Ess
Bartender
Posts: 9406
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
As I said in my first answer, I would bet that you are either:
1. attempting to send more than 2k in an HTTP GET request, which will truncate the data or
2. not encoding the data, which will cause problems with the request.
 
George Lin
Ranch Hand
Posts: 125
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Joe,


Originally posted by Joe Ess:
As I said in my first answer, I would bet that you are either:
1. attempting to send more than 2k in an HTTP GET request, which will truncate the data or
2. not encoding the data, which will cause problems with the request.


I am sending more than 2k data with POST data transfer approach of HTTP. I am using POST approach to transfer binary data of a large file through HTTP. I am wondering in my case whether I need to encode data. Does HTTP automatically encode data when we use POST approach other than GET?


regards,
George
 
Joe Ess
Bartender
Posts: 9406
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by George Lin:
I am wondering in my case whether I need to encode data.

Do not wonder. You must encode the data. For example, the space character is not allowed in a URL. Do you have any spaces in your data?
See RFC 1738 - Uniform Resource Locators, especially section 2.2. URL Character Encoding Issues, for more info.

Does HTTP automatically encode data when we use POST approach other than GET?

No. Again, take a look at the example I posted. In particular the use of the class java.net.URLEncoder.
 
George Lin
Ranch Hand
Posts: 125
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you very much Joe!


Originally posted by Joe Ess:

No. Again, take a look at the example I posted. In particular the use of the class java.net.URLEncoder.


I think URL is transfered into HTTP header and if it contains special characters (for example, space character), we should use URLEncoder to encode the URL. But if I transfer binary data through HTTP body, which method should I use to encode data?


regards,
George
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!