• Post Reply Bookmark Topic Watch Topic
  • New Topic

401 (Authentication issue) for Inputstream on parallel request  RSS feed

 
dubu sing
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have a Client application which connect, write and read data from a remote server. It's fine when i do all these activities in single (main) thread and all the time it write and read data from server without any issue. I have 100 remote servers and as long as i am doing connect, write and read individually with 100 servers one after another i am able to do all 3 activities successfully without any issue. But i am facing problem when i am trying to request all 100 servers in a loop by creating multiple (100) threads at one shot. I am able to connect and write to the server but while reading through InputStream it's throwing IOException with response code 401 which mean some issue with authentication. But same authentication key is used both the times (In single thread and multiple thread). If i put a sleep of 5 to 6 seconds while connecting each server i am able to connect, write and read successfully. All 100 servers have different url or IP. All the above mentioned Connect, Write and Read methods are in one class which implements Runnable. So creating a creating 100 threads in loop will have their own copy of these methods. Can anybody please guide me where am i wrong?

 
Ron McLeod
Bartender
Posts: 1603
232
Android Angular Framework Eclipse IDE Java Linux MySQL Database Redhat TypeScript
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It may be that either the sites or something between your application and the sites (proxy?) is enforcing rate limits (maximum number of queries per second), and is rejecting your request with a 401 response.  A 401 could indicate that you need to be authenticated to be served at a higher rate.
 
dubu sing
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If so "enforcing rate limits (maximum number of queries per second)" is there any way to know "What is the rate limits"?
 
Ron McLeod
Bartender
Posts: 1603
232
Android Angular Framework Eclipse IDE Java Linux MySQL Database Redhat TypeScript
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Check the response headers - there may be a custom headers that provide information (such as something like X-RateLimit: 50).
 
dubu sing
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I checked the headers as below.
null : [HTTP/1.1 200 OK]
content-type : [text/xml; charset=UTF-8]
Access-Control-Expose-Headers : [building-insights-server-version]
X-XSS-Protection : [1; mode=block]
date : [Fri, 16 Dec 2016 07:24:10 GMT]
Access-Control-Allow-Methods : [OPTIONS, GET, POST, PUT, DELETE]
Access-Control-Allow-Credentials : [true]
X-Powered-By : [Express]
Strict-Transport-Security : [max-age=31536000;]
Access-Control-Allow-Headers : [undefined]
Access-Control-Allow-Origin : [undefined]
content-length : [491]
building-insights-server-version : [2.5.0-RC7]
server : [N00AC39/2.16.3.30]
 
dubu sing
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You mean to say "X-RateLimit" of the 100 remote Servers which is throwing error while opening InputStream or the local Server where my application is running?
 
Ron McLeod
Bartender
Posts: 1603
232
Android Angular Framework Eclipse IDE Java Linux MySQL Database Redhat TypeScript
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
dubu sing wrote:You mean to say "X-RateLimit" of the 100 remote Servers which is throwing error while opening InputStream or the local Server where my application is running?

The X-RateLimit header name is just an example of what a server might include in a response i the actual headers used will eb API dependent.  This is how Twitter's API deals with rate limiting.

If you don't have any information on how the API is supposed to work, you will have to reverse-engineer its behaviour.  Check the headers in all responses - both successful and failed to see if you find any clues.  Try adjusting the rate at which you send your requests.  Check if sending the requests serially vs. concurrently makes any difference.  Try your application using a different Internet connection to see if it is something with your network/ISP which is interfering.

Also - you shouldn't blindly attempt to read data for each response. Typically non-200 responses do not carry any payload.
 
Consider Paul's rocket mass heater.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!