Win a copy of Java Mock Exams (software) this week in the Programmer Certification (OCPJP) forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

HttpURLConnection for multiple CURL commands

 
Aditi Deshpande
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am trying to execute a set of 200 curl commands on a remote host using Java. The jar file is deployed on the host and it is run on the remote host.
I am using the following code to do so.



The problem that I am facing is that some of the curl commands when executed, randomly show an error like "java.net.NoRouteToHostException: No route to host (Host unreachable)". I am able to ping the host and access the web interface and it seems to be working fine. There are a bunch of curl commands that I am executing. (About 200). Some of them work perfectly fine. Randomly, some command fails with the exception "java.net.NoRouteToHostException: No route to host (Host unreachable)". Retries may or may not work. The curl commands are spit out on a log.  When I notice that a curl command has failed, I copy the curl command and try to manually execute it. It works fine without any error even from another terminal on the same host.

One of the things I noticed was that increasing the wait time increased the chances of a retry working. It is like the garbage collector gets more time to free resources and hence it can execute the command without any errors on a retry. The jsonData that is passed to the function is formed using a complex SQL query and takes a few seconds to process. But all of the resources seem to have been freed after the jsonData is fetched.

I feel like this is a Java problem. Glad for any help in figuring out what is going on.

Thanks!
 
Ron McLeod
Saloon Keeper
Posts: 1286
143
Android Angular Framework Eclipse IDE Java Linux MySQL Database Redhat TypeScript
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Aditi Deshpande wrote:I am trying to execute a set of 200 curl commands on a remote host using Java.

Are you doing the 200 operations concurrently or sequentially?

Have you tried snooping the network traffic using something like  Wireshark or tcpdump to see what is happening?
 
Aditi Deshpande
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The operations are sequential. A for loop is used to call the function. They are not dependent on each other.
 
Ron McLeod
Saloon Keeper
Posts: 1286
143
Android Angular Framework Eclipse IDE Java Linux MySQL Database Redhat TypeScript
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I think you should be looking at your deployment platform's network connection or possibly the server you are trying to work with.

I tried 200 operations with your code and didn't hit any errors.  I was testing against a test server at httpbin.
 
Happiness is not a goal ... it's a by-product of a life well lived - Eleanor Roosevelt. Tiny ad:
the new thread boost feature: great for the advertiser and smooth for the coderanch user
https://coderanch.com/t/674455/Thread-Boost-feature
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!