RESTlet slower than Axis2?
Hello Guys! I've been reading about REST, and based on what people say it should be really faster than Axis2:
1) REST architeture is simpler than
SOAP
2) JSON is faster to marshall and demarshall than XML
3) Axis2 generated code is way more complex than RESTlet code
It made sende, and I decided to learn REST and ultimatelly run a benchmark. It consisted on comparing local call, with SOAP call and REST call. Base rule is kinda simple:
I created a class with a method that receives an int and a double as parameter:
It creates an array of doubles and loops this array filling it with the sum of provided double and array index. Then it returns this array.
For every
test, I created another loop, also based on the int, that calls operation(). It counts how many milisecs it takes for each call and for whole loop. Basically:
With first loop I can have some processing and also provide a variably big array to be marshalled and demarshalled, and with second loop I can run many calls involving many marshalls and demarshalls.
For developing the project, I used:
* For local test, it's the code above, doing direct call to operation()
* For SOAP test, I used Axis2 over Tomcat6 as server. Server has that operation, calls original operation() method, and sends its result back as response. XML processing is done by Axis2 default configs. For client I also used Axis2, with its internal HTTP client.
* For REST test, I decided to use RESTlet and Jackson, for both server and client. On server side it's also over Tomcat6, on the same war file that Axis2 is deployed. Server receives a GET request, parses int and double parameters (URL is "/operation/{size}/{starter}" resulting in
http://localhost:9090/Benchmark/rest/operation/100/11.5), call the same operation() method, then encodes double array as a JSON array using Jackson and returns it.
* RESTlet client is based on org.restlet.resource.ClientResource with RESTlet internal HTTP client (I wanted to use Apache HTTP Client, but had trouble making it work, and when using it, it just hang and stopped with no error message or closing the process). When it receives JSON response, it's parsed by Jackson.
To build whole project I used Eclipse Juno. With it I was able to build Axis2 server and client from Eclipse and deploy it into a war. RESTlet Eclipse plugin didn't work and I had to do all the job manually, but it was just a matter to edit web.xml to bind it. Eclipse successifully deployed both Axis2 and RESTlet servers together on a unique war, and if I wanted I could have also added some
Servlet and
JSP to serve traditional HTML pages for browser users.
I will provide Eclipse project and final server war and client jar if anybody is interested, I must change package names and clean some stuff.
Well, sadly the final benchmark result was disappointing. I have previously worked on an Axi2 project where I'd query a DB for 1000 rows and send each row as an Axis2 call to a remote server, resulting in 1000 requests, and it worked fine. But when I try to do a couble hundred calls with RESTlet I start getting timeout exceptions, like:
or
These kind of Exceptions are thrown in client after many requests (a bit more than 200) and break them (of course I could add try-catch inside the loop, but meh).
And finally, if I run a small enough loop that doesn't break RESTlet, I ended up seeing that it's slower than Axis2!
Having server and client running on the same machine and a loop size=200, local call took 9 ms, all individal call having 0ms.
SOAP Axis2 calls took mostly 2 to 4 ms, a few having 20 to 40 ms delay. Total time reached 761ms!
And when it was used Jackson + RESTlet +Tomcat on server and Jackson + RESTlet + RESTlet internal HTTP client on client, most calls took 3-6ms with a few having 10-20ms delay, and a total of 1008ms!
So, what do you guys think? Anybody have some experience with RESTlet and Apache HTTP Client? Could I get better results with a better client, and not have those timeouts? Could it be some config missing? but for Axis2 I also didn't make any config and had no problem.
I tested running Axis2 with 10.000 calls and got no error at all, while RESTlet barely is able to do 200. Again, I could have added try-catch inside loop, or some refresh, or dispose and reconstruct client object, but Axis2 didn't need any of that. For Axis2 client I don't even know what it uses for HTTP client and if it shares TCP sockets for multiple calls.
My biggest concert now is that, even if I can enhance RESTlet's server or client configs, change
Tomcat or internal client for more efficient pieces of s##, or whatever can be done, REST with JSON should be way faster than SOAP with XML. My best bet now would be use Jersey or another REST implementation and it work better, so that whole blame could come to RESTlet alone, but seriously I don't wanna spend time implementing it all again without real proof of big improvements.
RESTlet website is way simple, with a few FAQ and javadoc, some of its links are pointing directly to archive.org!! And although it's being activelly developed I don't see it having a good community.
