• Post Reply Bookmark Topic Watch Topic
  • New Topic

The fastest log in the West

 
Mark Herschberg
Sheriff
Posts: 6037
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm writing a general test harness. It's more than just a simple harness, because it can be used to performence test our servers (by acting like a client and banging away at it). Specifically, it will be used to do controlled timing tests, such that I can set a certain load/request frequency. Clearly, it will to need log of events. Because I'm doing timing tests, this log must be very nible, so as not to disturb the timing.
I don't need log levels. I may need channels, but think I will probably include channel information as I write the log, and can parse the log later to divide it up into channels. There's the obvious, creating a large StringBuffer and simply writing to it (memory should not be an issue while running the test).
Any other thoughts, ideas, or warnings?

--Mark
 
Yuri Gadow
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You might look into handing each item off to asynchronously, to another thread, process or server or some combination thereof. Of course it depends on how much log info you want in a run, in between log writes. If it's going to remain small, a StringBuffer would seem to be the way to go. On the other hand, if performance were top priority, some testing of appending versus pushing Strings onto an array might be in order. Though you might run into problems with GC affecting your timing, depends on the VM and how many. Stopwatches and interns don't do half bad either.
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hmmm... I suppose you could try replacing most of the text messages with numeric codes - call them event types. Send these to a DataOutputStream wrapped around a ByteArrayOutputStream. (I assume that a BufferedOutputStream would be redundant here, but you could test to be sure.) Or maybe it would be faster to set values in a byte array directly, if you want to try that. Each log entry could consist of writing an int for system time, an int for event type, and optional additional arguments - the expected number and types of arguments could be derived from event type. Likewise channel could be derived from event type. I'd avoid using Strings as much as possible - writeUTF() will work if necessary, but if you must have a lot of String info in the logs (i.e. changeable info that cannot be simply replaced by an int event code) then I'm guessing the original StringBuffer idea will be faster. Certainly the StringBuffer is more flexible - but if you really want the logging to be lean and mean, this may be the way to go. Naturally, you'd want to parse this byte array later, converting it into something more human-friendly.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!