• Post Reply Bookmark Topic Watch Topic
  • New Topic

What problem is solved by ByteBuffer  RSS feed

 
s ravi chandran
Ranch Hand
Posts: 579
6
Java jQuery
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

I work on a project that involves high volume data transactions. I would like to know what problem does ByteBuffer solves and if I can use it for my component.
 
Stephan van Hulst
Saloon Keeper
Posts: 7993
143
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
ByteBuffer is a tool that can be used to manipulate big blocks of binary data. It pairs a piece of memory with some variables that do bookkeeping, so you can perform relative operations without having to worry that you are reading from or writing to the wrong memory locations. It also provides some methods that will interpret the binary data as Java primitives.

For instance, here's how you could read some primitives from a file written on a Little Endian platform:

As you can see, there is quite some bookkeeping involved in this code, and you have to mess with low level bit operations.

Here's what you can do with ByteBuffer:

Another advantage is that ByteBuffer can be allocated directly in kernel space using allocateDirect(). When reading data from a file that way, you can read Java primitives directly from the block of memory that the disk controller writes to, instead of having that memory copied to user space first. This is actually only useful in very specific cases, so you'll have to profile your application to see if it will improve performance.
 
Rob Spoor
Sheriff
Posts: 21135
87
Chrome Eclipse IDE Java Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:It pairs a piece of memory

Or disk, through MappedByteBuffer.
 
s ravi chandran
Ranch Hand
Posts: 579
6
Java jQuery
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If I have a message packet serialized to byte[] which is of small size, but the volume of this packet will be high( max range can reach to 20-10K packets per second). Will using bytebuffer help in reducing the latency in read / write operations?
 
Jesper de Jong
Java Cowboy
Sheriff
Posts: 16060
88
Android IntelliJ IDE Java Scala Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That's hard to say in general, without knowing what your code does now exactly with byte arrays. It depends how big the packets are and what exactly has to be done with them, but processing 10-20K packets per second should not be a problem for a modern computer.
 
s ravi chandran
Ranch Hand
Posts: 579
6
Java jQuery
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Structure of the class is this:



This object is currently being converted to ByteMessage and pushed to a queue which is consumed by a Subscriber which will parse it and process the data.

Normal scenario the rate of publish is 10-20 entries, but during stress test it can reach millions.

Is it a scenario where bytebuffer might improve the performance?

 
Jesper de Jong
Java Cowboy
Sheriff
Posts: 16060
88
Android IntelliJ IDE Java Scala Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I can't say because I don't know how you are converting this to a ByteMessage.

But more importantly: Do you actually know that you have a performance problem in the part where it converts this object to a ByteMessage? If not, then it's probably not worth the time trying to change it.
 
Stephan van Hulst
Saloon Keeper
Posts: 7993
143
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Going by your class name, it doesn't appear to me that a ByteBuffer will be very useful to you. ByteBuffer is mostly interesting when dealing with high volume low level operations, such as in video or real-time processing.

You say your project involves high volume data transactions. Maybe you can elaborate on that a bit?
 
s ravi chandran
Ranch Hand
Posts: 579
6
Java jQuery
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is for a financial project where we have stock exchange market data updates being published as and when we receive it.

For stress testing of our component, the volume of entries can reach  millions of entries per second.

 
Stephan van Hulst
Saloon Keeper
Posts: 7993
143
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
How is the data published? Are you talking to a web service?
 
Henry Wong
author
Sheriff
Posts: 23295
125
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:
Another advantage is that ByteBuffer can be allocated directly in kernel space using allocateDirect(). When reading data from a file that way, you can read Java primitives directly from the block of memory that the disk controller writes to, instead of having that memory copied to user space first.


It doesn't have to be kernel space. A direct byte buffer is also great for communicating with C libraries that shares a buffer with the Java side. Since direct byte buffers are not affected by the garbage collector... meaning that it doesn't move in memory... both sides can share data easily (and without taking an expensive JNI call).

Henry
 
s ravi chandran
Ranch Hand
Posts: 579
6
Java jQuery
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Message is published through JMS.

So, is it a realistic expectation that if we have a stress test scenario, we would benefit from using ByteBuffer ( Direct / Non Direct ).
 
Stephan van Hulst
Saloon Keeper
Posts: 7993
143
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That's exactly what your stress test is for in the first place. Try conventional methods. If they are too slow for your stress test, you try non-conventional methods.
 
Rob Spoor
Sheriff
Posts: 21135
87
Chrome Eclipse IDE Java Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I don't think a ByteBuffer is the tool if you just need to convert an object to byte[] (which is what BytesMessage needs) and back. BytesMessage already has the methods you need to write and read your String and primitive fields. For the enums, I'd write and read the name (using valueOf to convert back).

I've actually done the same in a project once - I just have conversion from ByteMessage to my classes and vice versa, that uses the read and write methods of BytesMessage itself.
 
s ravi chandran
Ranch Hand
Posts: 579
6
Java jQuery
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Rob Spoor wrote:I don't think a ByteBuffer is the tool if you just need to convert an object to byte[] (which is what BytesMessage needs) and back. BytesMessage already has the methods you need to write and read your String and primitive fields. For the enums, I'd write and read the name (using valueOf to convert back).

I've actually done the same in a project once - I just have conversion from ByteMessage to my classes and vice versa, that uses the read and write methods of BytesMessage itself.

So there is no significant benefit as long as I am using it for ByteMessage?
 
Rob Spoor
Sheriff
Posts: 21135
87
Chrome Eclipse IDE Java Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
At least I don't see it.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!