• Post Reply Bookmark Topic Watch Topic
  • New Topic

Reading variable size buffer into an byte array  RSS feed

 
Mike Bates
Ranch Hand
Posts: 81
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am reading a buffer from a serial port that is consistently variable in size, for example, some times 8 bytes, 40 bytes, or 191 bytes. The byte array (record is 99 bytes)
with no indicator of an end point and the start is always LOO or (0x4c4f4f). Also, the very first response from the serial port is an ACK (0x06) that I need to drop.

Here is an example of the data output:


The code I started with is:


In reality, I want to append only 99 bytes to byteResult[] and forward that on to a record processor to parse the data.
I know I need to just process this at the byte level but have not been able to figure out an approach.

I am looking for guidance in approach (yes, an example would be great but guidance so I can push myself on coding it would be great as well.)
I know if I can get this figured out the solution can be used in other places as well.

Thanks
Mike
 
Steve Luke
Bartender
Posts: 4181
22
IntelliJ IDE Java Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You might want to look into using a ByteBuffer to collect data into. For example read from the source in chunks and copy the data to the ByteBuffer. Then, if the ByteBuffer is >99 bytes long:
-- consume the first 99 bytes
-- remove any of the unwanted bytes
-- replace any removed bytes with remaining bytes from the buffer
-- -- may need to re-read more bytes if not enough left in the buffer for the replacement
-- compact the buffer so leftover bytes are the beginning of the next record
-- send the 99 bytes back as a record

If you read up on ByteChannels and the NIO packages, then you might be able to more directly link your ByteBuffer to the source.
 
Campbell Ritchie
Marshal
Posts: 56584
172
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Too difficult for "beginning". Moving thread.
 
Mike Bates
Ranch Hand
Posts: 81
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Okay,

I took the idea and started coding (mix of pseudo code and code), it has not been tested yet.

Am I on the right track?


Riche, I did not realize this was to complex for beginning java and will assess before posting again there.

Thanks
Mike

 
Steve Luke
Bartender
Posts: 4181
22
IntelliJ IDE Java Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I don't think am following what you are doing. But I am pretty sure it isn't correct. First let's start with the loop where you are reading from the InputStream:


It is possible to continue to read through the entire stream this way, and then when the loop is completed numBytes would be -1. What happens if you first read 500 bytes from the Stream in one chunck, and then read 10 bytes from the stream, after which there are no more available bytes? Then the numBytes is set to 10, you lost the first 10 bytes read from the Stream and you have to reconstruct the rest by reading bytes 11->490 first, then reading 1->10. The best way to read from an input stream is like this:

This way, you will read in up to 1024 byte chunks. The actual amount read in each chunk is recorded in the numBytes variable. Inside the loop you can work with 0-numBytes to process every byte read. The loop knows to stop when numBytes = -1, which signals end of stream. So in your code, it might look something like this:


Now, I can't follow what you were trying to do in the rest of the code you posted. Here is what I would do though (and why I would go with ByteBuffer)

1) You have a number of bytes you read, which could be any amount from 0 to 1024. You could have read a small part of a 'report' of 99 bytes you need, or you may have read up to 10 'reports' with some extra bytes. So make sure you keep track of all the extra bytes that can come between chunks, dump everything into the ByteBuffer. This would mean that the ByteBuffer should be larger than your readBuffer just to make sure that no overflow occurs from a partial read immediately followed by a complete read:

2) The ByteBuffer has a bulk get() method that will fill an array. When building the report-sized array I would use this method. Something like:


3) Then it is just a matter of getting from the 'filling' stage in 1) to the 'reporting' stage in 2). This is just a matter of preparing the ByteBuffer for reading, rather than writing, then reading as many 'reports' as are present in the buffer, then making the buffer ready again for writing. Methods that will help you:
flip() - prepare for reading
remaining() - Check number of remaining bytes (if remaining() >= 99 then make report, for example)
compact() - prepare for writing

That last step is just a bit of glue, and you would probably end up calling the 'getRportArray()' method in a loop to handle cases when one read chunk reads > 99 bytes of data.

Any kind of byte checking to remove un-wanted bytes would be done on top of this (for example you can consume the InputStream up to the point where the ACK signal is reached BEFORE calling the stream reading code I provided up top).
 
Steve Luke
Bartender
Posts: 4181
22
IntelliJ IDE Java Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
p.s. I edited your first post so we don't have to scroll sideways a lot to view the post.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!