wood burning stoves*
The moose likes I/O and Streams and the fly likes Problem with reading binary file Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » I/O and Streams
Bookmark "Problem with reading binary file" Watch "Problem with reading binary file" New topic
Author

Problem with reading binary file

Jim Luckas
Greenhorn

Joined: Sep 09, 2013
Posts: 3
Dear ranchers,

Since some time, I'm wrestling with the following problem. At last, I'm turning to CodeRanch... Please stick with me; explaining it clearly will take some space.

I'm writing an Astronomy-program with my eldest son, and for that I'm trying to read in the Yale Bright Star Catalog file, containing coordinates of stars on the nightly skies. Anyone can download that database for free at http://tdc-www.harvard.edu/catalogs/bsc5.html and more specifically I'm using the binary version BSC5: http://tdc-www.harvard.edu/catalogs/BSC5 This is a binary file that contains an 28-byte header and 9110 records of 32 bytes each containing information on a particular star. Everything together is 28+9110*32=291548 bytes. There is some documentation on the header-format and the record-format available on the website (see http://tdc-www.harvard.edu/catalogs/catalogsb.html and http://tdc-www.harvard.edu/catalogs/bsc5.header.html and http://tdc-www.harvard.edu/catalogs/bsc5.entry.html). And you can see the plain-text data for the first and last record, so you can check whether you're doing it right. So far so good...

But now I do not get this data read in properly.

In fact I've been struggling (with partial success), but I'm sure there must be some beautiful, nice and clean way to do this in Java. I know about little-endian and big-endian, and it appears that the order is not consistent between integers and floats and doubles, but there are still some others problems as well.

Now I will provide you some information about the fields "right ascension" and "declination". These are coordinates comparable to our earthly latitude and longitude, but a bit different. Right ascension is measured in hours-minutes-seconds, with 24h being equal to a full circle (360 degrees or 2*PI radians). Declination is measured in degrees-minutes-seconds, and varies between +90 and -90, just like our latitude. So the example provides

right ascension: 00:05:09.900 declination: +45:13:45.00

which are represented in the binary file in hexadecimal by 8 byte each (double) as

75 98 CC 34 D3 13 97 3F and E6 19 C3 55 BF 42 E9 3F

So the question is: how to translate these byte-values to the right ascension and declination?

As an astronomer, I would like to provide some additional help :-) The right ascension and declination are expressed in radians in the binary file. I guess that 00:05:09.900 will be represented by the decimal number 0.02253655927777777777777777777778 or something in that neighbourhood, as 5 minutes and 9.9 seconds is about 0.0225 radians. Similarly, I guess that 45 degrees, 13 minutes and 45 seconds will be represented by a decimal number in the neighbourhood of 0.78939771203703703703703703703704 radians. I could be wrong, but the two hexadecimal values of 8 bytes each should somehow get translated to these decimal values.

So, I hope this is clear. I already thank you by now, for keeping up with me for so long. More thanks will follow if someone manages to decode the binary file! :-)

I something is not clear, though, please ask me and I will try to do my best.

Thank you once again!
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 18553
    
  40



You can try using the java.nio.ByteBuffer class to convert the 8 raw bytes into a primitive double. There are even controls on the class so that you can set it either litttle or big endian. However, if it doesn't work -- don't know what to say. If you got the wrong bytes, I guess you can try to debug / fix it. However, if the binary format is not in the IEEE 754 format supported by Java, you will have to write your own parser / converter for it.

Henry


Books: Java Threads, 3rd Edition, Jini in a Nutshell, and Java Gems (contributor)
Jim Luckas
Greenhorn

Joined: Sep 09, 2013
Posts: 3
Hi Henry,

Thank you for your reference to the nio-package. I didn't use it (but finally I maybe will!) because in the first place I want to learn about how this is done by swapping the bytes around. So here I show you some code that works with the star-catalog:



Works perfectly fine... However, the similar piece of code for longs/doubles does NOT work. And I don't know why... :-(



So if someone could point out what's wrong with this last piece of code, I would be very thankful!
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 18553
    
  40


Well, the data is definitely in little endian format. Just ran the eight bytes (from the first post) through the byte buffer and got the exact double floating point numbers (from the first post).

Java uses network byte order, which is big endian, so you will need to convert.

Henry
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 18553
    
  40

Jim Luckas wrote:However, the similar piece of code for longs/doubles does NOT work. And I don't know why... :-(



So if someone could point out what's wrong with this last piece of code, I would be very thankful!


Try casting the returned value from the readUnsignedByte() method, which is a primitve int, to a primitive long *before* shifting it left. Otherwise, it will be treated only as a primitive int -- which can't be shifted more that 32 bits.

Henry
Jim Luckas
Greenhorn

Joined: Sep 09, 2013
Posts: 3
Henry Wong wrote:
Try casting the returned value from the readUnsignedByte() method, which is a primitve int, to a primitive long *before* shifting it left. Otherwise, it will be treated only as a primitive int -- which can't be shifted more that 32 bits.
Henry


Thank you Henry! That solved the whole problem! I wouldn't have thought about that myself...

Thank you again!
 
Don't get me started about those stupid light bulbs.
 
subject: Problem with reading binary file
 
Similar Threads
how to convert the decimal degree format to degree minutes format of latitude and longitude?
Byte Literal
read text file and perform an operation on specific columns
Bit operatation question
question about wrappers..