• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Problem with reading binary file

 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Dear ranchers,

Since some time, I'm wrestling with the following problem. At last, I'm turning to CodeRanch... Please stick with me; explaining it clearly will take some space.

I'm writing an Astronomy-program with my eldest son, and for that I'm trying to read in the Yale Bright Star Catalog file, containing coordinates of stars on the nightly skies. Anyone can download that database for free at http://tdc-www.harvard.edu/catalogs/bsc5.html and more specifically I'm using the binary version BSC5: http://tdc-www.harvard.edu/catalogs/BSC5 This is a binary file that contains an 28-byte header and 9110 records of 32 bytes each containing information on a particular star. Everything together is 28+9110*32=291548 bytes. There is some documentation on the header-format and the record-format available on the website (see http://tdc-www.harvard.edu/catalogs/catalogsb.html and http://tdc-www.harvard.edu/catalogs/bsc5.header.html and http://tdc-www.harvard.edu/catalogs/bsc5.entry.html). And you can see the plain-text data for the first and last record, so you can check whether you're doing it right. So far so good...

But now I do not get this data read in properly.

In fact I've been struggling (with partial success), but I'm sure there must be some beautiful, nice and clean way to do this in Java. I know about little-endian and big-endian, and it appears that the order is not consistent between integers and floats and doubles, but there are still some others problems as well.

Now I will provide you some information about the fields "right ascension" and "declination". These are coordinates comparable to our earthly latitude and longitude, but a bit different. Right ascension is measured in hours-minutes-seconds, with 24h being equal to a full circle (360 degrees or 2*PI radians). Declination is measured in degrees-minutes-seconds, and varies between +90 and -90, just like our latitude. So the example provides

right ascension: 00:05:09.900 declination: +45:13:45.00

which are represented in the binary file in hexadecimal by 8 byte each (double) as

75 98 CC 34 D3 13 97 3F and E6 19 C3 55 BF 42 E9 3F

So the question is: how to translate these byte-values to the right ascension and declination?

As an astronomer, I would like to provide some additional help :-) The right ascension and declination are expressed in radians in the binary file. I guess that 00:05:09.900 will be represented by the decimal number 0.02253655927777777777777777777778 or something in that neighbourhood, as 5 minutes and 9.9 seconds is about 0.0225 radians. Similarly, I guess that 45 degrees, 13 minutes and 45 seconds will be represented by a decimal number in the neighbourhood of 0.78939771203703703703703703703704 radians. I could be wrong, but the two hexadecimal values of 8 bytes each should somehow get translated to these decimal values.

So, I hope this is clear. I already thank you by now, for keeping up with me for so long. More thanks will follow if someone manages to decode the binary file! :-)

I something is not clear, though, please ask me and I will try to do my best.

Thank you once again!
 
author
Posts: 23951
142
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator


You can try using the java.nio.ByteBuffer class to convert the 8 raw bytes into a primitive double. There are even controls on the class so that you can set it either litttle or big endian. However, if it doesn't work -- don't know what to say. If you got the wrong bytes, I guess you can try to debug / fix it. However, if the binary format is not in the IEEE 754 format supported by Java, you will have to write your own parser / converter for it.

Henry
 
Jim Luckas
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Henry,

Thank you for your reference to the nio-package. I didn't use it (but finally I maybe will!) because in the first place I want to learn about how this is done by swapping the bytes around. So here I show you some code that works with the star-catalog:



Works perfectly fine... However, the similar piece of code for longs/doubles does NOT work. And I don't know why... :-(



So if someone could point out what's wrong with this last piece of code, I would be very thankful!
 
Henry Wong
author
Posts: 23951
142
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Well, the data is definitely in little endian format. Just ran the eight bytes (from the first post) through the byte buffer and got the exact double floating point numbers (from the first post).

Java uses network byte order, which is big endian, so you will need to convert.

Henry
 
Henry Wong
author
Posts: 23951
142
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jim Luckas wrote:However, the similar piece of code for longs/doubles does NOT work. And I don't know why... :-(



So if someone could point out what's wrong with this last piece of code, I would be very thankful!



Try casting the returned value from the readUnsignedByte() method, which is a primitve int, to a primitive long *before* shifting it left. Otherwise, it will be treated only as a primitive int -- which can't be shifted more that 32 bits.

Henry
 
Jim Luckas
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Henry Wong wrote:
Try casting the returned value from the readUnsignedByte() method, which is a primitve int, to a primitive long *before* shifting it left. Otherwise, it will be treated only as a primitive int -- which can't be shifted more that 32 bits.
Henry



Thank you Henry! That solved the whole problem! I wouldn't have thought about that myself...

Thank you again!
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic