Win a copy of The Java Performance Companion this week in the Performance forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Write Binary File in Java to be read by a C Program and Vice Versa

 
Freddy Wong
Ranch Hand
Posts: 959
Eclipse IDE Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Suppose that I'm trying to create a binary file created in Java to be read by a C program. Below is the code.



On my PC, it printed like this

unsigned byte=12
unsigned short=53760
unsigned int=4

which is incorrect.

Similarly I'm trying to create a binary file created in C to be read by a Java program. Below is the code.




On my PC, it printed like this

unsigned byte=12
unsigned short=10619
unsigned int=13763584

which again is incorrect.

Any pointers what might go wrong here?

Thanks in advance.
 
Ernest Friedman-Hill
author and iconoclast
Marshal
Pie
Posts: 24211
35
Chrome Eclipse IDE Mac OS X
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You're assuming that sizeof(struct) is the sum of the sizes of the members (7 bytes) and they follow each other one after the other with no gaps; this is actually unlikely. Usually structs are padded to 4 or even 8-byte boundaries, depending on the compiler. I think most compilers these days would pack that struct into 8 bytes: one for the char, an unused byte, two for the short, and four for the int. Therefore when you read the data, it's aligned incorrectly, so you get weird numbers; and likewise with the Java code (which doesn't confirm that it reads the number of bytes it expects, either, and gets one more byte than the 7 it wants.)

So the sad truth is that the layout of the C struct is compiler dependent, and you'll have to determine what your compiler does in terms of padding, and then match it with your Java program. Once you do that, you should be fine.
 
Stephan van Hulst
Bartender
Pie
Posts: 5912
66
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You can make things much easier for yourself by providing some sort of custom OutputStream.


 
Freddy Wong
Ranch Hand
Posts: 959
Eclipse IDE Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Ernest, I guess that explains why my code doesn't work and thanks Stephan for the code. It's a much nicer code that mine
 
Akil Kumar
Ranch Hand
Posts: 83
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ernest Friedman-Hill wrote:You're assuming that sizeof(struct) is the sum of the sizes of the members (7 bytes) and they follow each other one after the other with no gaps; this is actually unlikely. Usually structs are padded to 4 or even 8-byte boundaries, depending on the compiler. I think most compilers these days would pack that struct into 8 bytes: one for the char, an unused byte, two for the short, and four for the int. Therefore when you read the data, it's aligned incorrectly, so you get weird numbers; and likewise with the Java code (which doesn't confirm that it reads the number of bytes it expects, either, and gets one more byte than the 7 it wants.)

So the sad truth is that the layout of the C struct is compiler dependent, and you'll have to determine what your compiler does in terms of padding, and then match it with your Java program. Once you do that, you should be fine.


Hi Ernest,

Normally these padding or packing should be done at the C program. For instance if I were to read or write some data from the Java program through the socket to the C server program. Doing all these conversions in the Java program wouldn't be expensive?

As per standard how the data should be sent from the C program? Should it be in network byte order or it can be little endian?

Thanks,
 
Jesper de Jong
Java Cowboy
Saloon Keeper
Pie
Posts: 15369
40
Android IntelliJ IDE Java Scala Spring
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It doesn't really matter if you do things like adding or skipping padding at the side of the C program or at the side of the Java program - the important thing is that the two programs understand each other. Doing these things in Java is not much more expensive than in C.

There is no common standard that says you have to send the data in big or little endian order.

There are standard protocols for exchanging data, and if I had to do something like this, I'd first investigate those protocols before trying to re-invent the wheel myself. For example, Google protocol buffers look like a good way to do this. You define in a special specification language what you data looks like, and Google's tool generates some code in C++, Java or Python that you can use to make the programs talk to each other. You don't have to worry about the low-level details, such as what the format of the bytes that are sent across exactly is.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic