How to read/write directly to/from a class???

[Prosenjit Banerjee]

Dear friends,
Can anybody please tell me how to read/write directly to/from a class?

Actually I want to read/write data just as I do in C using read() write() functions. There I just send the address of the structure variable and the data gets filled into the variable or gets written from the variable to the file.
Precisely, I want to pass the object variable and want it to be filled with data coming from the current position of the file.

Is it possible? If yes then please tell me how.

Thanks.

[Nitesh Kant]

Are you looking at persisting a java object on a file? If yes, we call it Serialization in java.
Using serialization you can write any java object that implements an interface java.io.Serializable on a stream.
Hope this answers your question.

[Prosenjit Banerjee]

Thank you Nitesh. But that does not solve my problem.

So, let me explain. I want to read a BMP file. In C I used to create two different structures and then by issuing only two read() functions I could fill whole of the structure variables.

In serialization, I suppose, it needs to remain some extra meta data info that helps to determine the size of each object. But a binary file such as a BMP file there remains only the data.

So, one way is to read each of the elements one by one (first 2 bytes, then one long varable and so on). But can't I read whole of the structure using just one statement?

Am I clear now. Please tell me.

[Nitesh Kant]

Ok, so you want to read/write an image file.
There is an api in java javax.imageio that provides classes to work on images.
Here is a relevant faq.
I have not worked on it so i could not help you much in that regards however, it must be as simple as reading data from any other stream.

[Prosenjit Banerjee]

Thanks but again you misunderstood me.

I not only want read an image file but I want to read any binary file whose file format is known to me.

I am giving an example.

Suppose the format of a file is as follows

Address | No. of bytes
----------------------
00      | 2
02      | 4
06      | 1
07      | 4

Now, following is a C code that helps me read such a file:

typedef struct
{
   int  a ;
   long b ;
   char c ;
   long d ;
} Data ;
 
Data data ;
 
FILE *fp = fopen("filename.dat", "r") ;
 
//Reading data in one statement
fread(&data, 1, sizeof(Data), fp) ;
 
// Printing the data:
printf("%d, %ld, %c, %ld\n", data.a, data.b, data.c, data.d) ;
 

Now, I want to write similar code in Java. Is it possible?

[ November 21, 2007: Message edited by: Prosenjit Banerjee ]
[ November 21, 2007: Message edited by: Prosenjit Banerjee ]

[Nitesh Kant]

Wohooo ....
you seem to introduce a new requirement in every post
This function in c looks interesting. How does it chop of the headers that you have given in the file i.e.:

Address | No. of bytes
----------------------

How do you represnt an image file (as you specified a BMP file) as a struct?
How is the encoding of the binary file specifically an image file handled?

I am pretty skeptical about one function in c handling all the type/formats of files without any special handling, but in java, as far as i know you can not seamlessly handle an image file at one hand and a formatted file, (as you have shown), on the other hand. Reason being the file you have shown has a very specific format that only your application understands. Image files have a different format that needs special handling.
You would need some code that can detect the type of the file and then trigger an appropriate handler. This handler can be simply a java API or a complex class that you write.
Serialization can help you if you do not have a specific file format. Serialization is a protocol and has it's own format of writing data on the streams. If you use serialization you do not have to do anything extra but just write the object using ObjectOutputStream and read it using ObjectInputStream.

[Vlado Zajac]

This is not possible since the layout of objects (or any other data) in memory is not specified and there is not way to access memory directly.

[Ulf Dittmer]

Originally posted by Nitesh Kant:
How do you represnt an image file (as you specified a BMP file) as a struct? How is the encoding of the binary file specifically an image file handled?

I am pretty skeptical about one function in c handling all the type/formats of files without any special handling.

Your skepticism is misplaced In C it works fine the way Prosenjit mentioned, but only for that one particular file type - you need a struct declaration like that for every file type you wish to access.

The Java equivalent to C structs are classes, so you'd need a class that has all these fields to represent the image. What Java does not have is an I/O capability similar to the one on C that fills in fields of a class in the order of the bytes of the input stream. That's why for just about any file format reader in Java, there's a class extending (File)InputStream to go along the class representing the actual object. That class encapsulates the knowledge of which values read from the stream go into which fields.

[Prosenjit Banerjee]

Dear Dittmer,
You have clearly understood my point. Thank you very much.

Can you show how to do by converting my code from C to Java? It will really help me a lot.
[ November 22, 2007: Message edited by: Prosenjit Banerjee ]

[Ulf Dittmer]

It is possible to write code that examines a class to see what kind of fields it includes, and maps the bytes of the InputStream accordingly, and even populates an object of that class with the proper values. But that's a lot of low-level code that I for one wouldn't want to write. Might be a good exercise to learn about class files and handling binary data, though.

My favorite approach to reading structured file formats involves creating a lexer using a tool like JFlex.

What's the best approach depends on the complexity of the file format.

[Nitesh Kant]

Originally posted by Ulf Dittmer:

Your skepticism is misplaced

With the miniscual C knowledge i have, i am not surprised hearing this

Prosenjit,
Is the file that you are reading created by some other process?
Is the file format fixed?
If not, then working in java, you might as well use Serialization. That will give you C like facility where in just using a single method on ObjectInput/OutputStream you can read/write the java class to/from a file.
Ofcourse, the file format will change and it will not be human readable/editable.

However, if the file format is fixed then you have to put in custom code to read/write from/to that file.

[Prosenjit Banerjee]

Thank you Nitesh.
The point is, if any kind of file (be it a .BMP, .ZIP, .ico or any file whose format I already know) I want to read/write at low level then how to do?

Dittmer seems to know exactly what I should do but is not willing to take that effort for me only (I surprised to know that I am the only person having such a query!). But if he could write such a code then so many of my friends and acquaintances will be benefitted from it.

I can ofcourse read/write such a file, but only one data type at a time. So, various formats the code for reading/writing the fill will be different and I don't want that. I only want to write a method what will read/write a file's format (or data) depending upon the structure (instance or a class for Java) passed through it.

I don't think this is impossible in Java. But cannot do it in Java. Please help me.

[Jim Yingst]

Java doesn't have a way to read all the fields at once, but there are certainly ways to read the fields one at at time. I recommend looking at the class DataInputStream. This is pretty simple to use for most common data formats.

[B][Prosenjit]: Suppose the format of a file is as follows

Address | No. of bytes
----------------------
00      | 2
02      | 4
06      | 1
07      | 4

[/B]
You can do something like this:

DataInput dataIn = new DataInputStream(new FileInputStream(file));
 
short a = dataIn.readShort();
int b = dataIn.readInt();
byte c = dataIn.readByte();
int d = dataIn.readInt();
 
// do something with the data...
 
dataIn.close();

What you do with the data after reading it is up to you, or course. That may vary drastically depending on the format.
[ November 24, 2007: Message edited by: Jim Yingst ]

[Ulf Dittmer]

There are a couple of points in your post that bear elaboration.

The point is, if any kind of file (be it a .BMP, .ZIP, .ico or any file whose format I already know) I want to read/write at low level then how to do?

The technique I outlined above for Java will not work for all file formats, only for simple ones. I'm not sure about BMP and ICO, but definitely not for ZIP. The same is true for a C implementation based purely on structs (like the one you mentioned before). The reason is that a fixed number of fields in a Java class is not sufficient for file formats that have a variable format (like ZIP). You don't even know how many files to read from the zip file until you actually open it, and have iterated through its directory. I suppose it's possible to extend the solution I outlined above with including arrays to other nested objects, but that would get tricky very fast.

What's more, there's the problem of fixed-length data fields. Let's say the file format contains in one spot character data that is terminated by some specific character. How would you teach the Java code that reads it in a general way that it should read until it finds a particular character? There are also file formats where the character that terminates strings can be changed midway through a file (HPGL is an example).

Dittmer seems to know exactly what I should do but is not willing to take that effort for me only (I surprised to know that I am the only person having such a query!). But if he could write such a code then so many of my friends and acquaintances will be benefitted from it.

First of all, the name is Ulf. If you're uncomfortable referring to someone by his first name, the polite thing is to do is to prefix his last name by "Mr". (yes, I'm a guy :-)

Indeed, I wouldn't want to write such code no matter how many people benefitted from it. It's the kind of problem where a generalized solution breaks down every time you apply it to a new problem (i.e., file format).

I can ofcourse read/write such a file, but only one data type at a time. So, various formats the code for reading/writing the fill will be different and I don't want that. I only want to write a method what will read/write a file's format (or data) depending upon the structure (instance or a class for Java) passed through it.

My advice is to give it up. There's a reason that no such code exists - it's a very complex undertaking. Tackle it one file format at a time with code that deals specifically with that one file format. Use of a lexer (like the one I mentioned before) can simplify this if you're comfortable with regular expressions.
[ November 24, 2007: Message edited by: Ulf Dittmer ]

[Prosenjit Banerjee]

Thank you very much Mr. Dittmer (kindly forgive me for my ignorance about your name). It is only my problem that as know very little of English I can not make you guys understand my problem.
I know that each file has it's own format.
I know that a single function cannot read any file and I have to write a seperate for each file.
I know that after reading bytes from a file I have to analyse them.

But if I know the structure of the file why it is not possibel to write a function that accepts the structure and read the file accordingly?

Just as I cited an example, suppose the format of a file is as follows:

    Address | No. of bytes
    ----------------------
    00      | 2
    02      | 4
    06      | 1
    07      | 4

so in C, I can use the fread() a function is already written to read the file as follows :

   typedef struct
   {
      int a ;
      long b ;
      char c ;
      long d ;
   } Form1 ;
 
   Form1 var1 ;
   FILE *fp ;
   ....
   ....
   fread(&var1, 1, sizeof(Data), fp) ;

Again if the format of the file is:

    Address | No. of bytes
    ----------------------
    00      | 1
    01      | 1
    02      | 2
    06      | 4
    07      | 1
    07      | 1

Now, in C:

   typedef struct
   {
      int a ;
      long b ;
      char c ;
      long d ;
   } Form2 ;
 
   Form1 var2 ;
   FILE *fp ;
   ....
   ....
   fread(&var2, 1, sizeof(Data), fp) ;

So, fread() function is such a function that reads a chunk of several bytes at once and fills the structure variable.

I just want to know that whether Java already has such a function as fread()? If yes then what is that function in Java? If not then can I create such a function in Java? If yes then how?

Please try to help me. Please don't give up.

[Ulf Dittmer]

I just want to know that whether Java already has such a function as fread()?

No.

If not then can I create such a function in Java? If yes then how?

I hinted at that above. You can create a class that has the required fields, and then analyze that class at runtime to see what fields it has, and read from the input stream accordingly, in the way Jim showed in his post. That will be brittle in the face of class changes. (One reason is that you'd be relying on the fields to be in the class file in the same order they are in the source file, and to be returned by the class file accessor methods in that same order. I'm not sure if that's guaranteed to work that way, it may be JVM-specific.) I would advise you not to pursue that idea.

[Nitesh Kant]

Originally posted by Prosenjit Banerjee:

Dittmer seems to know exactly what I should do but is not willing to take that effort for me only

Why will someone write the code for you in here? We are here to help people and give them directions and not to do their work.
I think everyone reading these posts know what you are trying to do but the point is that how you are trying to do is not correct and error prone(As Ulf pointed out)

Originally posted by Prosenjit Banerjee:

(I surprised to know that I am the only person having such a query!).

Well, you are not the only person who reads different type of files but they do not do it this way.

Originally posted by Prosenjit Banerjee:

I can ofcourse read/write such a file, but only one data type at a time. So, various formats the code for reading/writing the fill will be different and I don't want that. I only want to write a method what will read/write a file's format (or data) depending upon the structure (instance or a class for Java) passed through it.

As Ulf earlier pointed out, its not that simple. It gets tricky when the object to be read is complex. The object may have a reference to another class that may have references to another class and so on. And to make it complex, there may be a circular reference!
Moreover, by reflection you can not find the fields in the orde they appear in the class definition. This is what the javadocs for getFields() in Class says:

The elements in the array returned are not sorted and are not in any particular order.

I would say that you read the java Serialization specifications, it will tell you the complexities associated with this work.

Jim,
The DataInputStream is intended to be used in tandem with DataOutputStream. It can be used to read other file formats but then it depends how are the bytes written in the stream i.e. the high byte is written first or low byte. So, it will be error prone.

[Prosenjit Banerjee]

Originally posted by Nitesh Kant:
Why will someone write the code for you in here? We are here to help people and give them directions and not to do their work.

I have not requested anybody to write the whole code. I just requested to write some statements to show the examples.

Many people (including me) in the Javaranch write chunks of code to HELP other people, and this is one of the ways to help.

This is not my job but just my query. As I am new in java, I am trying hard to learn new things, that's all.

But thanks to you guys again to guide me. I think I have got my answer and am already writing such a code that suits my needs. Will post it very soon.
[ November 25, 2007: Message edited by: Prosenjit Banerjee ]

[Jim Yingst]

[Nitesh]: The DataInputStream is intended to be used in tandem with DataOutputStream. It can be used to read other file formats but then it depends how are the bytes written in the stream i.e. the high byte is written first or low byte. So, it will be error prone.

Yes, it depends on the file format. For big-endian formats it will generally work just fine; for little-endian formats, no. Care must also be used in reading strings, which can have many formats besides the one used by readUTF(). Still, I was more interested in suggesting a general strategy, not complete details. If one proceeds down this road, one can modify the data read in as needed. The whole process can be error-prone, depending on the subtle details of the file formats, as Ulf has been suggesting. I do not propose to try to explain all possible details in one post.

But, aside from methods used in DataInputStream, for some formats it may well be easier to just read everything into a byte[] and then write a set of methods for interpreting sets of 2, 4, or 8 bytes as primitives. Or use a java.nio.ByteBuffer, which has more methods than DataInputStream, including methods to set the byte order if little endian is required. This can be fairly straightforward for well-defined fixed-length formats such as Prosenjit seems to be imagining; for more complex file formats, there will doubtless be more complications.