• Post Reply Bookmark Topic Watch Topic
  • New Topic

Challenge to All Experts  RSS feed

 
Naveed Yahya
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm posting this problem third time, but none of expert is showing his/her expertise. Now it has
become a challenge to all who think they are experts


I'm getting StreamCorruptedException in my readObject() function. For the first time I'm able to serialize the objects(let's say two objects of Medicines class) to the file and also able to deserialize those objects successfully.
On subsequent runs I'm again able to deserialize those previously stored objects, but if I add any new Object, and try to read all objects from file then I can only read the previously stored objects successfully(that were stored on the first run) but not the last one.
while reading the last object it throws StreamCorruptedException.
here is the code:
================== Test Class =================
import java.io.*;
public class TestM
{
public static void main(String args[]) throws Exception
{
write();
read();
}
public static void write() throws Exception
{
File myFile = new File("med.dat");
FileOutputStream inFile = new FileOutputStream(myFile,true);
ObjectOutputStream OIS = new ObjectOutputStream(inFile);
Medicines objMed1 = new Medicines(1,"aa",1.1,32);
Medicines objMed2 = new Medicines(2,"bb",2.2,42);
OIS.writeObject(objMed1);
OIS.writeObject(objMed2);
OIS.close();
inFile.close();
}
public static void read() throws Exception
{
File myFile = new File("med.dat");
FileInputStream inFile = new FileInputStream(myFile);
ObjectInputStream OIS = new ObjectInputStream(inFile);
try
{
for(int ctr=0;;ctr++)
{
System.out.println("run : " + ctr);
Medicines objMed = new Medicines();
objMed = (Medicines)OIS.readObject();
System.out.println(objMed);
}
}
catch(EOFException e)
{
System.out.println("EOF Caught");
OIS.close();
inFile.close();
}
}
}
================ Medicines Class ================
import java.io.*;
public class Medicines implements Serializable
{
private int RecNo;
private String name;
private double price;
private long quatity;
public Medicines()
{
this.RecNo = 0;
this.name = null;
this.price = 0;
this.quatity = 0;
}
public Medicines(int Rec, String name, double price, long qty)
{
this.RecNo = Rec;
this.name = name;
this.price = price;
this.quatity = qty;
}
public int getRecNo()
{
return this.RecNo;
}
public String getName()
{
return this.name;
}
public double getPrice()
{
return this.price;
}
public long getQty()
{
return this.quatity;
}
public String toString()
{
return this.RecNo
+ "\t" + this.name
+ "\t" + this.price
+ "\t" + this.quatity;
}
}
================================================
 
Guy Allard
Ranch Hand
Posts: 776
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
CNR - Can Not Recreate.
This runs as expected on my system - EOFException thrown in "run 2".
My environment is XP/Pro JDK 1.4.0.
Guy
 
David Weitzman
Ranch Hand
Posts: 1365
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The format of serialized data is not intended to support this usage. Just as a quick hack, you could add this line to the end of the loop in the read() method and it'll work:

When you open an ObjectOutputStream, nobody knows how many objects will eventually be written to it. You might serialize an entire GUI with hundreds of objects. Since you can call writeObject() as many times at you want, there isn't any good way to detect the end of the object stream other then seeing the EOF. If something else is behind the serialized data the user is responsible for knowing when to stop reading in objects.
If you follow the link above you'll notice that when you open up a new object stream, it doesn't write more objects directly. First it prints a magic number (AC ED in hex) and a version code (00 05 unless you're using Java 1.1). These two pieces of data are meaningless in mid-stream and are responsible for the StreamCorruptedException.
If you really want to append to a pre-existing serialized stream you could write a custom FilterOutputStream to prevent the magic number and version code from being written, but beware -- serialized objects have special handles so that they can refer to each other. If you start appending to old data without knowing the old handles you might have handle clashes that cause unpredictable results! Because of the risk you're better off just using seperate ObjectInputStreams.
[ March 08, 2003: Message edited by: David Weitzman ]
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
[[Guy]: CNR - Can Not Recreate.
Try running the program a second time. The first time it writes and reads a new file, which works fine. But the second time it appends to the existing file, and this is what causes problems because the second header is unexpected.
The format of serialized data is not intended to support this usage.
Agreed.
[B]Just as a quick hack, you could add this line to the end of the loop in the read() method and it'll work:
[/B]
No good, I'm afraid. Doing this makes the old OIS available for garbage collection - when this happens, close() will be invoked, which will also close() the inner FileInputStream that you're still trying to read from with the new OIS. Another problem is that there's no guarantee that the old OIS didn't do some sort of readahead on the underlying stream, caching this info in an internal buffer somewhere. (I don't think this is actually an issue in the source code for 1.4.1, but there's no guarantee other JDK versions won't do something different.) Also Naveed never indicated that the objects would always be written in groups of two, so testing count % 2 == 1 is probably no good for the general case.
I think that a better hack would be to extend ObjectInputStream and ObjectInputStream to avoid writing and reading headers entirely. And while we're at it, we can encapsulate the guly logic for catching the EOFException.

I don't really like catching the EOFException, but haven't found a better way. Using available() isn't reliable, as it may return 0 for various reasons other than EOF. I tried using a PushbackInputStream to test a file for more data by reading one byte and then putting it back - but couldn't find a way to set this up in the HeaderlessObjectInputStream constructor before the super() constructor invocation. It would be nic to find another alternative though...
In any event these classes can be used fairly easily:
 
David Weitzman
Ranch Hand
Posts: 1365
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There's no documentation that says ObjectInputStreams automagically close themselves on finalization, although it shouldn't be surprising if they do. The possiblility of lookahead is valid, although probably unlikely. Anyway, use this if you insist:

No garbage collection, no loss of state. Solve the header problem however you want, although you still have the problem of object handle clashes.
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There's no documentation that says ObjectInputStreams automagically close themselves on finalization, although it shouldn't be surprising if they do.
You are correct. Moreover it seems that I was incorrect - on further study, an ObjectInputStream does not appear to close on finalization (in 1.4.1 at least). Some other streams do through (notably FileInputStream), and this fact is not documented as far as I can see. So as David says, "we shouldn't be surprised" if standard Java classes contain undocumented "features" like this - but I was wrong about this particular one. (I had discovered the FileInputStream feature in the past, and thought it was true for all streams.)
David's trick of making one long chain of ObjectInputSteams in order to avoid finalization closure (if it did happen) is clever. Naturally it could have some problems in term of performance and memory usage if the number of serialized objects is large. But we can't always have everything in one solution, and this is certainly at least good for "a quick hack" as it was originally described - and depending on the exact user requirements, this may be entirely adequate.
I think that all the solutions described so far (including mine) have the problem that they rely on undocumented behavior of ObjectOutputStream and ObjectInputStream. Once we have more than one header in a file (or no headers, as in my code), we're no longer following the Object Serialization Specification, and there are any number of ways this might bite us in the future if the OOS/OIS implementations change. As David noted, OOS/OIS were not designed for this usage, and so I think the "proper" solution to the original problem is to rewrite the entire file rather than simply appending onto the end. Of course this could get to be a notable performance hit if the number of objects is large. This problem can be alleviated by using a set of files, rather than one big file. Put them all in one directory and give them sequential names, like
med0001.dat
med0002.dat
med0003.dat
etc. You can either put each update in a separate file, or "append" (by rewriting) to just the latest file - but if the file size exceeds a certain value, create a new file rather than appending to the last one. You can mix & match these schemes in various ways too - e.g. use separate files for each individual update, but then periodically "compact" the files by reading all the little files and rewriting them as one big file (or a small number or largish files). Depends how much time you want to spend on the whole thing, and how much need there really is to optimize this process.
It's also worth noting that when using any of these schemes for rewriting a file rather than simply appending to it, we can also solve the problem of knowing when a file ends without catching EOFException. If you know how many objects will be in a file at the time you start to write it, you can simply store that number at the beginning of the file (after the header) using writeInt(). The OIS can use readInt() to recover this info, and then loop the appropriate number of times to read each object. I think this is preferable to catching EOFException - but it only works if you know how many objects there will be at the time you write the start of the file. Of course, catching EOFException isn't the end of the world either - it just looks a bit inelegant.
[ March 10, 2003: Message edited by: Jim Yingst ]
 
Maulin Vasavada
Ranch Hand
Posts: 1873
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi all,
i guess i had similar case...i got around it by creating a Vector of objects and dumping that vector to the file via ObjectOutputStream instead of writing individual objects...
i can see issues of re-loading whole vector and re-dumping it everytime afresh which can be a problem if we have thousands of objects in it but for me it was really not that big (i don't expect more than 10-15 string objects in my vector so i could afford going expensive way) ...
just a suggestion..
regards
maulin.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!