• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Opinions On Reading Data File

 
Anthony Watson
Ranch Hand
Posts: 327
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm just starting the assignment, and I'm already conflicted on how to proceed on reading the data file. Part of me thinks that I should hard code the meta data (such as total record length and individual field lengths) as static constants in my Data class. Another part says that doing so is bad because when the meta data changes, I would have to change my data class. Thus, I was considering reading in the meta data values from the file and using them to read in the records into a String array. The problem with this though is that any small change in the structure of the data file will cause methods like seek(4) to fail, and I'll still have to change my code anyway. This second way would also be harder for another programmer to maintain because its a little more complex logic. Any opinions on this subject?
 
Denis Spirin
Ranch Hand
Posts: 72
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello Anthony
Hardcoding header data (record length, offset etc) is a bad idea.
If you hardcode this data it means that you must have read them somehow. A number of questions arises:
1. How did you read them?
2. What is header for? You will simply skip it, and therefore there will be no need in header
at all.
3. What if there will be need to change, say, length of some record? If you do not hardcode metadata all you will have to do is to change dbfile (header and data); there will be no need to change DAO. In hardcoding approach you'll have to change code also.
The problem with this though is that any small change in the structure of the data file will cause methods like seek(4) to fail, and I'll still have to change my code anyway.

- thats not correct. Your DAO must be able to get all nessessary info from header.
Sure there is some stuff you'll have to hardcode, such as magic number.
Regards
 
Anthony Watson
Ranch Hand
Posts: 327
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If the magic cookie value was changed to 8 bytes instead of 4, that would throw off all my data values and I'd have to change my code to account for that. Thus, I was considering using the hardcoded header value approach because it is simpler logic, which is supposed to be preferred by the exam graders. Any other people with opinions on this?
 
Denis Spirin
Ranch Hand
Posts: 72
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
In spite of "simplicity" of hardcoded solution (I dont think it will be much simpler) you should notice that
1. You may get 40 points on Data class. Or you may not. Hardcoding is bad programming practice, it should be avoided as much as possible. Hardcoding means: less reusable, difficult to maintain. Yes, in instructions it is said that code should be simple. But not in this way. All you have to do is just parse header and extract information you need. If you wont do it, examiner will be very surprised. Youre gonna be SCJD, rite?
2. Magic cookie must be hardcoded (or taken from properties file) - its only purpose to "verify" data. Theres no reason to change it. But there might be need to change data structure.
Then you will have to change your code and distribute it to CSRs again. Etc.
Your user interface should be designed with the expectation of future functionality enhancements, and it should establish a framework that will support this with minimal disruption to the users when this occurs.

This is said about GUI, but I think this criteria is used to mark whole project.
[ April 30, 2004: Message edited by: Denis Spirin ]
 
Richard Everhart
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I've created a separate metadata class. The data class refers to this class in order to do things like determine the length of a record or to get a string of null charaters having the length of a particular field. Even though the length and names of the given fields are part of the database file in the header section, I've decided to hard code these values.
I agree that hardcoding in many cases is not the best way to go, but in this case I don't think it's all that bad. The reason is this is not some 3rd party database file - its ours. That is, if the file changes we will know. Database administrators know the schema of the tables they create, so, similarly we know the structure of this file. It could be argued that any change to the database file would require a code change - adding/removing a field or changing a field's length etc. So, simply being able to determine the length of field dynamically (reading it from the file's header section) would not solve the problem of having to recode and recompile when the schema changes - such changed could be made anywhere.
Therefore, my use of a metadata class I mentioned earlier. This isolates the data from its metadata, allowing them to vary from one another.
Disclaimer: I could be totally wrong. If so then please let me know.
Rich
 
Hanna Habashy
Ranch Hand
Posts: 532
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi:
I have just downloaded the assignment, and I haven't put much thoughts to it yet. My initial thought to this issue is the following:
When the application startup, or when a user choose a database file, a utility function should run and read the header data and the first record, instanciate an instance of a constant class, and intialize the constants with the header and metadata of the fields. In this way, it is not hard coded, and it is simillar to hard coded it in a constant class. The only difference is the intialization of the metadata is done it runtime.
any thought on this approch
 
Richard Everhart
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Hanna,
I like youre take on this problem. However, I'm wondering how you can use a constant object in the way you describe. Doing as you suggest assumes two things 1) contants objects are static finals and 2) since you're getting the information you need from the file, you don't know ahead of time how many of these fields you will need, in which case you can't have static finals. Now, you could have a predetermined number of blank finals - one for each field in a record. But, then they'd need a name. So, at this point you might was well just hard code their length too. An alternative may be to create some data structure for these fields: a Map where a field is keyed by its name or a List of fields, or both. Then read in the data and create the field objects as you describe.
Rich
 
Hanna Habashy
Ranch Hand
Posts: 532
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi Richard:
I was thinking about a type of collection. A hash map for example. first I read the number of field in a record from the metadata. Then I loop through the schema description to get the metadata of every field. In each loop, I add a key value pair in the map coresponding the the name of the field, and its length.
I am not sure if this approch will work. I have never worked with this type of the database. I am more familiar with RDB or text file.
 
Richard Everhart
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That's exactly what I was thinking, Hanna. Right now, I am using hardcoded constant objects, and I'm storing them in a HashMap. In light of this conversation, I'm going to change my code to be more dynamic and read the values directly from the header section of the database file.
Rich
 
Yanxin Zhao
Ranch Hand
Posts: 65
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have just downloaded the assignment,too. I have two problems about the db file reading. one is :How to read a magic cookie? the other is: when I read a non-number value of head from db file with readUTF() method, the VM will gives me EOFException. my code is that
System.out.println(db.readInt());
System.out.println(db.readShort());
System.out.println(db.readByte());
System.out.println(db.readUTF());
I do not know why? please tell me what 's wrong? or what should I do about the text-value reading? I meanning how to read it. is I read such value with a wrong mothod?
[ May 05, 2004: Message edited by: Yanxin Zhao ]
 
Zafer Abu saeed
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Lastly I can say for somebody:
Welcome to Ranch !
For the non-numeric values, you may read it as a byte array, because I believe they were written as a byte array (not as a String), that is;
the non-numeric values in the database file were written using write(byte[]) instead of writeUTF(), and so, when you want to read it you have to use read(byte[]) not readUTF (readUTF() will reach the end of file).
The byte array should have a size similar to the field's size (you can discover the field size from metadata).
I hope this will help.
 
Hanna Habashy
Ranch Hand
Posts: 532
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi Yanxin:
I had the same problem yesterday. I played with it until I figured it out. Here is what I found: First I use RandomAccessFile. All the numeric vlaue that consistes of four bytes, I read it as an integer. All numeric values consists of 2 byts, I read it as short. The schema of a field, I read it as UTF. The data section, I read it as bytes, then convert it to String
I hope that help
[ May 05, 2004: Message edited by: Hanna Habashy ]
 
Yanxin Zhao
Ranch Hand
Posts: 65
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Hanna:
I think my thought is the same to you about the data file oprating, but when I try to read non-numberic value, the VM gives me EOFException.
maybe my data file is wrote with write() method (but I think it is not that), but I do not know how to convert into String. Could you tell me how to do it? thanx. : )
 
Hanna Habashy
Ranch Hand
Posts: 532
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi Yanxin:
When I read non-numeric value, i.e a field in a record, I read the bytes into a byte array, then I pass the array as an argument to String constructor.
Reading the schema is different. I read the schema as UTF. The schema is encoded using a modified UTF-8 format, however the data records itself is written as bytes.
 
Yanxin Zhao
Ranch Hand
Posts: 65
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Hanna:
I understand what you said, but I do not know why I can not read the non-numberic value of head, does your code likes these:
System.out.printlb(db.readInt()); // read in magic cookie.
System.out.println(db.readShort()); // read in number of fields.
System.out.println(db.readByte()); // read in length of field name.
System.out.println(db.readUTF()); // read in the first field name.
I really do not know what should I do. : )
 
Hanna Habashy
Ranch Hand
Posts: 532
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yanxin:

System.out.println(db.readByte()); // read in length of field name.
System.out.println(db.readUTF()); // read in the first field name.

my instruction says that the length of a field follow the field name.
so, I do
db.readUTF();
db.readSourt();
First I read the field name, which is encoded in UTF format, then I read the 2 bytes numeric value as short.
Check out your instuction carefully
[Andrew fixed typo in the tag for [quote]]
[ May 06, 2004: Message edited by: Andrew Monkhouse ]
 
Yanxin Zhao
Ranch Hand
Posts: 65
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi:
I think I know why I can not read my non-numberic value from data file now, it is right what Zafer said(thanx so much), the non-numberic value is not write with writeUTF() method. I successfully read in such value by read byte in a an array, and then convert to String, then okay : ) how stupid I am, it spend me a day 's time.
thanx so much, Zafer and Hanna.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic