• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • paul wheaton
  • Jeanne Boyarsky
  • Ron McLeod
Sheriffs:
  • Paul Clapham
  • Liutauras Vilda
  • Devaka Cooray
Saloon Keepers:
  • Tim Holloway
  • Roland Mueller
Bartenders:

Deciphering the instructions

 
Ranch Hand
Posts: 98
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Okay, I have been over and over this, and it is bending my noodle. I understand the concept behind using a RandomAccessFile class to modify the database. However, I am not sure what kind of data I am dealing with in the database. The instructions say:

The format of data in the database file is as follows:
Start of file 4 byte numeric, magic cookie value. Identifies this as a data file 4 byte numeric, total overall length in bytes of each record 2 byte numeric, number of fields in each record
Schema description section. Repeated for each field in a record: 2 byte numeric, length in bytes of field name n bytes (defined by previous entry), field name 2 byte numeric, field length in bytes end of repeating block
Data section. Repeat to end of file: 1 byte "deleted" flag. 0 implies valid record, 1 implies deleted record Record containing fields in order specified in schema section, no separators between fields, each field fixed length at maximum specified in schema information
End of file
All numeric values are stored in the header information use the formats of the DataInputStream and DataOutputStream classes. All text values, and all fields (which are text only), contain only 8 bit characters, null terminated if less than the maximum length for the field. The character encoding is 8 bit US ASCII.


So my question is, how do I know which RandomAccessFile method to use? Do I use readByte since they are stored in bytes? Because it also says "numeric" which is throwing me for a loop. I can use readLine() which reads everything, but I only need to read a complete record at a a time. So for that are we talking about stores bytes, strings, chars, in an array?? I am truly confused. What does total overall length in bytes of each record 2 byte numeric mean? I thought each record has a length of 159 (for me anyway).

This part of the directions is really the bottleneck that is keeping me from moving on. I can't implement the methods until I can navigate the database, and I can't really do that until I can decipher these instructions. I'm hoping you guys can help me.

Thanks to you all.
Matt
 
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Matt,

Just try it one value at a time. And when you are finished, you'll see that it truely was not that hard (like it seems now ) I will put you in the right direction but of course I won't give the complete solution.

Start of file 4 byte numeric, magic cookie value.


So the 1st thing you have to read is a 4 byte numeric, which represents the magic cookie value. According to the javadoc of RandomAccessFile I'll use the readInt-method to do so.

4 byte numeric, total overall length in bytes of each record


Again 4 byte numeric, so again the readInt-method. This value represents the total overall length in bytes of each record.

2 byte numeric, number of fields in each record


Just 2 bytes now, having a look at the javadoc once again, readShort-method is the one you need. The value you have read identifies the number of fields in each record. Which is important for the next part, stay tuned!

Schema description section. Repeated for each field in a record: 2 byte numeric, length in bytes of field name n bytes (defined by previous entry), field name 2 byte numeric, field length in bytes end of repeating block


So this part is the schema description. Like the instructions say, repeated for each field, so you'll have to use a loop (you just have read the number of fields in a record). For each field in this schema description you'll have to read 3 things:
a) 2 bytes = number of bytes of the field name (readShort-method again)
b) n bytes = name of the field (create a byte[] with length equals to the value from a) and use the read-method that takes a byte[] as parameter
c) 2 bytes = length of the field (readShort once again)

Now it's up to you!
Kind regards,
Roel
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
By the way: seems that something went wrong with the indentation when you saved your instructions and that seems to throw you a bit off.

A bit better formatted you get this:

The format of data in the database file is as follows:

Start of file
4 byte numeric, magic cookie value. Identifies this as a data file
4 byte numeric, total overall length in bytes of each record
2 byte numeric, number of fields in each record

Schema description section.
Repeated for each field in a record:
2 byte numeric, length in bytes of field name
n bytes (defined by previous entry), field name
2 byte numeric, field length in bytes
end of repeating block

Data section.
Repeat to end of file:
1 byte "deleted" flag. 0 implies valid record, 1 implies deleted record
Record containing fields in order specified in schema section, no separators between fields, each field fixed length at maximum specified in schema information

End of file


Hope it helps!
Kind regards,
Roel
 
Matt Pavlovich
Ranch Hand
Posts: 98
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Roel,

Yes, you read my mind. I woke up this morning and have been going through the instructions line by line, and matching them up with the TestReadFile you posted. It is starting to make a lot more sense, and now with your explanation I feel I am now pointed in the right direction. Thank you!

Also, you are right about the formatting. Perhaps because I used Opera to view the file it messed it all up. The lack of proper formatting was really making it confusing. However, the properly formatted instructions you posted now make a whole lot more sense.

Thank you as always.
Matt
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Glad I could help!
 
Matt Pavlovich
Ranch Hand
Posts: 98
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Your help is invaluable. My blood pressure thanks you. Quick question, though. I am going through the TestReadFile and grasping it line by line and I just want to check one small bit:


The in.readByte() is always returning a value of "0". From the instructions I take this to mean it is a valid record, and that if it was not valid (or had been deleted) the value would be "1". Is that correct? And if so, then I guess it becomes my job to change that value to "1" if the record has been deleted but through my code logic I have decided I am not getting rid of deleted records. However, it seems it might be easier to just completely delete records, does it not?

Thanks tons,
Matt
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That is correct!

Actually deleting records will not be easier than just marking it as deleted (setting the flag to 1). Think for example about having to renumber the records each time a record is deleted, because otherwise the record number (which you use to determine the offset) will be incorrect and is "pointing" to another record.
 
Matt Pavlovich
Ranch Hand
Posts: 98
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Roel,

That is an excellent point! Thank you.

Matt
 
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am also confused with the instructions. Is the database schema saving numbers (such as size and ids) as numeric values or strings? For example it is saying the field length for "size" is 4, but if I save it as 4 bytes as a string, the max value can only be 9999. If I save it as an Integer, the max value can be Integer.MAX_VALUE. I've been treating these values as "strings" all along and now I am second guessing myself in testing my application -- thus my confusion

Thanks,
Kenneth
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If the instructions say something like "4 byte numeric" you are supposed to handle this value as a numeric, not as a string
 
Kenneth Logan
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So it does not say anything about numeric values in the database schema portion of my instructions and after testing trying to read a "size" in as a numeric value I got a very weird value. So I guess I was right -- it is treated as a string.

Thanks,
Kenneth
 
Matt Pavlovich
Ranch Hand
Posts: 98
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Further on that topic, here is a question: The instructions state that the character encoding is 8 bit US ASCII. So what method did they use to write this file? writeUTF() or writeBytes()? (Or something else?)Does anyone know? I think this would help me understand how I am supposed to create records. Thanks.
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Kenneth Logan wrote:So it does not say anything about numeric values in the database schema portion of my instructions


I guess you mean the "size" field, which is part of each record. That's indeed just a string like stated in the instructions: "Record containing fields in order specified in schema section, no separators between fields, each field fixed length at maximum specified in schema information". I also treated the field values of a record as Strings.
 
Kenneth Logan
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Makes sense. Thanks Roel!
 
Matt Pavlovich
Ranch Hand
Posts: 98
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Mmm...I think I now see the stupidity of my question. Using the DataInputStream makes it 8 bit US ASCII because we are writing bytes when we are using streams. It has nothing to do with writeUTF or writeByte. I think that is clear to me now but please anyone let me know if I am wrong.
 
Kenneth Logan
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Matt,

I used a StringBuilder to build my entire record and then got the entire string as a string of bytes using String's getBytes(String charsetName) method. The charsetName was the DB encoding (US ASCII). I then used RandomAccessFile.writeBytes(byte[] b) method to write the record to the database. I believe some other posts mention that RandomAccessFile is okay to use, and so I decided to use it. Now I should probably go back and see if that is the best tool to use (instead of just using DataOutputStream).

I Hope this helps,
Kenneth
 
Matt Pavlovich
Ranch Hand
Posts: 98
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Kenneth,

Thanks very much. That does help to make things clearer. Best of luck to you on your submission.

Matt
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic