While Loop + Datafile

[Ian Hamilton]

Hi all, how can I test if I have reached the end of the datafile using RAF?

At the moment I stop when bytes read = -1, however this means I read the record after reading the deleted flag which offsets each row by one.

while((bytesRead=rAF.read(record)) !=-1){
int deleted = rAF.read();
rows.add(new String(record));
}

This works fine but feels like overkill.

while(bytesRead !=-1){

int deleted = rAF.read();
bytesRead=rAF.read(record);
if(bytesRead != -1){
rows.add(new String(record));
}
}

[Ian Hamilton]

Correction: The first example means I read the record before the deleted flag - which is incorrect.

[Ian Hamilton]

If I do this it loops one too many times and duplicates the last row

while(bytesRead !=-1){

int deleted = rAF.read();
bytesRead=rAF.read(record);
rows.add(new String(record));
}

[Jeroen T Wenting]

                // if it is, check of end of record is also inside file
                desiredposition += RECORD_LENGTH + DELETE_FLAG_SIZE;
                result = desiredposition <= filesize;
                if (!result) {
                    throw new IOException(
                        "Database corrupt, filesize expected at least " +
                        desiredposition + " but found to be " + filesize);
                }

Checks whether the indicated record can be read from the file.
If the file isn't long enough to contain the record, throws an exception.
desiredposition is before this step set to the beginning position of the desired record (and checked to see if that falls inside the file).

filesize is simply calculated calling datafile.length() which returns the size of the file in bytes.

[Andrew Monkhouse]

Personally I like Jeroen's solution. Along similar lines, you can determine the number of records that should exist in the file before you start reading any of them (after you have read the meta-data and schema of course). So you could throw an exception if the file is the wrong size. And you could do a simple for loop over the expected number of records.

Regards, Andrew

[Jeroen T Wenting]

I chose not to do that as it wouldn't prevent the need to check on reading individual records anyway.
If you just check the filesize when opening the file, then a new record fails to get written properly at the end of the file leading to the last record being truncated, you'd get an IOException reading that new record.

So all I check for on opening is whether the metadata is complete and the magic number is correct.

[Ian Hamilton]

I understand checking the Magic Cookie value, but why check the metadata?

If you hardcode the metadata into your code why check the metadata at all because you already have the values?

[Jeroen T Wenting]

Who says I hardcode the metadata (OK, I do, but I do check it anyway)?
It's there, so it'd better not be corrupt.
And you need to read it anyway to find out where the actual records start.

[Ian Hamilton]

Thanks for your help guys. I opted for a combination of both your ideas

long fileSize = rAF.length();
if((fileSize-BYTES_BEFORE_DATA_SECTION)%(DELETE_FLAG_SIZE+recordLen) != ZERO){
throw new IOException("DATABASE CORRUPT!: Incorrect data format");
}

followed by

int rowCount = (int) (fileSize-BYTES_BEFORE_DATA)/(DELETE_FLAG_SIZE+recordLen);

while(rowCount > 0){

int deleted = rAF.read();
rAF.read(record);
rows.add(new String(record));
rowCount--;
}

I'm still discarding the deleted flag though. Something I need to think about

[Ian Hamilton]

* BYTES_BEFORE_DATA should actually be BYTES_BEFORE_DATA_SECTION.

[Ian Hamilton]

If you're checking the metadata you must have defined constants to valdiate it against - which means it must be hardcoded somewhere

[Jeroen T Wenting]

Not really. You could just check whether it has the size and format specified by the assignment.
My assignment lists explicitly the file has a 4 byte magic value, 4 byte offset of the first record, 2 byte number of fields per record, and then for each field 2 bytes indicator of length of field name, n bytes field name, 2 bytes field length.

Without knowing how many fields there are or what they're named I can thus check whether the metadata is correct.

I would have liked to read the actual metadata itself and use that but can't because I have a numeric field and the metadata doesn't indicate the datatype (only number of bytes).