• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

File Channels (a couple of points)

 
Tony Collins
Ranch Hand
Posts: 435
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have a couple of questions about file channels?
My spec says 'All numeric values in the header use formats of DataInputStream and DataOutputStream'. Does this constrain me to using DataInputStream to read in the header ? I don't see why it should, as Vlad implied in a post earlier.
Also are filechannel writes are atomic if you say move to the start of a record, then write the whole record ?
Max seems to sugest caching all records and then sycronizing on the cache while updating the underlying file. So it wouldn't really matter if you moved the position of the file channel around and wrote un-atomically to specific fields. Am I right here ? So I assume all reads/writes go through the cache ? Does't this lock up the whole filesystem when only a record needs locking ? Or is the whole file being re-written ?

Also has anyone got any good links to filechannel/nio tutorials/documentation.

Finally I think this project is quite challenging and intentionallt designed to incorparate the new features of SE1.4, correct ? My project is Bodgitt and Scarper 2.2.2., which is what I feel I'm currently doing with the IO.
Cheers
Great Forum much appreciate all the help and disscussions.
Tony
 
Bob Reeves
Ranch Hand
Posts: 64
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Tony:
I'll give my opinions on the questions your raised:
Does this constrain me to using DataInputStream to read in the header ?

I use RandomAccessFile because of its getByte, etc. methods. Work fine for me, or at least I think that it does!
Also are filechannel writes are atomic if you say move to the start of a record, then write the whole record ?

If you change the FileChannel's position, the write must wait until the disk drive performs a head seek. You can get answers to this question in the community, but I couldn't find any official answer from SUN.

caching all records and then sycronizing on the cache while updating the underlying file

I rejected caching because of the instruction: "A clear design ... will be prefferred to a complex one...". A cache isn't a requirement, which isn't the same thing as a LockManager.

Also has anyone got any good links to filechannel/nio tutorials/documentation.

From java.sun.com:
Eric S: Are there any good online tutorials or other references for learning about NIO?
Michael McCloskey: There is the O'Reilly book on NIO by Ron Hitchens (mentioned earlier), and I have seen others, such as the early adopter book for J2SE 1.4.


Of interest, they also said:
charltony: I've noticed problems with the old I/O, but haven't experimented with NIO yet. I don't understand the design ideas behind NIO; they seem much more complex.
Mark Reinhold: Yes, New I/O is more complex than old I/O. If you only need to solve simple I/O problems and performance is not a super-critical issue for you, continuing to use java.io may well be preferable.

Well, that's my $.02.
Tx
 
Vlad Rabkin
Ranch Hand
Posts: 555
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Tony,
Cashing the database is not a requirement.
Moreover cashing cas serious drawback: it doesn't allow to maintain a very large data file.
It is just my individual solution.

I beleive FileChannels is more powerfull solution.
I am just not sure that in my design it is very usefull. U can take also a look at the thread:
NX:Client crashed cause deadlock in LockManager
Cheers,
Vlad
 
Mike Southgate
Ranch Hand
Posts: 183
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
DataInputStream and DataOutputStream are implementations of the DataInput and DataOutput interfaces. RandomAccessFile is another implementation of these xfcs so you should be fine using raf. I have my assignment working (just cleaning up now, and making sure the threading and locking is up to snuff) and raf works fine for the Bodgitt and Scarper assignment.
ms
 
Tony Collins
Ranch Hand
Posts: 435
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm using a file channel to read record fields into a ByteBuffer array. This seems to work quite well, not sure if the API for RandomAccessFile's supports this.
For updating a record I wish to write a byte buffer array( representing a record) to the file, I (think I)need to encode the string to US 8-Bit ASCII before adding it to a byte buffer and then writing the byte buffer. I believe I need an encoder but how do I get one of these ?
Any ideas how to convert a string to 8-bit US ASCII ?
Tony
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
[Vlad]: Cashing the database is not a requirement.
Moreover cashing cas serious drawback: it doesn't allow to maintain a very large data file.

This is a potential limitation, but I don't see it as a big problem. I can handle 100000 records without even changing the -mx for my JVM. I think if a customer wants more than that, they should really think about getting a real database, not a text file.
[Vlad]: I beleive FileChannels is more powerfull solution.
I think that the choice of FileChannel vs RAF is mostly separate from the choice to cache or not to cache. Personally I use a FileChannel and I cache all records in memory, and I'm quote happy with the results.
 
Max Habibi
town drunk
( and author)
Sheriff
Posts: 4118
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Tony Collins:
I'm using a file channel to read record fields into a ByteBuffer array. This seems to work quite well, not sure if the API for RandomAccessFile's supports this.
For updating a record I wish to write a byte buffer array( representing a record) to the file, I (think I)need to encode the string to US 8-Bit ASCII before adding it to a byte buffer and then writing the byte buffer. I believe I need an encoder but how do I get one of these ?
Any ideas how to convert a string to 8-bit US ASCII ?
Tony

Try , int)]here
M
[ July 19, 2003: Message edited by: Max Habibi ]
 
Tobias Nettels
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all,
I use file channel in combination with a cache too. One given method of the data class is described as follows:
// Reads a record from the file. Returns an array where each element is a record value.
public String[] readRecord(long recNo) throws RecordNotFoundException {}
My questions to that:
1) If I use a cache I do not read a record from the file (as described) but from the cache. So does this match the requirements?
2) It is described that each element of the returned String[] is a record value. So am I not allowed to put the recNo to the returned [] cause its no record value?
Both questions probably look a little bit like I'm doing things too much by the book, but I'm german, my english is rather bad and therefore I'm sometimes not sure how strict things are meant.
A last question:
Is it right that the method deleteRecord have to be implemented but aren't used in the application?
Thanks
Tobias
 
Philippe Maquet
Bartender
Posts: 1872
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Tobias,

1) If I use a cache I do not read a record from the file (as described) but from the cache. So does this match the requirements?

Yes. By reading a record from the cache, you indirectly read it from the file.

2) It is described that each element of the returned String[] is a record value. So am I not allowed to put the recNo to the returned [] cause its no record value?

  • I think their doc is badly written : we must understand "each element of the returned String[] is a field (or column) value"
  • No, you are not allowed to put the recNo in the array. But BTW, the caller knows it already, as recNo is the method parameter, right ?



  • but I'm german, my english is rather bad and therefore I'm sometimes not sure how strict things are meant.

    You english is very understandable (at least by a french-speaker like me ).

    A last question:
    Is it right that the method deleteRecord have to be implemented but aren't used in the application?

    Yes. You may see the Data development as a separate assignment, offering general-purpose database access. You won't use it in this application, but you must implement it (as createRecord) and make sure that it works well. Other people (or even you) may need it in other future developments.
    Cheers,
    Phil.
     
    Tobias Nettels
    Greenhorn
    Posts: 8
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    Thanks Philippe
     
    • Post Reply
    • Bookmark Topic Watch Topic
    • New Topic