• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Managing File resources

 
Stephen Anson
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,
I have a design that doesn't use record caching.
From the required Data class I delegate file access to a RecordIO class.
RecordIO contains two File objects one for reading and another for writing.
Previously I tried creating new File objects for every read / write of a Record, but all the object creation seemed to slow things down, and there was the potential to run out of resources.
I am considering rewriting this so I have a ThreadPool of worker threads that have some number of File objects available to them.
Am I making this too complicated.
How are others managing their File access objects.
cheers Steve Anson
 
Andrew Monkhouse
author and jackaroo
Marshal Commander
Pie
Posts: 12014
220
C++ Firefox Browser IntelliJ IDE Java Mac Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Steve,

Why do you need all those instances of the file access class? What difference do you expect it to make if you have only a single instance of the file access class?

I presume you are going to imply that you are going to get better performance by having all these instances of the file access class, but in reality you are still dealing with only one physical file. So it is unlikely to make that much of a difference.

Caching all the records in memory for read access is more likely to make a difference.

All of which becomes redundant when you realise that you are not going to get any extra marks for making a more performant solution .

Regards, Andrew
 
Stephen Anson
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Andrew this is new territory for me; my thoughts went something like this:
1. I didn't want to cache all the records as I didn't feel this would scale too well with large numbers of records
2. Cached records would have some persistence mechanism anyway.
3. A reader and a writer would allow concurrent reading and writing of the file provided records were locked appropriately.
4. Cached records might work at close to light speed but File IO would be a bottle neck anyway??

Am I barking up the wrong tree here?
Any help much appreciated
thanks Steve
 
Inuka Vincit
Ranch Hand
Posts: 175
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
1. Doesn�t the approach your taking take up more memory than using a cache though. Maybe I misunderstood but you create two file streams for each record correct? In my short experience of Java I have only experienced one instance where the JVM ran out of memory and that was running a java network simulator for my thesis. Even that I was able to correct by using the -Xmx -Xms flags.
2. Yes
3. Yes your approach should give a significant performance advantage(theoretically) .
4. Yes definitely. I am caching records and the file IO is my big bottle neck there can be only one file operation for a given time.

Although your ideas is very good, your over complicating things. The assignment calls for a pretty simple program without any performance requirements. Furthermore it specifies that the program should be architected so that a Junior programmer should be easily able to read the code, which basically calls for clean code and simple functionality. Besides that you wont get any more points for all the effort you put into making your program perform well(like Andrew said).

I chose to use caching, and the only reason I chose to go this route is because the code because very much cleaner(easier to code in my part) and easier to read(IMHO). As for performance I have only one file handle, which causes a bottle neck only for the write condition, but that is not a requirement.

Code the requirements, define classes/interfaces so that future improvements can be easily added just like you would at work(because just like taxes and death you know requirements are going to change ) and please stop torturing your self like this.
 
Stephen Anson
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sorry I explained that poorly.

>I basically have a singleton FileIO class that has one reader and one writer.

>My thoughts were that a caching solution would need File I/O anyway- so mine omits the cache hoping to make it simpler, it appears that I must be wrong here but I can't see why!

any help appreciated
Steve
 
Andrew Monkhouse
author and jackaroo
Marshal Commander
Pie
Posts: 12014
220
C++ Firefox Browser IntelliJ IDE Java Mac Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Steve,

>My thoughts were that a caching solution would need File I/O anyway- so mine omits the cache hoping to make it simpler, it appears that I must be wrong here but I can't see why!


Are you asking about how caching avoids the File I/O bottleneck? If so ...

With a cache all the records would normally be read only once. The majority of operations are going to be "reads" and "finds", which means that they will be working on the in-memory copy of the data - no file access at all.

You will still need some way of updating the cache and persisting the data to file whenever a user updates a record, but that is a far less common operation, so it should not be a significant bottleneck.

Regards, Andrew
 
Stephen Anson
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thanks,
i'm convinced now!
Steve
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic