Win a copy of Programmer's Guide to Java SE 8 Oracle Certified Associate (OCA) this week in the OCAJP forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

File instead of Database table

 
Raghav Mathur
Ranch Hand
Posts: 641
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Just wondering whether it is possible to load the table(from the DB) contents into a file once and use the data for later use . That is extracting the data from the file instead of Extracting it from the database everytime.
 
jason adam
Chicken Farmer ()
Ranch Hand
Posts: 1932
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sure it's possible, you just have to write the file out in a way that the application knows how to pull from. Not sure how efficient that is, but it's definitely doable.
Maybe instead you could create a holder object(s) that keeps the data from the database and use that for access and manipulation, and when you're done write it back to the database. It's a lot quicker to access things in memory than in the file system.
 
Raghav Mathur
Ranch Hand
Posts: 641
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I,am using object arrays to store the items fethched from the database.I am working on MVC architechture. I don't what will be the implication when the amount of data increases . Will my Array objects be able to hold such and amount of data( say 10000 rows) ??
That is th reason i was thinking of an alternative solution of storing data in a file and access it .
What do you say ?
[ March 23, 2004: Message edited by: raghav mathur ]
 
jason adam
Chicken Farmer ()
Ranch Hand
Posts: 1932
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I wouldn't use an array, I would use some sort of Collection like an ArrayList instead. Personal preference, and keeps you from being confined to a set size.
Avoid writing to the file, especially with that amount of data. You're I/O would start to get very costly.
Depending on your database implementation, it may already be keeping things in memory so pulling from the database on a semi-regular basis might be the best solution. Holding that many objects in memory with a collection might be duplicating the effort.
Perhaps giving a flow of the application would be helpful, as well as how this application is being implemented (what type of application server, etc.) Might look into EJB's if it is applicable.
But bottom line, I wouldn't write that data to a file, the I/O stuff can be the least resource friendly way to do things.
 
Raghav Mathur
Ranch Hand
Posts: 641
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The Application server is Jboss Tomact . The concept is an EJB concept (Object Arrays).The Application server is Jboss Tomact . The concept is an EJB concept (Object Arrays). Since it is a web application , Connecting it to the database often reduces its speed , especially when it comes to MIS reports.
that was the reason i thought of object arrays . Does it have a limitation?
 
Stefan Wagner
Ranch Hand
Posts: 1923
Linux Postgres Database Scala
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The array-index is limited to MAX_INTEGER.
And the physical memory is of course a limitation.
I wouldn't use a file, if the data changes frequently.
If you have 10,000 rows, and delete the first, you have to write 9,999 files.
Is your application the only source of changes?
Else: How do you handle changes from another source?
Why is the database-connection slow?
Do you always create new connections/ statements?
This is often the cause of slow db-performance. (Perhaps better forums: jdbc, performance).
If your handling of the jdbc-connection is 'best practice', perhaps the database-settup is semi-optimal? Are there indexes on the critical columns?
Perhaps you can talk to the database-administrator.
There are a lot of problems, database are invented for, to solve them.
Handling datapersistence, security, access, concurrency, searches, inserts, deletes, ... on your own might lead to reinvent databases.
 
Raghav Mathur
Ranch Hand
Posts: 641
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

I wouldn't use a file, if the data changes frequently.
If you have 10,000 rows, and delete the first, you have to write 9,999 files.

If storing each row in a file is the only solution then it is definately not a good idea .. i would agree on that.


Why is the database-connection slow?
Do you always create new connections/ statements?

The database connection is not slow i would say but if i encounter a situation where i need to make chlid-connections while looping through the parent connections then it definately reduces the load time of my JSp page.The net effect is that i am making a coneection for each row as i print a certain value.Yes i can use joins but to a certain limit.
 
jason adam
Chicken Farmer ()
Ranch Hand
Posts: 1932
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
And since we've moved into efficient use of JDBC connections, I think I will send this to that forum. I think we all pretty much agree writing to a file is a bad thing to do
 
Stefan Wagner
Ranch Hand
Posts: 1923
Linux Postgres Database Scala
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
And please excuse my attention-error; of course you don't have to write 9.999 files but rows.
How many connections to you have at once?
I've heard of people making one connection per statement, but one connection per row?
 
Raghav Mathur
Ranch Hand
Posts: 641
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Stefan Wagner:

How many connections to you have at once?
I've heard of people making one connection per statement, but one connection per row?

Yes. There comes a situation when i am left with no choice but to use another connection to retrieve a piece of data . That means when i am looping through the recorset and retireving each row , i am also making a connection to retrieve some information related to the information i am retrieving through the recordset.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic