• Post Reply Bookmark Topic Watch Topic
  • New Topic

how to solve the data problem when we don't have backend database support?  RSS feed

 
Edward Chen
Ranch Hand
Posts: 798
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
if we do a stand-alone application without backend database support, we have to read from /write into a file.We have other options ?

if we use a external file, the problem is speed. How we get the better speeed ? one way is to use multi-thread, another way is to load into all data into a build-in collection, such as linked-list, hashtable. this solution problem is , it will take time to load / save data into file.

This is what I am thinking. I want to know, in the real world programming, how to solve the data problem when we don't have backend database support?

Thanks
 
Joe Ess
Bartender
Posts: 9406
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
By "back-end database" I suppose you mean an enterprise database system like Oracle or MSSql. There's a ton of options. You need to choose the one that fits your particular requirements. I've worked on applications that required full SQL compliance because we were working with large data sets and complex queries. In this case I used MySql in a client-server fashion on the same computer.
In another application I needed complex query capability, but the data set was small enough that I felt I could embed the database in the app itself. Cloudscape fit the bill back then, and now it's been contributed to the open-source community by IBM. I've also had my eye on Berkeley DB for use as an embedded DB. It's very popular among GNU utilities.
RandomAccessFile can be used to construct an extremely fast embedded database if you don't have a lot of relational data or complex queries.
Those options probably just scratch the surface. . .
 
Adrian Yan
Ranch Hand
Posts: 688
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
How much data do you have? I wrote an application without any "database" support, simply using XML. My data set was around 50MB, it wasn't notciably slow. Also, you might be able to add indexing to the XML files to help search and load.
 
Edward Chen
Ranch Hand
Posts: 798
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, Joe, Thanks for your great reply.

"the data set was small enough I could embed the database in the app itself. Cloudscape fit the bill back then"

---- question: when you use InstallAnyWhere or the other java installer to package the application into exe file, the Cloudscape will be packaged inside ?

----- question: how small the data set is fit for cloudscape? 50 mb ? 500 mb?

"I've also had my eye on Berkeley DB for use as an embedded DB. "

----- question: could you give me more links about "Berkeley DB"?

Thanks. Your response really light me up.

Edward
 
Edward Chen
Ranch Hand
Posts: 798
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Adrian Yan:
simply using XML. My data set was around 50MB, it wasn't notciably slow. Also, you might be able to add indexing to the XML files to help search and load.


Hi, Adrian, thanks.

How do you do XML file and indexing as a data set? could you give me more web links or book-ISBN ? I really want to know it.

I want to ask, how long it will take to load 50MB XML file into a Vector object? How long it will take to save back 50MB file when application exits ?

Thanks
 
Joe Ess
Bartender
Posts: 9406
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Edward Chen:

---- question: when you use InstallAnyWhere or the other java installer to package the application into exe file, the Cloudscape will be packaged inside ?

You can distribute Cloudscape as a JAR, so you would include it with your installer like any other file. Just make sure it gets into your app's classpath.


----- question: how small the data set is fit for cloudscape? 50 mb ? 500 mb?

The data set I was using was pretty small, a couple of megs at most. The Cloudscape developers said it was scalable to gigs/millions of records.


----- question: could you give me more links about "Berkeley DB"?


Google for it and you'll get over half a million hits. It's supposed to be the most widely deployed embedded database in the world.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!