One of my favorite uses of these forums is to discuss design decisions. First of all explaining what I'm trying to do is a a useful exercise plus there's susally a good discussion of the trade offss.
I couldn't decide which froum this went in, JDBC, Servlets or here. So Since this forum has the words General in the name here goes.
I've been working for about 18 months on an apache/tomcat/servlet based enterprise app that is a data visualization tool into a a large database (5PB) of scientific data distributed over multiple sites around the world. I've got most of the basic tools in place that the user's need and have shifted the development work to hardening and addressing administrator's needs.
Right now I have too many tables, options and explanatory text strings embedded in the code. It was a conscious decision to delay generalizing that part until the user functionality matures. Whether that was wise or not is not the debate I'm looking for.
So the project underway is to move appropriate arrays and tables into the program's mysql database (as oppesed to the exteral scientific data storage) and create pages to create, delete, and edit these things.
The question is do we create a lot of specialized tables. I envision 10, 20 maybe 50. They will be small but they will be formatted close to the Java fields and objects they populate.
Or should we limit the number of tables and make an effort to serialize these data structures into a general form in the table.
Formatted text will have a separate table for help balloons and error messages. The total rows in these tables may hit a thousand over the life of the project.
I just can't see a good reason to go one way or the other. I can't see a big difference in performance or development time.
I'm looking for an opinion or a a discussion on how people have implemented this sort of thing.
Thanks in advance,
It's not what your program can do, it's what your users do with the program.
Joe Areeda wrote:I just can't see a good reason to go one way or the other. I can't see a big difference in performance or development time.
Well, another option might be to look at config or properties files. For things like text prompts and messages, Java already has a reasonably well-defined system for storing those externally, which also includes multi-language support (if you ever decide that you need it).
It's an awful lot easier to send a flat file (or set of them) to a translator, than to expect them to update a database.
However, this is only really an option for relatively static and monolithic data; for stuff that is likely to need a lot of updating, or that has a complex structure, a database may well be a better choice.
Another alternative: Rather than putting all this "program-control" stuff into a centralized database, what about a separate embedded one using something like JavaDB? Then you could simply ship a copy of the embedded db with your jar. Obviously, this might entail a separate update process for that db for your users (or just send them a message to download the latest version of the jar), but it might be more flexible - and faster - than making it a part of your big centralized database.
Or possibly some combination of the two?
Be interested to see what you come up with.
Isn't it funny how there's always time and money enough to do it WRONG?
Articles by Winston can be found here
When I started this project I asked some of the senior scientists for whom English was not their first language if they felt internationalization was desirable. The response varied from only if it's really easy to not worth spending any time on. Evidently English is the language of science and everyone can work with it, even if they can barely carry on a conversation.
I would say the data under consideration is almost static. We currently have 2 servers running this project and are about to deploy two more. One of the new ones is in Germany and will be looking at a different set of scientific data. That's the motivation to do it now.
I expect this data will be set up from scratch (almost) on installation and modified rarely, so all of your suggestions seem doable. I already have a simple text configuration file that contains information needed to access the database so a separate JavaDB or Sqlite db for parameters wouldn't be difficult.
I haven't manipulated property files, only read them. I'll have to study up.
Thank you for the discussion. You've given me some good ideas to ponder.
Well, this has finally bubbled up to the top of the priority list.
I've decided (at least for this refactoring cycle) that there are reasons for using multiple techniques. Since I often get better feedback from the Ranch than my coworkers, allow me to continue the discussion here.
I think this can be reduced to where to find the data stores. I am using the constant "/usr/local/<my project name>"
Text configuration file
This still contains the information we expect the admin who installs it to edit directly. It is of the form <key>=<value> (with optional comments). The long term plan is to have a Java/Swing application to walk you through the installation. The simple format is mainly due to my extreme dislike of using vi to maintain xml files.
Right now this contains the information needed to access the database, and how to authenticate users.
I expect these to be much faster than storing data in the data base. Options that change rarely and are read often will go here. A big advantage of property files seem to be the ease of storing complex objects (yet to be confirmed). Also the property file needs to stay small and fast, both are relative terms.
In this discussion the big things that get saved here are per user preferences. The decision will be based on a) how often do they change and b) how big we expect them to grow.
As usual I get a significant benefit just from putting this kind of decision making into words. The comments and suggestions are invaluable since they usually include things I don't know and haven't thought of.