Win a copy of Securing DevOps this week in the Security forum!

Tim Holloway

+ Follow
since Jun 25, 2001
Tim likes ...
Android Eclipse IDE Linux
Forum Moderator
Tim Holloway currently moderates these forums:
Long-time moderator for the Tomcat and JavaServer Faces forums. Designer and manager for the enterprise server farm, which runs VMs, a private cloud and a whole raft of Docker containers.

These days, doing a lot of IoT stuff with Arduinos and Raspberry Pi's.
Jacksonville, Florida USA
Cows and Likes
Total received
In last 30 days
Total given
Total received
Received in last 30 days
Total given
Given in last 30 days
Forums and Threads
Scavenger Hunt
expand Rancher Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Tim Holloway

Welcome to the Ranch, Bob!

What you are describing sounds like a problem domain called ETL, which stands for Extract, Transform, and Load. Basically any situation where you take data from one or more sources, do some automated fiddling with it (optional) and output it to one or more destinations.

The good news is that there are already some very powerful tools to do just that.

Although Linux/Unix comes with some good applications itself, sometimes you need something a little more integrated. There are at least 2 Java-based products that can do ETL work: Talend and Hitachi Pentaho DI.

I've worked with (and contributed to) the Pentaho product. It's admittedly a little strange at times, but it's very powerful and it comes with a GUI flow editor that makes it easy to create transformation workflows, including doing things like identifying fields found in database tables or spreadsheets/CSV files, fixing up bad data, excluding duplicate records, splitting and merging fields, value lookups and much more. You can find the community edition on Sourceforge.
1 hour ago

Campbell Ritchie wrote:

Tim Holloway wrote:. . . . Some typewriters even had a ribbon shift key.  . . . .

I used to have one of them. It had three positions, •. The white   was used for cutting stencils, without using the ribbon. I gave up buying black and red ribbons because I never seemed to use the red.

Oh yeah. It wasn't a key, it was a little lever.

I think the primary advantage of the red ribbon was in doing homework where you'd print quotations in red.

Campbell Ritchie wrote:

Tim Holloway wrote:. . . in the real world, banks actually didn't update balances in real time. Instead, they posted transactions that were batched up and applied overnight. . . .

They had a paper bank statement sheet for each customer, which was taken out of the file and put into an adding machine, and all transactions were printed on it by the adding machine; negative figures caused it to use red ink. The sheets were duplicated and once a month one copy was posted to the customer as a statement. That is what my bank statements looked like for the first year I had the account; when I moved to a different branch, they used a basic computerised system.

Young whippersnapper! In my day, they wrote it all down on a sheet of paper and gave it to Scrooge & Marley where Bob Cratchitt would write it all down in the ledger books!

Seriously, you're either out-aging me or the fact that at one time my town was a regional banking and insurance capital meant that for a change we weren't vying for the most technologically backwards spot in the nation.

Red/black ink ribbons are a thing. Some typewriters even had a ribbon shift key. Not only banking machines had dual-colour capability, though, some of IBM's general business tabulating machines did as well.

Noorul Hameed wrote:Physical is not a parallel, or even ATM/Online could be parallel.

I would be very interested to hear that statement translated into English.

If you've got 3 people attempting to alter the same account "in parallel", I'd consider those to be concurrent operations, to use the technical term for it. What you want to avoid is simultaneous operations, where the operations are not merely concurrent (parallel), but acting independently.

And yes, you need some sort of synchronization interlock to maintain data integrity while the concurrent operations are going on.

The question is false. The danger is not that 3 simultaneous requests would overdraw the account, but that the account balance would be corrupted.

Allow me to demonstrate. To keep things clearer, I'll use only 2 participants, rather than 3.

Let's say that we have two gentlemen, Mr. Singh and Mr. Chandraputra who share the same account. Mr. Singh goes to the bank and withdraws 50000 rupees. Mr. Chandraputra concurrently goes to an ATM and withdraws 3000 rupees.

The shared computer system shows a balance of at least Rs 50000, so the teller hands Mr. Singh the cash and debits the acccount online. However, while the account transaction is in process, the ATM transaction kicks in. Both transactions pull the same starting balance, but because they are not merely concurrent, but are simultaneous, the updating will occur without taking the other person's transaction into account. So at the end of both transactions, instead of showing a net debit of Rs 53000, the account could potentially only show the debit of Rs 3000 and the bank has just lost Rs 50000.

Therefore you do need a synchronisation mechanism. What kind of mechanism depends on your IT infrastructure. If there's a master Java application handling all withdrawals, regardless of source, you could synchronise on the javabean representing the account (if there is one), most definitely.

More commonly, though, you'd lock the account data at the database level, do the transaction, then commit the change, Or if the account has overdraft protection and the withdrawal would exceed the balance, reject the change and rollback the transaction.

On a historical note, in the real world, banks actually didn't update balances in real time. Instead, they posted transactions that were batched up and applied overnight. Even ATM transactions weren't posted in real time. These days, probably everyone does actually update in real time, as traditional mainframe-style batch processing is out of fashion.
Tomcat may be able to find a JDK from the Windows registry, but this would be a solution of last resort. Java is designed so that multiple JDK's and JRE's of different (or even the same) version can be installed on a machine and in operation at the same time. So JAVA_HOME is the way to explicitly and unambiguously inform Tomcat which one to use. And, again, it MUST be a JDK. A JRE lacks the extra resources that Tomcat needs.

Beyond that, all I can say is definitely run a vanilla Tomcat from a command prompt and leave the Windows-specific options for Tomcat until you can get the standard version running.

You get a "404" error on the JSPWiki pages because the JSPWiki webapp did not start up correctly so Tomcat shut it off. As I said, I can't tell exactly why, but using a JRE to run Tomcat might explain it.
22 hours ago
I have a bigger doubt. DON'T WRITE YOUR OWN SECURITY. Unless you are a full-time professionally-trained security professional, it's about 95% certain that someone can crack your security system. And, based on what I've seen over the years, about an 85% chance that non-technical people will be able to bypass security in 15 minutes or less. That includes systems designed by the in-house "genius".

J2EE/JEE defines a standard container-based security subsystem. In nearly 20 years, I've never heard of it being breached. It's pre-written, pre-debugged, tested by security pros, well-documented, and best of all, it blocks most security attacks before they can attack the web application itself. Consider using it.

Spring also provides a finer-grained security system itself, which I believe actually can work in concert with container security.

So there's no excuse for writing your own security management in most cases.
1 day ago
Welcome to the Ranch!

I did a quick search and found this:

Note that your SQLite database has to be built with authentication enabled for it to work.
If you're talking about the filename that comes in as part of file upload, you cannot.

This "filename" is sent by the client as part of the uploaded data package. It can actually be whatever the client wants it to be, but historically on Internet Explorer it was the fully-qualified pathname on the client's machine of the file being sent. For most other browsers, it was the relative pathname of the file or even just the simple filename itself without a path.

In any event, we are fortunate that an HTTP server cannot reach back into the client's filesystem, since that would make a whole constellation of security exploits possible. Indeed, I'd argue that IE's inclusion of the entire path is already giving the server too much information about what's stored where on the client's machine as it is. Thus the upload filename is mainly for convenience purposes. It's commonly used to create a matching file somewhere on the server machine (and the path where that file lives is unrelated where the client kept the original copy), and it can be displayed as part of a confirmation message, but it has no set purpose of its own.
1 day ago

Kaine van Gemert wrote:

Again, in JSF, the action method "payExpense()" can invoke whatever business logic it wants, knowing that all of the Model values have been vetted and updated before payExpense() has been called.

Say one of the model values is the persons balance, if you just update this based on the user input, before calling the business logic, it would update the balance to be the balance -  the expense. Isn't it the business logic that determines whether or not the transaction can succeed? It would check the user has sufficient funds, or if it's in another currency for example then maybe some currency exchange might need to happen etc.

That's why it's desirable to keep a separate GUI backing model from the business data model. The GUI model can keep the current control values, but ideally you would not invoke the business logic every time a control value changes.

Consider a dialog box. You have lots of controls (sub-views) within a container view (the dialog box). Almost always there's a set of buttons like "OK", "Cancel", sometimes "Apply" and occasionally "Reset". Changing the individual control values alters the dialog box Model, but that's (hopefully) not the business data model. So the command buttons conventionally work like this:

Apply: Invoke the business logic, but leave the dialog displayed
Reset: Reset all the dialog controls to their original values
Cancel: Discard all changes to the dialog control and dismiss the dialog from the display
OK: Invoke the business logic like Apply does, but also dismiss the dialog from the display

Of course, other options are also possible. For example, a dialog might have a "Save" and a "Save As..." button. But in all cases, the business logic doesn't fire until you explicitly tell it to do so. And hopefully the first thing the "Fire" operation does is validate the data and abort the operation if it's invalid.

JSF takes this one step further, since automated validation is available. The backing bean isn't updated at all unless each and every one of the control values submitted is accepted by its validator(s). Still, even then I recommend keeping the backing bean as a work area rather than applying the data values directly to the business objects. And JSF validators are primarily syntactic, not semantic, so if, for example, a time-of-day field for an appointment maker runs 8-5 when the day field is Monday-Friday but 8-noon on Saturday, that's best handled in the introduction logic of the action method.

And of course, this is also where you'd check to see if an account had a balance great enough to support a transaction.
1 day ago

Kaine van Gemert wrote:You mention you name all properties 'setXxx()', I get what you mean, but how does this apply to lists or maps? I see how that makes sense for most properties but as soon as it is a collection or map of some sort the 'setXxx()' would logically mean set the entire collection/amp, how would you modify this approach to work within collections/maps?

There's actually an Apache project called beanUtils that has been widely used to implement the various features of the JEE Expression Language (EL). It's instructive to read, since you can see not only what EL does, but how it does it.

In the case of a Model Bean containing an ordered collection (List or array) or a map, you're not talking to the collection, so you have a secondary qualifier. In the case of a List/array, it's a subscript. In the case of a map, it's a key. In EL it looks like this:

Note that in EL, the variable part of the expression starts with a '#' or '$'. The '$' dates back to JSP days, and is a read-only reference. Since JSF uses a read/write Model, the '#' marker tells EL that the data flows both ways.

In a non-EL environment such as Swing or AWT, you'd bind the sub-object as part of your setup logic. Whether the control is primitive or complex would determine whether you'd qualify the individual element. For example, a listbox would take an ordered collection. A Combobox would take an ordered collection for the dropdown list, but would also map to a string property for the selected value. Which is the same as you would for EL-based web controls.
1 day ago

Campbell Ritchie wrote:I was thinking to make only the getXXX methods public and other mutating methods, e.g. in this case addMoney(...) or payExpense(...), to be restricted to the package containing the model.
There may be another way to do that with packages exported and not exported, but I have no experience with that new feature.

That would be a rather dirty solution, since you're basically positing a read-only model and the "writing" side of it would be business methods.

I'm going to fall back on JSF again since not only is it something that I am extremely familiar with but also it exemplifies a MVC framework, and in particular, one based on Injection. In JSF, the controllers do all the reading and writing of backing bean properties from the UI side (what the business side does is up to the business side's designer), and since the JSF Backing Beans are, per the standard POJOs, that means that the controllers do not go rummaging around looking for public field access or random methods, but explicitly demand that any mutable property'd mutator method be named "setXxxx()". The actual EL that ties the Model Property to the View doesn't specify a method call, but rather the property's name. The framework can then easily calculate which method to invoke on the Model based on the name and execution context.

One of the things that makes this sort of approach desirable besides simplifying the framework design is that it allows separation of the business trigger from the controllers and their data pushing. Again, in JSF, the action method "payExpense()" can invoke whatever business logic it wants, knowing that all of the Model values have been vetted and updated before payExpense() has been called. It's not tied to one particular Model property. Contrast this with a method that fires the minute that you change the value of a control on the View. In many cases, incomplete data entry - since you have to move from one control to another, updating each in turn unless you're far more adept at keyboard and mouse than I am - means that business logic has no safeguards against operating on an incomplete or invalid data set.

This is amplified even further when you're talking about desktop-style GUIs such as Swing where a "Save Project" trigger can come from a hotkey, menu action, toolbar button, and/or other GUI controls and even automated scripting interfaces. In such cases, all of those inputs would be simplest if they could all fire the same event instead of having custom logic for each type of action control.
1 day ago

Campbell Ritchie wrote:

Tim Holloway wrote:. . . reason not to use package-scope for properties is because it's hard to read and maintain code where someone can come in more or less unexpectedly and meddle with your toys. . . . .

I was thinking of fields being private, as they always should be, and some methods being restricted to package‑private access.

We disabled the edit button a long time ago because we had problems with people editing code after it had been replied about.

While I tend to agree, the authors of the JPA spec, for one, don't restrict themselves that way.

You can define a JPA Entity class in one of 2 modes:

Field Access - where you access the columns directly


Property Access - where you use the set/get methods

The access type is locked in when you design the class - everything must be either Field Access or Property Access for the class.

Note that Field Access variables are not package-scope, they are public.

While I don't care for this approach myself, I think an old project I worked on required Field Access for historical reasons or something.
1 day ago

Kaine van Gemert wrote:I can't find an edit button so I'm just going to post here.

I was just wondering are package-private methods/members considered bad practice? I only ask as I haven't really come across them when reading other code.

The idea of public, protected and private in Java comes, I think from C++, where the distinctions were more important and there was no package hierarchy. C++ also had a concept known as "friend" that allowed other classes restricted access into what would otherwise be private areas.

The biggest reason not to use package-scope for properties is because it's hard to read and maintain code where someone can come in more or less unexpectedly and meddle with your toys. Vertical meddling (via inheritance) is fairly easy to track, but horizontal meddling generally means a much larger number of potential meddlers and they don't identify themselves in the meddled object.

One case where package scope is very useful, however, is in unit-testing where the test modules need limited access to the internal features of the class under test.
2 days ago
In the abstract MVC architecture, the View is the display(s) and the Model(s) the repository for data value(s) being displayed. The Controllers are what link the two.

A View is generally organized as a tree of view elements (a directed acyclic graph). You have your parent (which is generally a window, although windows themselves may be children of a larger display element tree), the window has menus, toolbars, a client area, and so forth. In the client area (or areas, in the case of a multi-pane window) you have sub-areas, going all the way down to the atomic element such as labels, textboxes, and radio buttons.

To bind these elements to a model or models, you have the Controllers. A Controller's sole purpose in life is to make sure that the data value displayed in the View matches the data value in the model and vice versa.

This update process may be instantaneous, or it may be done in response to a trigger event, such as a web page submit initiating the HTTP Request/Response cycle, but once a change has been committed to a control or its model, the controller is obliged to see that the other side is updated to reflect that change.

In short, a "pure" MVC system has no business methods because all it's about are keeping the Model and View in sync. Triggers to incite business processes are sort of a "leakage" out of the pure MVC world and into the unknown, as it were.

I've mentioned the many/many aspect of MVC. My favorite classic example is a View containing a table and a graph, perhaps a pie chart. Both the table and chart may be representing the same data from the same model but in visually different ways. Change the model, however, and both views must update. Likewise, a display may obtain data from more than one source (Model).

Like Views, Controllers are usually formed into trees more or less following the control elements in the View tree, but linking each control element with a Model element on a one-to-one basis. Controllers have a very limited application so it's common to use canned controllers. In JavaServer Faces, for example, an application programmer almost never codes a Controller, since the Controller logic resides in the dispatcher (FacesServlet) and in the implementations of the various JSF tags.

When I use the word "container" for the model, it would be just as accurate to say that a Model is a heterogeneous tuple of values with discrete identities. The values themselves may be compound objects (class intances, including collections). In a similar vein, the parent of a View is the container for its sub-Views as they may in turn contain lower-level sub-Views. Controllers are linkages between value identities and sub-views, so they are not directly hierarchical, but there are compound controllers as well. The most famous of which is probably the Combo Box, although a collection of Radio Buttons would also be assembled as a collection of individual co-ordinated button elements.
2 days ago
Ideally in a Model/View/Controller architecture, the Model is a passive container for the GUI (or GUIs, since MVC supports many-to-many model/view relationships). That's usually not entirely true because, for example the mouse/menu/hotkey/button action controllers often fire methods in the Model, but close enough. Your business logic is normally responsible for taking the model data and acting on it, including passing it on to the persistence layer, if applicable. And except in the very simplest of cases, business logic should not be part of the Model object.

One issue I've often seen in JavaServer Faces is that people have attempted to exploit the fact that both JSF GUI models (Backing Beans) and ORM Entity Models are POJOs and tried to use the same object for both. Which, admittedly can be done, although only in a limited fashion. However since there are all sorts of "gotcha"s in this sort of design I generally recommend not doing that.

Never missing a chance to be pedantic, you don't actually "pass data" to the GUI when designing with MVC. That's what the Controllers are for. And for passing data values from the GUI back to the model, keeping them in sync.
2 days ago