Avor Nadal

Ranch Hand
+ Follow
since Sep 15, 2010
Avor likes ...
Java Netbeans IDE Postgres Database
Cows and Likes
Total received
In last 30 days
Total given
Total received
Received in last 30 days
Total given
Given in last 30 days
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Avor Nadal


In my company, I've been asked to find out why an application of the intranet, from time to time, consumes 100% of the CPU time and gets stuck there. The server is a legacy system, which uses CentOS 6.7, JBoss 4.2.2 GA and JDK 1.5.0 Update 22. The application is an ERP running in JBoss.

The first thing that I've done is to use the command top -H -p <pid> to find out which thread was the one exhausting the CPU and see what . Then, I've run jstack <pid> to get a thread dump and associate the PID of the thread at the OS level, with the thread ID or name at the JVM level. However, this version of Java is old and doesn't show the PID at the OS level (in hexadecimal), as newer versions do. It even doesn't show the actual thread name, just a generic name of the form "Thread XXXXX". Is there a way to do this association in this version of Java? If so, How?

I've also accessed the JMX Console provided by JBoss, using a web browser. I hadn't used this version for years, but I've found something interesting under jboss.system:type=ServerInfo. This MBean provides a method called listThreadCpuUtilization() which is close to what I need. But it doesn't show the CPU usage in a given time interval, but the total usage since each thread was created. So I've created a shell script that queries this MBean every X seconds and calculates the usage in such interval. What's the problem now? That the JMX Console remains inaccessible when the ERP exhausts the CPU. So it's not much helpful.

Since the machine has a X server running in it, I've also tried to run JConsole and VisualVM. JConsole says that the JBoss process is not a managed VM. VisualVM lists the process, but gets stuck when I want to see the details.

Can you help me, please?

Thank you!
1 week ago
Thank you for such a detailed explanation. I see that I'm already applying most of your recommendations to my code (fortunately). Specially the part about the internal resources. Indeed, I've an unchecked exception dedicated only to wrap checked exceptions that (should) never occur.

About adding a "Details" button to show the stack trace, I consider it a very good idea. I'm going to try to apply it to my projects from now on.

Again, thank you fro your advices!
1 year ago

Stephan van Hulst wrote:Yes, that's a good strategy. There's actually a pretty good way to deal with this if you're in control of most of the layers, and if you can design consistent APIs.

It comforts me to know that it's an acceptable way of dealing with exceptions. I still had some doubts.

Stephan van Hulst wrote:Like I said, for unchecked exceptions it doesn't really matter that much, because they're caused by bugs in your application that necessary will be have to fixed by a programmer, not by the end user. A generic message will do, if you don't want the application to crash catastrophically.

For checked exceptions, the code knows ...

Perfect. That's exactly how I'm doing it. I let the unchecked exceptions bubble up, whereas I wrap the checked ones with other exceptions that are appropriate to the current layer.

Stephan van Hulst wrote:Even then, if you want to prevent a long chain of checked exceptions, you can throw a checked exception that's appropriate for your abstraction layer, and as its underlying cause pass the cause of the wrapper that you caught. If you do that consistently, there will only be two exceptions: The wrapper that's appropriate for your abstraction layer, and the underlying cause that's at the very root of the issue:

Good idea! I'm going to investigate how I can adapt that to my code. It's an interesting way of easing the navigation through the exceptions chain.

Question about wrapping checked exceptions that are too specific to the implementation, but which are solvable by the final user.

I still want to have something clarified. Would you be so kind to give your opinion, please?

In your example, I've seen that you've used exceptions that belong to the same abstraction layer or that, at least, are related. But what do you do when the exception is from a lower layer, and in spite of that, the user could solve it?

I'm referring to errors that depend on external resources with which the user can interact. Basically, these are errors related to the file system (file permissions, lack of space...), the network (network is down...) or other sources of data input.

What would you do in these circumstances?

Option A: wrap these exception(s) with an unchecked exception. Then, let it bubble up, log it in the top layer, and show a generic message to the user. Maybe with a reference to the log file. That would require the user to be able to read stack traces and speak English, of course.

Option B: dissect the exception chain in the top layer, to build a human-readable message. Just as we would do with the "normal" checked exceptions. In this case, we would need to work with exceptions belonging to the implementation, which are subject to change in the future. So it's a little risky.

But the alternative is to work with the original messages from the exceptions and parse them to get details, and build localized messages. This would make the application very brittle too, because just a comma moved here or there, and the application would crash too.
1 year ago
Thank you for your answer, Stephan. Yes, they're checked exceptions.

I'm asking this question because I believe that the information that the outermost exception can provide, is very limited. I'm referring to the exception that is on the top of the exceptions chain. Very often, I can only show a very generic message to the user, because this exception doesn't contain information that is useful enough.

With regard to this matter, some days ago I've started a new project and I've been reconsidering the design of my exceptions. After many tests, I've came to the conclusion that the best way of reusing code and of not violating the separation of layers, is to wrap the exceptions thrown by the implementation code with a single wrapper exception. One that belongs to the next higher layer. It doesn't matter if doesn't "say" much about the true error: I let top layer of the call stack to traverse the chain to know what really happened.
1 year ago

Many guides recommend us that, when we work with abstraction layers, we always wrap/chain the exceptions thrown by the internal code, with exceptions that belong to our own abstraction layer, or that at least have something to do with it. That seems reasonable and I agree with that.

However, there is something that worries me constantly. The wrapper exceptions always hide some information that could be important for the user to solve the problem appropriately. Such information is contained in the wrapped/chained exceptions, but we're supposed not to inspect them in higher layers, because that would violate the separation in layers. So, How are we supposed to inform the final user about that details? Do we show her a simple message, relative to our layer, but recommend her to check the logs if she wants more details? Even though the content of a log file may be too technical or may not be in her native language? Or are we "allowed" to inspect the wrapped exceptions if the user wants more details, to show that information in a form that is clearer that in the logs?

I'm aware that there are details that only an advanced user could understand, so she could look for them directly in the logs. But there is other type of information, like the lack of space in disk or that the connection is down, that could be understood by an average user. In those cases, it would be useful to show them something more descriptive than the typical "The operation failed". What do you think?

Thank you.
1 year ago

All Java EE servers that I know allow to deploy exploded applications/modules. This makes it possible to customize the configuration files and the deployment descriptors specifically for each deployment, which is great. But this also means that, every time that you release a new version, you've to manually decompress the file(s) and re-configure the same parameters again and again.

Because of that, I'm experimenting with deploying my applications as pure EAR/WAR files, and loading the configuration specific to my application from a properties file, situated out of the compressed file. The path to load that file is obtained from a system property, which is passed to the server on start.

I like this solution, but I'm realizing that there are aspects related to the deployment that can't be externalized. Indeed, I doubt that this situation ever change, given the nature of Java EE. For example, the data-source used for the persistence unit must still be indicated in the "persistence.xml" file. Or the context root of a Web module, or the virtual host in which it has to run, which have to be set in a file specific of each server ("jboss-web.xml" in the case of Wildfly).

So my question is this: Is the idea of deploying a static EAR/WAR file just an illusion in most of the real situations? Because, except the very simple applications, all them require that you configure at least the data-source to the database.

Thank you.

Jeanne Boyarsky wrote:After three days of no replies, I suspect there isn't a better approach. Which is certainly possible. Maven is big on "the Maven way". And then the IDE doesn't exist so it isn't a problem to change the files in the original source directory.

Yes, I guess there isn't any other solution. I'll have to conform to the way of doing things of Maven, or to go back to Ant. Thanks!
2 years ago
Thank you for answering me, Jeanne. That's something that I also considered at the beginning. But I ruled the possibility out because, as you mention, it didn't seem clean/elegant. Specially because NetBeans notices the changes in the files and you can see how it re-scans the file tree. But I've to admit that, among all the options presented here, it's the only one that would really work. At least until someone presents a cleaner way, of course.
2 years ago

I want to know it it's possible, with Maven, to post-process the original source files of my project, generate the resulting files in an intermediate source directory, and make "maven-compiler-plugin" to compile from there instead of from the original directory. By "post-process" I mean to process the source files with something that replaces text sequences, be it a custom plug-in or a typical command such as sed or perl. But my question is not related to this last point, but to configuring the directory.

With Ant it's moderately easy. But with Maven, it doesn't seem so. I prefer not to change the <sourceDirectory> tag in <build>, because I still want that the original directory is the reference one. One of the reasons for this is that NetBeans, which is the IDE that I'm using, wouldn't show the original files in its project panel, but the post-processed ones. I couldn't edit the original files with ease. I would lose the ability to refactor, the automatic dependency check, etc.

One idea that I've had is to try with the <includes> configuration parameter of the maven-compiler-plugin. But I've read that it was deprecated several versions ago.

Another idea is to use the build-helper-maven-plugin and its add-source goal. To add the directory of post-processed files as an additional source, and expect that the compiled files overwrite the ones of the original. However, the compiler actually fails indicating that the classes are duplicated (I've just tested it).

Do you have a better idea? Thank you!
2 years ago

Dave Tolls wrote:Creating indexes, and getting the correct indexes on a table are how you handle large data sets.

By "registries" do you mean "rows"?

If so, then 27 million, especially if most of them are in tables that only have primitives (eg numbers) that were previously in your ARRAY columns, is not a great deal.
Those tables will be quick look ups, with the correct indexing.

Yes, I meant "rows". I usually use the term "registry" in my Java EE applications to be as much generic as possible about the storage back-end that I use. Even though I always use a database. But yes, in terms of relational databases, the right term is "row". Sorry ;) .

I agree about the importance of using the appropriate indexes. But back in that time, when I made the switch from an ARRAY column to actual rows, I had not much idea about indexes and the query planner. So it was horrible to see such a decrease of performance :S . Nowadays I always create the indexes that I need. But I admit that, from time to time, I wonder whether people use ARRAY columns, and whether my decision was the right one. But by your words, it seems that that type of column isn't so common .

I also take the opportunity to say that sometimes, at least in PostgreSQL, indexes are not enough. Because the query planner also takes into account some configuration parameters, and it may decide not to use your index if it considers that it will be more expensive that a sequence scan. Indeed, yesterday I experienced one of these situations. I almost become crazy, he he he. It turned out to be because of the value of the "random_page_cost" parameter, which was set to 4. The expected time cost calculated by the planner was not accurate at all. I did some tests using the EXPLAIN command and forcing the use of index scans, and the planner happened to be completely wrong. By a width margin. So I decreased that parameter to 2 and the the situation improved a lot. Obviously, all these matters depend on the hardware and OS of each machine. It's an art... From which I still have to learn a lot.
Hello Roel:

Thanks for your answer. Really it's not a question directly related to one of my current projects. I simply was curious about the fact that you can't get an array more efficiently, with primitives instead of Objects.

Besides that, Is this type of column so rare? By your words, it sounds like that. It's true that I have only used it once, and because I didn't design the BBDD myself. But I thought that people used this type more frequently, to reduce the quantity of registries in a table and improve the performance. For example, when I had to re-design the BBDD that I've mentioned, I decided to replace that column of type ARRAY with actual registries (arranged "vertically" instead of "horizontally"). I made this change because, in my opinion, the elements of the array broke the design and homogeneity of the rest of tables, which always expanded "vertically". But after the switch I went from 1,214,300 registries, to... 27,420,200! That's a big difference. I had to modify and create some indexes to fix some SQL queries that had turned very slow. So I've always wondered if it was the right decision, and whether people use the ARRAY type for these specific scenarios.

I'm curious as how JDBC handle arrays of primitive types. I know that it can directly work with arrays of bytes (byte[]). But I'm referring to the other primitive types: int[], long[], etc. I've been checking the java.sql.Array interface and it seems that it only can work with arrays of Objects that you must cast explicitly: Integer[], Long[]... Am I right?

Thank you.

Junilu Lacar wrote:Not everything that is machine readable has to be in binary format. JSON, for example, is machine readable as well as human readable. It's machine readable because it has a specific format that can be processed programmatically. CSV is another machine readable format. Properties (key=value pairs) are also machine readable. Free formatted prose does not lend itself well to being machine readable with current technology. You'd need to have some kind of natural language parser and interpreter. Maybe IBM Watson will lead us there someday, but I digress. Anyway, I hope that clarifies what I meant.

OK, clarified. So you referred to use a syntax that is standard, not a custom implementation. Right? Before posting my questions here, I read at least 2 articles that used JSON as output, just as you recommend. They called it "structured log". One of that articles was Structured Logs at Nextdoor. I think I'll give it a try. It shouldn't cost much to move from one format to the other one. Thanks.

It's funny, because the application server Wildfly follows an inverse philosophy. Its CLI outputs JSON as result of an operation, whereas its logs are more human-readable than that, he he he.

Tim Cooke wrote:In your other topic about Logging I mentioned that the company I currently work for uses Splunk to capture the logging output from application servers. I also mentioned that it also serves as a pretty darn good data analysis tool too. Well Splunk loves key=value pairs and is really good at finding them quickly, so taking your example I could write a simple search for "sessionId=97fd45ef21225b079967efd" to obtain an audit of everything that occurred during that user session.

Great. Specially the part of generating a ticket automatically. That could save me some time when an error occurs. I took a quick look to the Splunk website some days ago. But I still have to dig in it much more. Again, thank you for the recommendation.
2 years ago

Les Morgan wrote:I like your format better, and I agree: it is a difference audience that you write for when you do log files--people that read them tend to be more left brain dependent, so more info in a smaller space is better instead of flowery sentences.

Wonderful. I was afraid that the advantages of this format were too subjective. Thanks.

Junilu Lacar wrote:You can take it a step further and format all or a subset of your log messages so that they are machine readable. That way, it's easier to automate. You could put a marker on the machine-readable log messages or you could append them to a separate log altogether.

What do you mean with "machine readable"? I'm not sure if I've understand you correctly. Do you refer to some kind of binary format?
2 years ago

Junilu Lacar wrote:The applications I support log most user activity. We also have very long User Agreements that users need to accept when they sign up and use our apps.

Unfortunately we're a small company with little resources. We don't have a legal department. But I'm trying to put more attention to the legal terms of our applications. Specially because the European Union is very strict in that sense. So yes, I guess that we'll have to ask for support from a consultant office. Thank you ;) .
2 years ago