• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Devaka Cooray
  • Paul Clapham
Sheriffs:
  • Tim Cooke
  • Knute Snortum
  • Bear Bibeault
Saloon Keepers:
  • Ron McLeod
  • Tim Moores
  • Stephan van Hulst
  • Piet Souris
  • Ganesh Patekar
Bartenders:
  • Frits Walraven
  • Carey Brown
  • Tim Holloway

My forum hangs sometimes after maxxing out DB connections

 
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It seems to happen about once a day. The forum just freezes up. It gets 51 DB connections and then it stops working. Usually I can fix that by shutting down Tomcat, and then shutting down Postgres, and then restarting them, but now it isn't coming back up at all. The site is getting about 50,000 hits per day.

Any ideas on this?

It's great software, but it doesn't seem very stable.

Thanks

[originally posted on jforum.net by CaliforniaCCW]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

CaliforniaCCW wrote:It seems to happen about once a day. The forum just freezes up. It gets 51 DB connections and then it stops working. Usually I can fix that by shutting down Tomcat, and then shutting down Postgres, and then restarting them, but now it isn't coming back up at all. The site is getting about 50,000 hits per day.

Any ideas on this?

It's great software, but it doesn't seem very stable.

Thanks



Which version of jForum you use?
Can you provide Tomcat's stdout_xxx.log file created just after jForum hangs?
[originally posted on jforum.net by Serge Maslyukov]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is the latest JForum, 2.1.6. This is what the process table looks like when it hangs:



(etc)

Only particular threads "hang" when this happens. It's happening right now in fact. Somehow I can't read a particular thread until this UPDATE waiting thing clears on the thread, but it never does until I restart Tomcat. Any ideas?

[originally posted on jforum.net by CaliforniaCCW]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

CaliforniaCCW wrote:This is the latest JForum, 2.1.6. This is what the process table looks like when it hangs:



(etc)

Only particular threads "hang" when this happens. It's happening right now in fact. Somehow I can't read a particular thread until this UPDATE waiting thing clears on the thread, but it never does until I restart Tomcat. Any ideas?


When you say "only particular threads hang" does that just mean it seems to be random? Or are the threads tied back to a particular forum or user?

[originally posted on jforum.net by GatorBait3]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It's seemingly random. Some threads become "locked", such that if you click on them, they never load. I look in the processes and there are these "update waiting" processes. I have to kill Tomcat to get it going again.

This is totally repeatable. It happens once a day.

Any ideas?

[originally posted on jforum.net by CaliforniaCCW]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

CaliforniaCCW wrote:It's seemingly random. Some threads become "locked", such that if you click on them, they never load. I look in the processes and there are these "update waiting" processes. I have to kill Tomcat to get it going again.

This is totally repeatable. It happens once a day.

Any ideas?


Probably, all of these "update waiting" processes are waiting on a table with foreign keys that is locked -- possibly locked by one of your "idle" processes. If any of the "idle" processes are not critical (e.g., one is just some statistics collector), try killing it and restarting it. Basically, some sql code has locked the table and hasn't freed the lock. This could be due to a seriously ugly postgres bug (some residual hash table problem) or it could be due to a plain ugly sql script bug (e.g., in jforum code, itself).
[originally posted on jforum.net by rseaton]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Better answer -- wait for version 2.1.7 (or whatever the next release is). v2.1.6 is dated 3/1/06. Note the following comment and it's date.

"Jforum deadlock problem solved?
Posted June 19, 2006

When cleaning out the needless jforum logic that determined how to handle non-transactional DB connections (sakai should always be transactional), I inadvertantly removed the commit() call. This should have been a pretty easy thing to debug��? if the DB connection is never committed, nothing should ever be saved to the database.

Oh, how naive! In sakai, we have a database connection pool. When jforum finished it�۪s request/response thread, it was returning its connection to the pool like any well behaved application should. The problem was that other application services were grabbing that same dirty connection, which was never committed and never rolled back. When that other service called commit() or rollBack(), the sql issued from jforum would suffer the same fate."

In more detail, it seems that Jforum code to update the thread-viewed and user-posted counts may be leaving tables locked. (Bad Cody!)
[originally posted on jforum.net by rseaton]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It's 99% likely a bug in JForum's SQL in that case. I'm running the latest version of Postgres (8.1.4 I think). PG is a very stable and mature database.

Any ideas on catching this bug?

Edited: Ah, ok, I just saw that post. It is a bug in JForum. Any ideas on when 2.1.7 is coming?

[originally posted on jforum.net by CaliforniaCCW]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well,

this is kinda confusing. If someone could help on this, it will be great.

CaliforniaCCW, JForum is very stable, but obviously not bug free. We'll do our best to fix all reported problems quickly, and we count with your help - and everybode else!

Let us know if you find anything new. I'll check the database code and error handling, to make sure nothing is being left behind.

Rafael
[originally posted on jforum.net by Rafael Steil]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Rafael Steil wrote:Well,

this is kinda confusing. If someone could help on this, it will be great.

CaliforniaCCW, JForum is very stable, but obviously not bug free. We'll do our best to fix all reported problems quickly, and we count with your help - and everybode else!

Let us know if you find anything new. I'll check the database code and error handling, to make sure nothing is being left behind.

Rafael


Thanks, Rafael. Apparently, when CaliforniaCCW is seeing a large number of visits and/or posts, one of the database processes looks like it's forgetting to unlock a row (or table) and this is causing other db processes to continue waiting on the lock. The user symptom is that attempting to visit a particular message (or section) never finishes loading, but other messages (or sections) can be loaded and/or refreshed. The admin symptom is that some db processes are sitting idle. When this happens, from the user's perspective there are typically more and more messages (or sections) which become autistic in this fashion until nothing on the forum site will load/refresh. My guess is that from the admin perspective, more and more idle db processes are waiting on locks (maybe not the same locks) until there are no more available db processes to reuse for new db requests. This appears to be a new behavior that you haven't seen before with Jforum. Thanks.
[originally posted on jforum.net by rseaton]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Rafael. It's 100% reproducible. We would probably get 100,000 hits a day if it didn't go down so often, so this is a high-volume site. Is there anything I can do to help you figure this out? I could set up any type of debugging you need, let you log in by SSH, etc. You name it. The site goes down at least once a day now.

[originally posted on jforum.net by CaliforniaCCW]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello
I am evaluating JForum to integrate it on a hight traffic website.
Looking at the code of the DAOs, I have noticed a disturbing thing : the jdbc operations are not wrapped in try/catch/finally blocks
This could leads to very bad things such as database cursor leaks.
Below a page explaining in detail how should be wrapped JDBC operations : http://www.javalobby.org/java/forums/t18930.html

Hope it helps

Alex


[originally posted on jforum.net by alexvictoor]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
At this point I would not recommend JForum for a high-traffic site. I have a high-traffic site and I have to restart the server at least once a day due to leaks of various kinds.

Other problems with JForum: It uses a cool page template system, but would be easier to use if it used plain old JSP. There are many more tools for JSP these days, and now JSP has expression language, etc, so it's quite nice. In fact, JSF would be a better way to go also. Another problem is JForum does not yet ship with a build.xml file so I'm not even completely sure what's the right way to build it.

As for using JDBC, there is no right way to use JDBC in a modern Java app. JDBC should not be used except in a few cases. The right way to do things now is with EJB3 or perhaps Hibernate. If you use EJB3 and Seam, JBoss will handle all the database and SQL issues. You can use this to build a whole site that has no need for JDBC or SQL code.

[originally posted on jforum.net by CaliforniaCCW]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I do not think there is no right way to use JDBC, I think it is not a good thing to use JDBC directly in your code, without any wrapper framework.
Spring-jdbc, iBatis ... there are many good frameworks out there.

CaliforniaCCW, if you are considering using stuff such as EJB3/seam and so on, you might look at Jboss portal bb module.



[originally posted on jforum.net by alexvictoor]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We already have so many posts within JForum, I don't think it would be possible to convert them to any other format.

On that subject... it sure would be cool if there were a common forum dump XML standard, so I could, say, dump all my forum data (users, posts, etc) into this XML file from VBulletin and then import it into JForum, that kind of thing. Right now, when you get a forum going in one type of software it is very hard to switch.

When I have time I might make my own fork of JForum, by moving it over into JBoss and switching more to EJBs and JSF for presentation.

[originally posted on jforum.net by CaliforniaCCW]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

alexvictoor wrote:Hello
I am evaluating JForum to integrate it on a hight traffic website.
Looking at the code of the DAOs, I have noticed a disturbing thing : the jdbc operations are not wrapped in try/catch/finally blocks
This could leads to very bad things such as database cursor leaks.
Below a page explaining in detail how should be wrapped JDBC operations : http://www.javalobby.org/java/forums/t18930.html

Hope it helps

Alex



CVS of jForum's DAO located here:
https://jforum.dev.java.net/source/browse/jforum/src/net/jforum/dao/

can you point to me where try/catch/finally are missed?


[originally posted on jforum.net by Serge Maslyukov]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
IMHO moving JForum to lighter solution such as spring and spring-JDBC whould be more effective. You wont get any permormance /reliability gain with JSF. JSF needs to keep a "control tree" in the http session... Your application will need more ram.
FYI an alpha release of Jboss forum, using JSF and hibernate, will be available at the end of the month.
[originally posted on jforum.net by alexvictoor]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
My deep apologies !!
I do not see any try/catch/finally missed in the CVS.
However, in the last stable release (2.1.6 if I am not wrong), the JDBC operations are not wrapped in try/catch/finally blocks.

[originally posted on jforum.net by alexvictoor]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
alexvictoor, the code is new for 2.17.

CaliforniaCCW, I'm adding to the cvs a new key to SystemGlobals.properties named "c3p0.extra.params", where it's possible to pass additional parameters to the connection pool.

The default value is



and it's possible to use any other configuration property of cC3P0.

We're trying to figure out how the database hangs. That's a relly anoying behaviour.

Rafael
[originally posted on jforum.net by Rafael Steil]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok, the changes are in the cvs now.

Setthing checkoutTimeout=120000 and debugUnreturnedConnectionStackTraces=true may help to trace the problem.

There are other interesting options in the C3P0 documentation

Rafael
[originally posted on jforum.net by Rafael Steil]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Rafael, if I am not wrong CaliforniaCCW is using jforum2.16 . Perhaps the new DAO code from the CVS would solve his problem...

[originally posted on jforum.net by alexvictoor]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I see.

I guess it's time to prepare an official beta build of 2.1.7

Rafael
[originally posted on jforum.net by Rafael Steil]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok, I put the beta download at http://www.jforum.net/beta/latest_dev.jsp

Who could test it, we appreciate.

Rafael
[originally posted on jforum.net by Rafael Steil]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

alexvictoor wrote:Rafael, if I am not wrong CaliforniaCCW is using jforum2.16 . Perhaps the new DAO code from the CVS would solve his problem...



Yes I'm using 2.16, the official release. I haven't tried any of the CVS code. When there is an official 2.17 release I'll definitely run it. I'm guessing that my forum starts hanging when people perhaps click "cancel" while a thread is loading, and that leaves some connection hung. With try-catch-finally that should fix it.

I'll be monitoring this forum eagerly for the next release. Everyone on my forum is eager for it too because the thing gets hung at least once a day. We have a lot of traffic, up to 80,000 hits a day lately. We would have more if the software didn't hang up.

[originally posted on jforum.net by CaliforniaCCW]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
CaliforniaCCW, the current cvs code is the one which will go as production. There isn't any planed changes to the database code.

I know that it sounds ridiculous to ask you to put a development package in your very busy website, so, if you want, I can be online on a specific time of the day and give you any assistance needed, for example.

Rafael
[originally posted on jforum.net by Rafael Steil]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Rafael Steil wrote:CaliforniaCCW, the current cvs code is the one which will go as production. There isn't any planed changes to the database code.

I know that it sounds ridiculous to ask you to put a development package in your very busy website,



I'm ready to do it! It's a good idea. I believe in testing code on production sites, in cases where it's not critical to the business.

so, if you want, I can be online on a specific time of the day and give you any assistance needed, for example.



Thank you for the offer Rafael! Actually if you can point me to a tar or a zip file that I should grab and deploy, I'll do it and post my experience here. We're a fairly high-traffic forum, with about 100,000 hits a day, about 100 posts a day, so we're a good test site.

My question: with the new tar ball, can I just install the jar files, or do I need to reinstall the .htm files also? The reason I ask is because i have made a few customizations to the templates so it would be easiest if I could just replace jar files.

Anyway... please send me a link to a zip file, or a CVS command to use and I'll put it on our live server over the weekend.

A few recommendations btw: I hope it has an ant build.xml file in it. It's much easier to work with software that has that. And if you're using CVS I would look into SVN. CVS hurts my brain. SVN is good.

Thanks again for the excellent forum software. Yes it has bugs and instability, but once that is resolved this is going to be a lot better than everything else, especially that horrible PHP forum software. Did you know that when vbulletin stores strings in the DB, it stores them with HTML-entities? It is painful to think about.

[originally posted on jforum.net by CaliforniaCCW]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello Jforum,

Can anyone point me to a zip file of a somewhat stable code release? My forum is now going over 100k hits a day and we are having outages several times a day. If we didn't have outages we would be over 200k hits a day. I would be deeply thankful if there's a release I could install to stop these outages. As I said above I would be happy to provide any type of debugging info related to this, even including letting the JForum developers ssh into the machine to see what's going on during the frequent outages.

Thanks

[originally posted on jforum.net by CaliforniaCCW]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sorry man. Here you go:

http://www.jforum.net/beta/latest_dev.jsp

Rafael
[originally posted on jforum.net by Rafael Steil]
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!