• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Any Insight on how JavaRanch scales

 
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Guys,
I was just wondering if the JavaRanch team could provide some insight to how JForum is deployed to scale on this site.
You appear to be the largest implementation of JForum so some pointers would be appreciated.

The gist from what i understand on the JForum site is that it doesn't scale horizontally (ie you can't distribute one instance across several boxes)
http://jforum.net/posts/list/4898.page
http://jforum.net/posts/list/2553.page

Rafael basically says that it scales on one box and the best we could is run a cold backup connected to the common DB. Nor does it look like offloading reads to a replicated DB are allowed.

One solution that is posed is that in theory connecting several instances with a common cache (ie ehcache with multicast messages to advertise dirty caches) should work. However this doesn't appear to be supported.

Any information would be helpful.
Thanks
--Matthias
 
Trailboss
Posts: 23778
IntelliJ IDE Firefox Browser Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Others will be able to provide more accurate data, but I think we are currently using about 5% of our CPU and we are serving about 6.5 million page views per month. And this is with caching turned off.

I've been thinking that if there was a need to run on more than one box, it would be wise to divide the forums: some forums would be on one box and the rest on a second box. But I don't see that happening.
 
m lathe
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the reply.

So the application runs on one box (presumably commodity hardware) and the DB on another box (presumably MySQL on commodity hardware)?

Do you run a cold/warm standby?

We have also considered running parallel instances, where some boards are hosted on one silo of boxes (ie app and DB) and others are hosted on another silo. This is in effect just a "poor man's" sharding. Did you consider what would happen for shared resources (notably post counts etc?) would you just sync this in the background somehow?

Thanks again
--Matthias
 
paul wheaton
Trailboss
Posts: 23778
IntelliJ IDE Firefox Browser Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Who are you making forums for that you would need something bigger than what we have here?
 
m lathe
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It's for a customer website for a company. You're probably correct that the load will not be as large as what is found here. However I think it's more of a question of how do you build in redundancy so you can roll and test changes without bringing down the site and also how to deal with crashes without having to start up a cold backup.

I guess the lesson here is that even the big installs of JForum only deploy one instance. No one seems to attempt to cluster several instances with a cache.

Thanks
 
author and iconoclast
Posts: 24207
46
Mac OS X Eclipse IDE Chrome
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
We do have a completely separate install that we use as a test server, so after developers test locally, we deploy to the test server and bang on that. But I'll agree that JForum isn't really designed for five-nines operation.

 
author & internet detective
Posts: 41860
908
Eclipse IDE VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

m lathe wrote: Did you consider what would happen for shared resources (notably post counts etc?) would you just sync this in the background somehow?


I would imagine you would still have one database and just multiple app servers.

m lathe wrote:However I think it's more of a question of how do you build in redundancy so you can roll and test changes without bringing down the site


Who tests changes in production?

m lathe wrote: also how to deal with crashes without having to start up a cold backup.


The software doesn't crash. The only time we are down is for deployments. Which in a real company would be scheduled. If you mean the box crashing, you can have the software on another box and route the web server to it. Of course, starting up another instance of Tomcat doesn't take long.
 
m lathe
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Jeanne, your comments are apt. A suitably fast DB (worst case Oracle RAC) would probably allow allow you to run multiple app servers. That being said there might be some issues with how JForum writes to the DB that count on only one app running. One example might be to queue writes in the app layer and periodically flush them to the DB (ie view counts). I'm not sure if it does this, but it might.

Jeanne Boyarsky wrote:
Who tests changes in production?



Everyone should test in production, but you are right it shouldn't be the first time you test the change. If you are trying to get 100% uptime you really don't want to release a product to your customer until it was verified in production. So while you are doing functional testing in the test environments, you don't want to just slap a war into production and walk away. Ideally you have two nodes, you take one away from the customer, roll your change, verify, add it back and do the same to the second node.

Thanks everyone.

Please holler if you have any pointers for how to cluster JForum.
 
Jeanne Boyarsky
author & internet detective
Posts: 41860
908
Eclipse IDE VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You are correct that JForum caches things in a way that prevents clustering. I meant have an app server upp that nobody can access just in case something happens to the first one.

I don't see a forum as being so mission critical though. I work for a bank and we are still allowed X hours of downtime a year.
 
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
JForum supports several cache implementations, though, one of them being EhCache, which is billed as a "distributed cache". So you should be able to configure it to handle caches in multiple JVMs: http://ehcache.org/documentation/distributed_caching.html

That is probably easier to set up -and, IMO, architecturally cleaner- than having separate servers handle separate forums (which by itself wouldn't solve all caching issues).
 
reply
    Bookmark Topic Watch Topic
  • New Topic