Win a copy of Beginning Java 17 Fundamentals: Object-Oriented Programming in Java 17 this week in the Java in General forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Tim Cooke
  • Campbell Ritchie
  • Ron McLeod
  • Liutauras Vilda
  • Jeanne Boyarsky
Sheriffs:
  • Junilu Lacar
  • Rob Spoor
  • Paul Clapham
Saloon Keepers:
  • Tim Holloway
  • Tim Moores
  • Jesse Silverman
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Piet Souris
  • Frits Walraven

Storm Applied: Deploying Storm on popular infrastructure services

 
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Sean, Matthew and Peter!

As an independent developer with average sysadmin skills, one problem I've faced while exploring big data products is deploying them cost effectively at modest scale on infra services like AWS and Linode.
Often, the steps to do this the right way are not there, and discovering them becomes a matter of (costly) trial and error.

Does your book cover *deploying* Storm at scale on such services? What are your opinions on its ease of deployment?
 
Author
Posts: 14
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
We don't cover any specific service such as AWS etc, we do however go through how to deploy Storm.

Deploying Storm is relatively easy. We strongly advise that you use a tool like Puppet to allow for recreateable deploys.
From there, you can easily deploy new nodes to the Cluster and what not.

My #1 piece of advise if you are setting up a cluster is to pay close attention to Zookeeper. Its a vital part of a Storm cluster.
Make sure that you are running Zookeeper machines with fast disk as its IO intensive and a lack of stability with Zookeeper
will impact on your entire cluster.
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you for the reply and the ZK tip! It's good to know Storm is fairly easy to deploy. Planning to try it out.

Wishing you great success with your book promotions!
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic