• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Storm Applied: Deploying Storm on popular infrastructure services

 
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Sean, Matthew and Peter!

As an independent developer with average sysadmin skills, one problem I've faced while exploring big data products is deploying them cost effectively at modest scale on infra services like AWS and Linode.
Often, the steps to do this the right way are not there, and discovering them becomes a matter of (costly) trial and error.

Does your book cover *deploying* Storm at scale on such services? What are your opinions on its ease of deployment?
 
Author
Posts: 14
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
We don't cover any specific service such as AWS etc, we do however go through how to deploy Storm.

Deploying Storm is relatively easy. We strongly advise that you use a tool like Puppet to allow for recreateable deploys.
From there, you can easily deploy new nodes to the Cluster and what not.

My #1 piece of advise if you are setting up a cluster is to pay close attention to Zookeeper. Its a vital part of a Storm cluster.
Make sure that you are running Zookeeper machines with fast disk as its IO intensive and a lack of stability with Zookeeper
will impact on your entire cluster.
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you for the reply and the ZK tip! It's good to know Storm is fairly easy to deploy. Planning to try it out.

Wishing you great success with your book promotions!
 
Drove my Chevy to the levee but the levee was dry. A wrung this tiny ad and it was still dry.
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic