wood burning stoves 2.0*
The moose likes Hadoop and the fly likes Hadoop Clustering on Single machine Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of The Java EE 7 Tutorial Volume 1 or Volume 2 this week in the Java EE forum
or jQuery UI in Action in the JavaScript forum!
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "Hadoop Clustering on Single machine" Watch "Hadoop Clustering on Single machine" New topic
Author

Hadoop Clustering on Single machine

Mohamed El-Refaey
Ranch Hand

Joined: Dec 08, 2009
Posts: 119
Can I install a Hadoop cluster on a single machine? and How I can try its scalability features?

Regards,
Mohamed


Best Regards, Mohamed El-Refaey
www.egyptcloudforum.com
Arun Vasu
Greenhorn

Joined: Oct 05, 2010
Posts: 24

Yes, you can try a cluster on single machine. The good way to try out this is by using virtual machines.
I have tried it with VM of Centos O/S. The configuration I tried on my Windows 7 system is

Hardware: 4 gb ram @ 1600mhz, Interl i5 processor and 5400 prm hdd.

Software: Cloudera cdhv3, VMWare player v5, the virtual machine files for Centos O/S

Hadoop configuration: One Master node( Holds namenode, secondary name node and jobtracker), Three slave nodes (runs datanode and tasktracker on each)

This set up will help you to learn all the aspects of Hadoop technology.

Share if you find any difficuilties to try out it, am happy to help you!
Mohamed El-Refaey
Ranch Hand

Joined: Dec 08, 2009
Posts: 119
Arun Vasu wrote:Yes, you can try a cluster on single machine. The good way to try out this is by using virtual machines.
I have tried it with VM of Centos O/S. The configuration I tried on my Windows 7 system is

Hardware: 4 gb ram @ 1600mhz, Interl i5 processor and 5400 prm hdd.

Software: Cloudera cdhv3, VMWare player v5, the virtual machine files for Centos O/S

Hadoop configuration: One Master node( Holds namenode, secondary name node and jobtracker), Three slave nodes (runs datanode and tasktracker on each)

This set up will help you to learn all the aspects of Hadoop technology.

Share if you find any difficuilties to try out it, am happy to help you!


Thank you for sharing this, did you face any difficulties in network configurations for this cluster of VMs?
Arun Vasu
Greenhorn

Joined: Oct 05, 2010
Posts: 24

Make sure you have given proper entries in /etc/hosts file in the form of "ip address<tab>host_name".
Edit /etc/sysconfig/network and change the HOSTNAME= line to reflect your machine’s name. And your machine has an internet connection.

These are the only necessary network configuration you needs.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
 
subject: Hadoop Clustering on Single machine