Win a copy of The Java Performance Companion this week in the Performance forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Decommission of Node

 
jain Shasha
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi

I want to understand how the decommission of a datanode happen on a hadoop cluster.
Also how much time does decommissioning would take.

Waiting for the reply.

Thanks.
 
sreekanth jella
Greenhorn
Posts: 10
Java Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
From Hadoop definitive guide book:
The decommissioning process is controlled by an exclude file, which for HDFS is set by the dfs.hosts.exclude property and for MapReduce by the mapred.hosts.exclude property. It is often the case that these properties refer to the same file. The exclude file lists the nodes that are not permitted to connect to the cluster.

To remove nodes from the cluster:
1. Add the network addresses of the nodes to be decommissioned to the exclude file. Do not update the include file at this point.
2. Update the namenode with the new set of permitted datanodes, with this command:
% hadoop dfsadmin -refreshNodes
3. Update the jobtracker with the new set of permitted tasktrackers using:
% hadoop mradmin -refreshNodes
 
jain Shasha
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the reply

Also how much does the decommission of a datanode took.
Because when i try to do the above steps it was taking a lot of time and i saw the same status Decommission in progress, It never changed to Decommissioned
Also what happen to the data store on the datanode (which we will be decommission) ?
 
sreekanth jella
Greenhorn
Posts: 10
Java Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The data that exist in this data node will be replicated by HDFS to available data nodes. I have not tried practically, but nodes should be decommissioned when the command "% hadoop dfsadmin -refreshNodes" is issued.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic