• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Bear Bibeault
  • Junilu Lacar
  • Martin Vashko
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Knute Snortum
Saloon Keepers:
  • Ron McLeod
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Scott Selikoff
  • salvin francis
  • Piet Souris

NameNode,DataNode and hdfs concepts

 
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
1)I have fair understanding of the hadooop and MapReduce concepts .but still I need to understand these concepts on a higher technical level. I know that NameNode stores the metadata information about the data and DataNode stores the actual data.BUT I am not understanding what makes the node as NameNode and Data Node? i mean what special things do we need to make it a NameNode and DataNode?

2)What is the difference between HDFS and local file system in hadoop ? I have understood that the mapper output is stored in local file system and the reducer output is stored on the HDFS. Also if there is no reducer then the mapper output is stored directly on the HDFS .What I am not understanding is where is HDFS and where is local file system located? How do I differentiate between the two as far as hadoop is concerned?

thanks
 
Ranch Hand
Posts: 354
Eclipse IDE Oracle Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

nawab wajid wrote:I have fair understanding of the hadooop and MapReduce concepts .but still I need to understand these concepts on a higher technical level. I know that NameNode stores the metadata information about the data and DataNode stores the actual data.BUT I am not understanding what makes the node as NameNode and Data Node? i mean what special things do we need to make it a NameNode and DataNode?



There are configuration files that specify the slave/data nodes.

nawab wajid wrote:
2)What is the difference between HDFS and local file system in hadoop ? I have understood that the mapper output is stored in local file system and the reducer output is stored on the HDFS. Also if there is no reducer then the mapper output is stored directly on the HDFS .What I am not understanding is where is HDFS and where is local file system located? How do I differentiate between the two as far as hadoop is concerned?



HDFS and Local Filesystems are non-overlapping. However, the File I/O is generic so you could use that to write to local FS as well as HDFS (see scheme in fs.default.name). The HDFS would not even be visible to the native 'ls' command.
 
Hot dog! An advertiser loves us THIS much:
Java file APIs (DOC, XLS, PDF, and many more)
https://products.aspose.com/total/java
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!