Win a copy of The Little Book of Impediments (e-book only) this week in the Agile and Other Processes forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

NodeManager NameNode ResourceManager DataNode SecondaryNameNode not running

 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
i followed this link to install hadoop http://thepowerofdata.io/setting-up-a-apache-hadoop-2-7-single-node-on-ubuntu-14-04/
at the end when i run
the response i get is
while according to the link i should be getting

i also tried to run the daemons manually by commands

but i get [fatal error] like this


please help
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android C++ Java Linux PHP Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Did you copy and paste contents of core-site.xml and other .xml files from that site as is? Your xml files shouldn't contain lines like "...". Those are in the article just to show that the file can have other configuration properties before or after the one shown.
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator


this is my core-site.xml.
so i need to remove '...'
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android C++ Java Linux PHP Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes.
Also, "fs.default.name" was deprecated in 2.x in favour of "fs.defaultFS". Deprecated names usually work, but there are no guarantees that every version will support them going forward. It's recommended to not use deprecated names.
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
so what do i need to write instead of

after removing '..' i ran
but when i ran the response i am getting is

 
Karthik Shiraly
Bartender
Posts: 1210
25
Android C++ Java Linux PHP Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
so what do i need to write instead of fs.default.name

Read my previous reply again. And see the link given.

Exception in thread "main" java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId:
file:/usr/local/lib/hadoop-2.7.0/etc/hadoop/core-site.xml; lineNumber: 21; columnNumber: 2;

So, open core-site.xml in a text editor and see what character is at line #21 column #2.
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
i changed "fs.defaultFS" but yet it was showing errors at some line number. i gone through the files in editor where i found out that i have used the tag
twice in cases of all 4 .xml files (core-site,mapred-site,yarn-site,hdfs-site)(as i had just copy pasted from online source)
after removing it i ran the commands

and finally i got the required response

 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thanks a lot Mr. Karthik Shiraly for efforts and patience. it was great help.
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android C++ Java Linux PHP Python
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Good to know it's working now

I'll close this thread as resolved.
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
now that it's successfully installed, what steps would you suggest me to learn hadoop. i am sure by now you know my level in coding. which is poor. so please suggest me some path to understand and learn hadoop. i want to start my career in hadoop as soon as possible and i am ready to give 6-8hours a daily for it.
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android C++ Java Linux PHP Python
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
i want to start my career in hadoop as soon as possible and i am ready to give 6-8hours daily for it.

Reality:
I gather you're a fresher or recently out of college, yes? When I google "hadoop jobs", I don't see anything for freshers. Even the most minimal experience required was 2+ years.
So you are going up against the reality of how companies hire for these technologies.

The reason is simple.
For companies, these tools - hadoop/spark/storm, etc - are mere tools. Just glorified excel sheets, but on a massive scale, to extract useful business information out of their data.
If they could use excel, they'd definitely prefer that. But sometimes excel is too puny, and they have to resort to these big hammers.
Ultimately, they really want from tools is to derive maximum business value using them, and the fastest way is to hire people already
proficient and experienced in writing data mining algorithms using these tools.
Extracting information out of business data is a difficult problem on its own.
They don't want to waste time or money training employees from the ground up for this.

Required skills:
So at any interview, they'll start by testing your fundamentals of data structures, algorithm analysis, java/python, databases, SQL, and popular data formats like XML or JSON.
It also helps to understand some basics of statistics, because the quality of data as well as results of data mining are judged statistically.
These are the most fundamental skills necessary to write distributed data mining algorithms.
If you don't do well in them, they are unlikely to proceed to asking questions on technologies like hadoop, and they won't believe anything else you say.
Much practical knowledge in these areas can be picked up by simply working as a developer.

So before learning hadoop or whatever...

Learn your data structures and algorithm fundamentals.

Learn java better. Learn how to write code for basic data processing tasks like parsing and processing text files, XML and JSON.

Learn to use databases and SQL.

A lot of data science is actually done using python, and higher level technologies like Pig and Hive that build upon Hadoop.
Learn to write hadoop jobs in java, python, pig and hive.

Of course, to write these jobs, first you need problems to solve.
So think up your own data science problems to solve, or use something like Kaggle, and build a portfolio of them on Github or your own site.
They could be something very personal - like processing your own electricity bills.
Or you could try many of the public datasets available (check out data.gov.in for Indian open datasets).

Roadmap:
Research some small and mid-sized companies that have teams doing data science and data engineering, and is hiring freshers, but not necessarily for those teams.
Try and get hired in them. If you know your fundamentals well, small or mid sized companies won't reject you summarily for not having required experience.

Later, with some experience, you can try shifting to their data team. Even if that doesn't work out, you'd have built up enough skills to jump to another company's data team.
There are 2 kinds of data analysis - "exploratory" (where you extract information from available data), and "predictive" (where you predict what will happen from available data).
The latter comes in the realm of machine learning, and would be somewhat too advanced for you at this point. So concentrate on exploratory data analysis for now.
When you get hired as a data engineer, you'll anyway eventually run into predictive analysis at some point and you can pick up that skill then.

Good luck!
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thanks for helping me understand the scenario in IT for fresher.
being a fresher, would it be helpful for me if i clear hadoop certification as well as OCA(java crtification) .
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android C++ Java Linux PHP Python
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
akshay naidu wrote:being a fresher, would it be helpful for me if i clear hadoop certification as well as OCA(java certification) .


I doubt including any of these certifications in your resume makes it stand out in the minds of HR or even technical interviewers in India. It certainly won't hurt, but it's unlikely to help. It's my personal opinion.

But from a knowledge improvement point of view, probably yes, if you have found that you learn things better when studying for certifications.
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thanks for all the advices.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic