• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

verify hadoop installation

 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am very new to Hadoop. I am trying to run few sample prog. in Standalone (or local) mode. I have installed Cygwin in my windows machine. I am trying to run the command “hadoop version” to verify whether hadoop installed successfully or not. I have downloaded hadoop-2.2.0 version.



I have downloaded hadoop : C:\Users\Pavan\Downloads\

I tried to setup the PATH variable in Cygwin as below


$ echo $JAVA_HOME
C:\Program Files\Java\jdk1.6.0_32
$ export HADOOP_INSTALL=/cygdrive/c/Users/Pavan/Downloads/hadoop
$ export PATH=$PATH:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin

I am trying to run the command “hadoop version” and end up with the following error. Please help me in resolving this error.

$ hadoop version
java.lang.NoClassDefFoundError: org/apache/hadoop/util/VersionInfo
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.VersionInfo
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: org.apache.hadoop.util.VersionInfo. Program will exit.
Exception in thread "main"
 
Ranch Hand
Posts: 46
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
mm maybe your environment is not defined properly,

I share my environment defined in my cygwin user's .bashrc file [CYGWIN environment]

export JAVA_HOME=/cygdrive/c/Java/jdk1.6.0_23/
export JRE_HOME=/cygdrive/c/Java/jre6
export HADOOP_INSTALL=/home/saivulle/apps/hadoop-1.2.1
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin:$PATH

then from cygwin command window

$ hadoop version
Hadoop 1.2.1
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152
Compiled by mattf on Mon Jul 22 15:23:09 PDT 2013
From source with checksum 6923c86528809c4e7e6f493b6b413a9a
This command was run using /C:/cygwin/home/saivulle/apps/hadoop-1.2.1/hadoop-core-1.2.1.jar
 
Bartender
Posts: 2407
36
Scala Python Oracle Postgres Database Linux
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you are a complete beginner, I would strongly recommend you try one of the pre-configured Hadoop virtual machines (VMs) available from various suppliers. Hadoop is really horrible to install and configure, especially if you want to try using some of the related tools like Pig, Hive, etc. There are lots of compatibility problems between different versions of the various libraries, and it's generally a huge PITA to get working the first time you try it.

If your main goal is to learn how to install/administer Hadoop, then fine, install it yourself. But if your initial aim is to find out how to use Hadoop to do interesting things with Big Data, make life easy on yourself by using a pre-configured installation.

Cloudera has a Quickstart VM and Hortonworks has its developer sandbox. Both of these are Linux-based VMs with all the main Hadoop-related tools pre-installed and configured correctly, and the VMs can be downloaded for free (the files are pretty big) and run inside the (free) VirtualBox virtualisation environment.

Hortonworks sandbox comes with a lot of built-in tutorials and gives you a browser-based interface to upload files to HDFS, try out Hadoop processing with Pig, Hive etc.

Cloudera's VM is used as the basis for the free Udacity course Introduction to Hadoop and MapReduce, which is aimed at complete beginners and will take you through the basics of what Hadoop does. Choose the "free courseware" option when you sign up for the course.
 
danthuluri pavan kumar
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Thanks all for you response. I recently found the below information and thought might be useful to other hadoop Enthusiasts.

As of the last release, Hadoop 2.2 supports Windows natively. Thus, old workarounds like Cygwin are now un-supported. Please refer to the new documentation mentioned below (and copied here) for latest instructions for Hadoop on Windows.

https://wiki.apache.org/hadoop/Hadoop2OnWindows

If you are running Hadoop 1, then please check out branch-1-win and use that for running Hadoop 1 natively on Windows.


Now I am exploring hadoop-1.2.1 on windows using (cygwin + eclipse europa) , can anyone guide/help me in finding the information on eclipse plugin ?
 
Any sufficiently advanced technology will be used as a cat toy. And this tiny ad contains a very small cat:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic