Set up Hadoop on single node

In this Blog, you will be eduacted about the set up of hadoop on your machine.

• Hadoop requires a working Java 1.5  installation. However, using Java 1.6 or 1.7 recommended for running Hadoop 

• Download Java 1.7

• Set java path

• Do ls -a , this command will show u all hidden files in your home folder

• Or you can go to the Home folder and press ctr+h , it will show u all hidden files

• Search for ~/.bashrc file in home folder

• Add thes lines at the end of file in .bashrc file

 # Java Home Set up
            export JAVA_HOME="/usr/lib/jvm/java-7-oracle"
      export PATH=$JAVA_HOME/bin:$PATH
 

• Configuring SSH

• Hadoop requires SSH access to manage its nodes, i.e. remote machines plus your local machine

     if you want to use Hadoop on it (which is what we want to do in this short tutorial). For our Single

     Node Hadoop Set up we therefore need to configure SSH access to localhost

• Install ssh on your system if not present , open Terminal and Type this command

 sudo apt-get install ssh  

• Now check that you can ssh to the localhost without a passphrase:

 $ ssh localhost 

• If you cannot ssh to localhost without a passphrase, execute the following commands:

 $ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys 

• The first line create DSA key-pair with an empty password Generally, using an empty password is not recommended, but in this case it is needed to unlock the key without your interaction (you don’t want to enter the passphrase every time Hadoop interacts with its nodes).

• Second, you have to enable SSH access to your local machine with this newly created key.

• Now $ssh localhost

• Your localhost permanently added

• Now download the latest stable release of of Hadoop from apache website. hadoop tar file

• Downloaded hadoop-2.5.1.tar.gz file

• Unzipped the file in any folder.

• Unzipped the file in /opt folder it will look like /opt/hadoop-2.5.1 , you can also rename the folder from hadoop-2.5.1 to any name

• Set Hadoop Path

• Add thes lines at the end of file in .bashrc file

 #Hadoop Home Set up
export HADOOP_HOME="/opt/hadoop-2.5.1"
export PATH=$HADOOP_HOME/bin:$PATH

• Edit hadoop-env.sh in hadoop-2.5.1/etc/hadoop/hadoop-env.sh

 # The java implementation to use.
export JAVA_HOME="/usr/lib/jvm/java-7-oracle"
export HADOOP_PREFIX="/opt/hadoop-2.5.1"
 

• Re-open your teminal and Type hadoop, it will show script

• Your hadoop is set up in standlone mode.

• Run command hadoop version

• You will see the output

oodles@oodles-Latitude-3540:~$ hadoop version

Hadoop 2.5.1

Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r 2e18d179e4a8065b6a9f29cf2de9451891265cce

Compiled by jenkins on 2014-09-05T23:11Z

Compiled with protoc 2.5.0

From source with checksum 6424fcab95bfff8337780a181ad7c78

This command was run using /opt/hadoop-2.5.1/share/hadoop/common/hadoop-common-2.5.1.jar