Running Hadoop on CentOS Linux (Single-Node Cluster)

Follow the bellow steps:

1).Install CentOS iso image in VM Workstation.

2).download java1.6 based on CentOS 32 bit or 64 bit

for 32 bit 
jdk-6u43-linux-i586-rpm.bin

3).Give executable permission to jdk-6u43-linux-i586-rpm.bin

$chmod 755 jdk-6u43-linux-i586-rpm.bin

4).Install java from root user(where java is there) 

#./jdk-6u43-linux-i586-rpm.bin

5).export java home 
#export JAVA_HOME=/usr/java/jdk1.6.0_43

6).Download required hadoop version
ex:hadoop-1.0.3.tar.gz

7).come to trainig user(training)
#su training
password:password

8).unzip hadoop tar file
$cd hadoop
$tar -zxvf hadoop-1.o.3.tar.gz

9).making hadoop recoznize java
 cd /hadoop/conf
$vi hadoop-env.sh
and add following line
export JAVA_HOME=/usr/java/jdk1.6.0_43
save and quit

10)configuring HADOOP_HOME directory to hadoop instalation directory
export HADOOP_HOME 

11). Goto your home dir
$cd ~

open .bashrc file in vi editor and add following lines
export HADOOP_HOME=<hadoop installed location>
export PATH=$PATH:$HADOOP_HOME/bin

note:
a).add user to sudoers file
goto root user and open /etc/sudoers file and add following line
training ALL=(ALL) NOPASSWD:ALL

b).making update-alternatives working
add following line to .bashrc file in u r home dir

export PATH=$PATH:/sbin:/usr/sbin:/usr/local/sbin

c).make jps running

goto home $cd ~

open .bashrc file and add following line

export PATH=$PATH:/usr/java/jdk1.6.0_43/bin

d).for ssh configuration 
switch to root user

#ssh-keygen
enter
enter
enter

#cp .ssh/id_rsa.pub .ssh/authorized_keys
#chmod 700 .ssh
#chmod 640 .ssh/authorized_keys
#chmod 600 .ssh/id_rsa

#cp .ssh/authorized_keys /home/training/.ssh/authorized_keys
#cp .ssh/id_rsa /home/training/.ssh/id_rsa
#chown training /home/training/.ssh/authorized_keys
#chown training /home/training/.ssh/id_rsa

#chmod 700 /home/training/.ssh
#chmod 640 /home/training/.ssh/authorized_keys
#chmod 600 /home/training/.ssh/id_rsa

open /etc/ssh/ssh_config file and
make StrictHostChecking no

#service sshd restart



12).set all configurations

#vi /home/training/hadoop/hadoop-1.0.3/conf/core-site.xml
<property>
  <name>fs.default.name</name>
  <value>hdfs://localhost:8020</value>
</property>

#vi /home/training/hadoop/hadoop-1.0.3/conf/mapred-site.xml

<property>
  <name>mapred.job.tracker</name>
  <value>localhost:8021</value>
</property>

#vi /home/training/hadoop/hadoop-1.0.3/conf/hdfs-site.xml

<property>
  <name>dfs.replication</name>
  <value>1</value>
</property>


13).To format the namenode

$hadoop namenode -format

14).start all the services

$/home/training/hadoop/hadoop-1.0.3/bin/start-all.sh

15).Open browser and check weather the services start or not

 type the browser http://localhost:50070 or http://localhost:50030



16).Installing eclipse in node

change to root user and do the following
#

download eclipse 
ex:eclipse-java-europa-winter-linux-gtk.tar.gz

create a dir eclipse under /home/training

copy the downloaded file to eclipse folder and untar the file

tar -zxvf eclipse-java-europa-winter-linux-gtk.tar.gz

17). change permissions for eclipse dir

chmod -R +r /opt/eclipse 

18).Create Eclipse executable on /usr/bin path

touch /usr/bin/eclipse

chmod 755 /usr/bin/eclipse


## Open eclipse file with your favourite editor ##
nano -w /usr/bin/eclipse

## Paste following content to file ##
#!/bin/sh
export ECLIPSE_HOME="/home/training/eclipse"

$ECLIPSE_HOME/eclipse $*


19).bring eclipse icon on desktop

## Create following file, with our favourite editor ##
/usr/share/applications/eclipse.desktop

## Add following content to file and save ##
[Desktop Entry]
Encoding=UTF-8
Name=Eclipse
Comment=Eclipse SDK 4.2.1
Exec=eclipse
Icon=/home/training/eclipse/icon.xpm
Terminal=false
Type=Application
Categories=GNOME;Application;Development;
StartupNotify=true

after successful installation goto Applications->programming->eclipse->right click and ->addthis launcher to desktop

launch eclipse by double clicking eclipse icon on desktop

click on new project-->select mapreduce project-->click on configure Hadoop Install dir adnd give <hadoop install location>

20).Installing hive

download any stable version of hive 
ex:hive-0.10.0-bin.tar.gz

unzip the downloaded file
$tar -zxvf hive-0.10.0-bin.tar.gz

open .bashrc file and add following lines

export HIVE_HOME=/home/training/hive-0.10.0-bin
export PATH=$PATH:$HIVE_HOME/bin


$hadoop fs -mkdir /tmp
$hadoop fs -chmod a+w /tmp/
$hadoop fs -mkdir /user/hive/warehouse
$hadoop fs -chmod a+w /user/hive/warehouse

do the following to change log4j warnining issue
Change org.apache.hadoop.metrics.jvm.EventCounter to org.apache.hadoop.log.metrics.EventCounter in 
/home/training/hive-0.10.0-bin/conf/hive-log4j.properties.
/home/training/hive-0.10.0-bin/conf/hive-exec-log4j.properties.

To refer the websites:

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
http://www.bromlays.com/en/left/hadoop-single-node-configuration-centos/
http://linux.jamesjara.com/2012/02/installing-hadoophivederby-on-centos.html

http://linux.jamesjara.com/2012/02/installing-hadoophivederby-on-centos.html