How to solve Datanode error in Hadoop

Sometimes the datanode daemon process does not start successfully when you run the startall.sh script in hadoop on both the single and multinode clusters.

The Datanode log gives java.io.IOException .Further it explains that namespaceid of namenode and datanode are not compatible.

This problem can be averted by changing the namesapce id of the datanode.

To change the namespace id of datanode follow these steps

1. Check the value of hadoop.tmp.dir in conf/core-site.xml file. 
2. As my value is /app/hadoop/tmp i will open that directory.
3. Now /app/hadoop/tmp/dfs/name/current/VERSION represents the version file of name node which contains its namenode.
4. /app/hadoop/tmp/dfs/data/current/VERSION contains the namespace id of datanode.
5. Both of these namespace ids need to match.
6. Delete the file /app/hadoop/tmp/dfs/data/current/VERSION and restart the namenode.
7. Now Datanode will also start normally.

The  Version file looks something like this

    

Configure eclipse map reduce plugin for hadoop

Hadoop Eclipse Plug-in can be downloaded from this link .

The Hadoop Eclipse Plug-in provides tools to ease the experience of Map/Reduce on Hadoop. Among other things, the plug-in provides support to:

1. create Mapper, Reducer, Driver classes;
2. browse and interact with distributed file systems;
3. submit jobs and monitor their execution.

Now to configure the plugin you need to follow these steps

1. Move  the jar file you have downloaded in the plugins folder of your eclipse installation.
2. Restart the Eclipse and switch to  map-reduce perspective from Window-> open perspective ->other 
3. Now set your DFS location by selecting create new location in Map/Reduce locations area.
4. Enter the name of your location and specify the ports.
5. Add the port numbers of JobTracker and DFS as specified in conf/mapred-site.xml and  conf/core-site.xml files.

  6.  Now create new map-reduce project and configure your Hadoop installation directory. 

  7. Now you can start off by creating mapper and reducer classes.

  8. To run your project select run on Hadoop in the run configuration.