Sunday, March 17, 2013

How to solve Datanode error in Hadoop

Sometimes the datanode daemon process does not start successfully when you run the startall.sh script in hadoop on both the single and multinode clusters.

The Datanode log gives java.io.IOException .Further it explains that namespaceid of namenode and datanode are not compatible.

This problem can be averted by changing the namesapce id of the datanode.

To change the namespace id of datanode follow these steps
  1. Check the value of hadoop.tmp.dir in conf/core-site.xml file. 
  2. As my value is /app/hadoop/tmp i will open that directory.
  3. Now /app/hadoop/tmp/dfs/name/current/VERSION represents the version file of name node which contains its namenode.
  4. /app/hadoop/tmp/dfs/data/current/VERSION contains the namespace id of datanode.
  5. Both of these namespace ids need to match.
  6. Delete the file /app/hadoop/tmp/dfs/data/current/VERSION and restart the namenode.
  7. Now Datanode will also start normally.
The  Version file looks something like this