Working with Hadoop under Eclipse

Last week i was setting up development environment for Hadoop under the Eclipse IDE by following the instructions provided by Apache  . But the installation wasn't as smooth as one would hope. Some hassles were there which are not documented (or maybe were invisible to me). Anyways i am writing about the errors faced by me in this blog.

Maven installation

• you need maven to run the mvn command.
• While the documentation at maven download page is given for Windows 2000/XP or UNIX based systems.But it does not map correctly on win 7.
• In Windows 7 one need to make the  M2_HOME and M2 environment variables (see the installation instructions here) as system variables and then they should be added to the path.
• Other instructions are same as documented in release notes by APACHE.

 Now after running the $ mvn install -DskipTests command the building of the projects                started. But one of the projects Apache Hadoop Common Project failed to build and reported MojoFailureException.

• You need to install protobuf (protocol buffers) and add it to your path variable to build the project successfully.Download it from here.

Now the next project to fail in build was hadoop-yarn-common.

• The build process failed because windows was not able to run "saveVersion.sh" file successfully.
• You need to add following patch to the Pom.xml file in hadoop yarn common to allow windows to run the  "saveVersion.sh" file successfully.

After these steps the build process will complete itself successfully.and you can run the next command to create eclipse project.

$ mvn eclipse:eclipse -DdownloadSources=true -DdownloadJavadocs=true

Now the projects will be ready to be imported in eclipse.

Hope these points would save other developers some time in installation.:-)

FBcoverprofilecombo

This post explains the Html5 and JavaScript app Create Facebook cover and profile pic combo.The major elements used in this app are Canvas and Drag and drop API of Html5.

Drag and Drop API is used to upload the images from the user.It makes the look and feel of the app trendy and reliable.Following event handler in JavaScript does that dirty work:

 dropzone.on('drop', function(e) {  
           //prevent browser from open the file when drop off  
           e.stopPropagation();  
           e.preventDefault();  
           dropzone.removeClass('hover');  
           //retrieve uploaded files data  
           var files = e.originalEvent.dataTransfer.files;  
           processFiles(files);  
           return false;  
      });  

Now the file is processed to create a cover image with the use of Canvas element of HTML5 in following function.The image settings are set depending on the field chosen (page or profile).Canvas element does the task of resizing the image.

 var getCanvasImage = function(image) {  
           //define canvas  
           var croping = $('input[name=croping]:checked').val();  
         //check for page or profile  
           if(croping == 'crop') {  
                imgWidth = 958;             imgHeight = 450;  
           }else{  
           imgWidth = 958;            imgHeight = 419;  
           }  
           var canvas = document.createElement('canvas');  
           canvas.width = imgWidth;  
           canvas.height = imgHeight;  
           var ctx = canvas.getContext('2d');  
           //draw canvas image       
           ctx.drawImage(image,0,0,image.width,image.height,0,0,imgWidth,imgHeight);   
           //convert canvas to jpeg url  
           return canvas.toDataURL("image/jpeg");  
      }  

After the image is created The above function returns the source of the cover image.Now this image is processed to create profile photo in the following event handler and later appended to the body of document.

 createprofile.on('click',function(){  
           var cw,ch,a,sx,sy;  
      var croping = $('input[name=croping]:checked').val();  
      if(croping == 'crop') {  
                cw=180;ch=180;sx=28;sy=240;  
           }else{  
           cw=190;ch=190;sx=28;sy=220;  
           }  
           var covershhh=document.getElementById("cov");  
           var canvs = document.createElement('Canvas');  
           canvs.width = cw;  
           canvs.height = ch;  
           var ctx = canvs.getContext('2d');  
           //draw canvas image                           
           ctx.drawImage(covershhh, 28, sy, cw, ch, 0, 0, cw, ch);
           var oImg=document.createElement('img');       
           oImg.id="gdfg";                      
           var a=canvs.toDataURL("image/jpeg");  
           oImg.src=a;
           document.getElementById("result2").appendChild(oImg);           
      });  

Note:All the measurements are taken are in pixels.In next post i will share some of the creative timelines created by this app.Till then good bye!!

Don't forget to give your feedback and encouragement.