学步园

    昨天从官网下的介质，运行第一个程序时，有如下警告信息：

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

   发现是本地库不能正常加载，从网上找了一些资料，总结起来对于我这个以学习为目的，而不单纯以解决问题为目的菜鸟来说，最直接有效的方法，就是自己编译。

   搜刮一通资料后，主要总结如下：

   1、安装protobuf   http://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz

./configure --prefix=/home/hadoop/local/protobuf-2.4.1  
make -j12  
make install 

    2、安装cmake  http://www.cmake.org/files/v2.8/cmake-2.8.12.tar.gz

./configure --prefix=/home/hadoop/local/cmake-2.8.12  
make -j24  
make install 

    3、安装maven  http://mirrors.hust.edu.cn/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz

       注：不要安装最新的3.1.1，因兼容性问题，会出现Java.lang.NoClassDefFoundError: org/sonatype/aether/graph/DependencyFilter之类的错误。

     因下载的是编译好的安装包，解压后，设置环境变量即可。

    4、安装findbugs  http://prdownloads.sourceforge.net/findbugs/findbugs-2.0.3.tar.gz?download

     解压后，设置环境变量即可

    5、获取hadoop 2.2.0源代码  http://mirrors.cnnic.cn/apache/hadoop/common/stable/hadoop-2.2.0-src.tar.gz

    6、解压hadoop源代码，将产生hadoop-2.2.0-src目录

    7、打补丁，补丁地址：https://issues.apache.org/jira/browse/HADOOP-10110，否则编译会有如下报错：

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:
[ERROR] /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13] cannot access org.mortbay.component.AbstractLifeCycle
[ERROR] class file for org.mortbay.component.AbstractLifeCycle not found

       补丁修复方法，就是编辑hadoop-common-project/hadoop-auth/pom.xml，参照如下文本修改和添加信息

     </dependency>
     <dependency>
       <groupId>org.mortbay.jetty</groupId>
+      <artifactId>jetty-util</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.mortbay.jetty</groupId>
       <artifactId>jetty</artifactId>
       <scope>test</scope>
     </dependency>

8、配置环境变量，编辑profile文件，增添如下信息：

export JAVA_HOME=/usr/java/jdk1.7.0_45
export FINDBUGS_HOME=/home/hadoop/findbugs-2.0.3
export PATH=$PATH:$HADOOP_HOME/bin:/home/hadoop/local/protobuf-2.4.1/bin:/home/hadoop/local/cmake-2.8.12/bin:/home/hadoop/apache-maven-3.0.5/bin:/home/hadoop/findbugs-2.0.3/bin:$JAVA_HOME/bin:$JAVA_HOME/jre/bin



9、不管你用什么样的办法，让环境变量生效
10、cd /home/hadoop/hadoop-2.2.0-src
11、
mvn package -Pdist,native,docs -DskipTests -Dtar
   12、编译完成后，正常的返回信息摘录如下：

[INFO] Apache Hadoop Pipes ............................... SUCCESS [8.970s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [2.757s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.098s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [1:20.217s]
[INFO] Apache Hadoop Client .............................. SUCCESS [11.118s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.524s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16:17.277s
[INFO] Finished at: Fri Dec 06 21:50:51 CST 2013
[INFO] Final Memory: 81M/247M
[INFO] ------------------------------------------------------------------------  
        13、在/home/hadoop/hadoop220/hadoop-2.2.0-src/hadoop-dist/target可以找到编译好的hadoop-2.2.0.tar.gz