[Hadoop] Windows平台编译Hadoop2.6.0笔记

环境

64位虚拟机及64位 Windows Server 2008 R2

所需工具

• JDK7
• Maven
• .NET Framework 4
• Microsoft Windows SDK 7.1 安装前一定要先卸载比Microsoft Visual C++ 2010 x86Redistributable - 10.0.30319 更高的版本。
• Microsoft Visual C++ 2010 Service Pack 1 Compiler Update for the Windows SDK 7.1
• Cygwin (x64)
• Protocol Buffers 2.5.0
• CMake 3.2.1 安装时选择添加CMake到所有用户的PATH环境变量。
• hadoop-2.6.0源文件压缩包 解压至c:\hadoop-2.6.0-src

编译Hadoop2.6.0

1. 进入Windows SDK 7.1 Command Prompt
2. 在c:\执行buildHadoop.bat，其内容如下：

   setlocal
   
   set Platform=x64
   set CYGWIN_ROOT=C:\cygwin64
   set JAVA_HOME=C:\tools\Java\jdk7
   set M2_HOME=C:\tools\apache-maven
   set MS_BUILD_PATH=C:\Windows\Microsoft.NET\Framework64\v4.0.30319
   set MS_SDK=C:\Program Files\Microsoft SDKs\Windows\v7.1
   set CMAKE_PATH=C:\tools\CMake
   set PROTOBUF_PATH=C:\tools\protoc-2.5.0-win32
   set PATH=%PROTOBUF_PATH%;%CMAKE_PATH%\bin;%MS_BUILD_PATH%;%MS_SDK%\bin\;%MS_SDK%\Include\;%M2_HOME%\bin;%CYGWIN_ROOT%\bin;%PATH%
   set HADOOP_SRC=hadoop-2.6.0-src
   pushd hadoop-2.6.0-src
   attrib * -R /S
   icacls * /grant Everyone:(OI)(CI)F
   set log=..\mvn.log
   mvn package -Pdist,native-win -DskipTests -Dtar -e -X >> %log% 2>&1
   popd
   

编译结果：

• C:\hadoop-2.6.0-src\hadoop-dist\target/hadoop-2.6.0.tar.gz
• C:\hadoop-2.6.0-src\hadoop-dist\target\hadoop-dist-2.6.0-javadoc.jar

解决故障

缺ammintrin.h文件

c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\intrin.h(26): fatal error C1083: Cannot open include file: 'ammintrin.h': No such file or directory [C:\hadoop-2.6.0-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]

解决方法： 将此ammintrin.m下载改名为ammintrin.h，置于C:\ProgramFiles (x86)\Microsoft Visual Studio 10.0\VC\INCLUDE\目录下。

参考

Hadoop 2.6.0 Installation Guide (Windows)
Yet Another Hadoop Build Tutorial for Windows
Build instructions for Hadoop
Fix problem when mex .cpp file