hadoop 的安装

1. 必要条件 要有java，检测javajava -version, 没有的要自行安装

2. brew 安装hadoop brew install hadoop

3. 配置环境变量 vim ~/.bash_profile

   添加以下内容：

   export HADOOP_HOME=/usr/local/Cellar/hadoop/3.1.1/libexec
   export HADOOP_HOME_WARN_SUPPRESS=1
   

   使环境变量生效：

   source ~/.bash_profile
   

4. Go to /usr/local/Cellar/hadoop/2.8.0/libexec/etc/hadoop, then open hadoop-env.sh 添加如下配置： ``` export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true -Djava.security.krb5.realm= -Djava.security.krb5.kdc="

export JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk-10.0.1.jdk/Contents/Home"

修改```core-site.xml```文件

hadoop.tmp.dir /usr/local/Cellar/hadoop/hdfs/tmp A base for other temporary directories. fs.default.name hdfs://localhost:8020

修改```mapred-site.xml

<configuration>
    <property>
        <name>mapred.job.tracker</name>
        <value>localhost:8021</value>
    </property>
</configuration>

修改hdfs-site.xml文件

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

Before running background program, we should format the installed HDFS first, executing command hdfs namenode -format, when terminal returns a long inforamtion like:

17/07/02 16:11:05 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
......
17/07/02 16:11:07 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at haodemacbook-pro.local/192.168.1.4
************************************************************/

表示HDFS已经安装成功。

输入hadoop可查看是否安装上

1. Launch Hadoop 先登陆上 ssh localhost Go to /usr/local/Cellar/hadoop/3.1.1sbin, execute: ``` ./start-dfs.sh # start HDFS service

./stop-dfs.sh # stop HDFS service

Ternimal will return the following information:

Starting namenodes on [localhost] localhost: starting namenode, logging to /usr/local/Cellar/hadoop/2.8.0/libexec/logs/hadoop-zhanghao-namenode-HaodeMacBook-Pro.local.out localhost: starting datanode, logging to /usr/local/Cellar/hadoop/2.8.0/libexec/logs/hadoop-zhanghao-datanode-HaodeMacBook-Pro.local.out Starting secondary namenodes [0.0.0.0] ``` It means the local service launched successfully, then open Resource Manager in browser through the link http://localhost:50070, you can see the following page.