Hadoop HA

时间  2019-11-09
标签 hadoop 栏目 Hadoop 繁體版
原文   原文链接

Hadoop HA

什么是 HA

HA是High Available缩写，是双机集群系统简称，指高可用性集群，是保证业务连续性的有效解决方案，通常有两个或两个以上的节点，且分为活动节点及备用节点。一般把正在执行业务的称为活动节点，而做为活动节点的一个备份的则称为备用节点。当活动节点出现问题，致使正在运行的业务（任务）不能正常运行时，备用节点此时就会侦测到，并当即接续活动节点来执行业务。从而实现业务的不中断或短暂中断。java

hadoop HA机制介绍

hadoop2.0的HA 机制有两个namenode，一个是active namenode，状态是active；另一个是standby namenode，状态是standby。二者的状态是能够切换的，但不能同时两个都是active状态，最多只有1个是active状态。只有active namenode提供对外的服务，standby namenode是不对外服务的。active namenode和standby namenode之间经过NFS或者JN（journalnode，QJM方式）来同步数据。 active namenode会把最近的操做记录写到本地的一个edits文件中（edits file），并传输到NFS或者JN中。standby namenode按期的检查，从NFS或者JN把最近的edit文件读过来，而后把edits文件和fsimage文件合并成一个新的fsimage，合并完成以后会通知active namenode获取这个新fsimage。active namenode得到这个新的fsimage文件以后，替换原来旧的fsimage文件。 这样，保持了active namenode和standby namenode的数据的实时同步，standby namenode能够随时切换成active namenode（譬如active namenode挂了）。并且还有一个原来hadoop1.0的secondarynamenode，checkpointnode，buckcupnode的功能：合并edits文件和fsimage文件，使fsimage文件一直保持更新。因此启动了hadoop2.0的HA机制以后，secondarynamenode，checkpointnode，buckcupnode这些都不须要了。node

搭建 hadoop HA 集群

环境

linux: CentOS-7.5_x64 hadoop: hadoop-3.2.0 zookeeper: zookeeper-3.4.10linux

机器规划

主机名 IP 安装软件 运行进程
node-1 192.168.91.11 hadoop NameNode,ResourceManager,DFSZKFailoverController
node-2 192.168.91.12 hadoop,zookeeper NameNode,ResourceManager,QuorumPeerMain,DFSZKFailoverController
node-3 192.168.91.13 hadoop,zookeeper QuorumPeerMain,DataNode,NodeManager,JournalNode
node-4 192.168.91.14 hadoop,zookeeper QuorumPeerMain,DataNode,NodeManager,JournalNode

前置准备

四台机器须要ssh免密登陆，node-2,node-3,node-4须要安装zookeeper、java环境shell

集群搭建

# 下载
$ wget http://mirrors.shu.edu.cn/apache/hadoop/common/hadoop-3.1.2/hadoop-3.2.0.tar.gz

# 解压
$ tar -zxvf hadoop-3.2.0.tar.gz

# 配置系统的环境变量
$ vim /etc/profile

export JAVA_HOME=/usr/local/jdk1.8.0_191
export PATH=$PATH:$JAVA_HOME/bin

export HADOOP_HOME=/export/servers/hadoop-3.2.0
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin


# 进入解压目录配置环境变量
$ cd $HADOOP_HOME

# 配置hadoop-env.sh 添加下面配置（不配置启动会报错）
$ vim etc/hadoop/core-site.xml

export JAVA_HOME=/usr/local/jdk1.8.0_191

export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_JOURNALNODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root

export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root



# 配置core-site.xml
$ vim etc/hadoop/core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

    <!-- HA 配置指定 hdfs 的 nameService 为ns -->
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://ns</value>
    </property>

    <!-- HA 配置，指定zookeeper地址 -->
    <property>
        <name>ha.zookeeper.quorum</name>
        <value>node-2:2181,node-3:2181,node-4:2181</value>
    </property>

    <property>   
        <name>hadoop.tmp.dir</name>   
        <value>/export/data/hadoop/temp</value>
    </property>

    <property>
        <name>hadoop.proxyuser.root.hosts</name>
        <value>*</value>
    </property>

    <property>
        <name>hadoop.proxyuser.root.groups</name>
        <value>*</value>
    </property>

</configuration>



# 配置hdfs-site.xml
$ vim etc/hadoop/hdfs-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

    <!--指定hdfs的nameservice为ns，须要和core-site.xml中的保持一致 -->
    <property>
        <name>dfs.nameservices</name>
        <value>ns</value>
    </property>

    <!-- bi下面有两个NameNode，分别是nn1，nn2 -->
    <property>
        <name>dfs.ha.namenodes.ns</name>
        <value>nn1,nn2</value>
    </property>

    <!-- nn1的RPC通讯地址 -->
    <property>
        <name>dfs.namenode.rpc-address.ns.nn1</name>
        <value>node-1:9000</value>
    </property>

    <!-- nn1的http通讯地址 -->
    <property>
        <name>dfs.namenode.http-address.ns.nn1</name>
        <value>node-1:50070</value>
    </property>

    <!-- nn2的RPC通讯地址 -->
    <property>
        <name>dfs.namenode.rpc-address.ns.nn2</name>
        <value>node-2:9000</value>
    </property>

    <!-- nn2的http通讯地址 -->
    <property>
        <name>dfs.namenode.http-address.ns.nn2</name>
        <value>node-2:50070</value>
    </property>

    <!-- 指定NameNode的edits元数据在JournalNode上的存放位置 -->
    <property>
        <name>dfs.namenode.shared.edits.dir</name>
        <value>qjournal://node-3:8485;node-4:8485/ns</value>
    </property>

    <!-- 指定JournalNode在本地磁盘存放数据的位置 -->
    <property>
        <name>dfs.journalnode.edits.dir</name>
        <value>/export/data/hadoop/journaldata</value>
    </property>

    <!-- 开启NameNode失败自动切换 -->
    <property>
        <name>dfs.ha.automatic-failover.enabled</name>
        <value>true</value>
    </property>

    <!-- 配置失败自动切换实现方式 -->
    <property>
        <name>dfs.client.failover.proxy.provider.bi</name>
        <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
    </property>

    <!-- 配置隔离机制方法，多个机制用换行分割，即每一个机制暂用一行-->
    <property>
        <name>dfs.ha.fencing.methods</name>
        <value>
            sshfence
            shell(/bin/true)
        </value>
    </property>

    <!-- 使用sshfence隔离机制时须要ssh免登录 -->
    <property>
        <name>dfs.ha.fencing.ssh.private-key-files</name>
        <value>/root/.ssh/id_rsa</value>
    </property>

    <!-- 配置sshfence隔离机制超时时间 -->
    <property>
        <name>dfs.ha.fencing.ssh.connect-timeout</name>
        <value>30000</value>
    </property>

    <property>
        <name>dfs.ha.namenodes.jn</name>
        <value>node-3,node-4</value>
    </property>

</configuration>

# 配置mapred-site.xml
$ vim etc/hadoop/mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>

    <property>
        <name>yarn.app.mapreduce.am.env</name>
        <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
    </property>

    <property>
        <name>mapreduce.map.env</name>
        <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
    </property>

    <property>
        <name>mapreduce.reduce.env</name>
        <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
    </property>

</configuration>


# 配置yarn-site.xml
$ vim etc/hadoop/yarn-site.xml

<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<!-- Site specific YARN configuration properties -->

    <!-- 开启RM高可用 -->
    <property>
        <name>yarn.resourcemanager.ha.enabled</name>
        <value>true</value>
    </property>

    <!-- 指定RM的cluster id -->
    <property>
        <name>yarn.resourcemanager.cluster-id</name>
        <value>yarn-ha</value>
    </property>

    <!-- 指定RM的名字 -->
    <property>
        <name>yarn.resourcemanager.ha.rm-ids</name>
        <value>rm1,rm2</value>
    </property>

    <!-- 分别指定RM的地址 -->
    <property>
        <name>yarn.resourcemanager.hostname.rm1</name>
        <value>node-1</value>
    </property>

    <property>
        <name>yarn.resourcemanager.hostname.rm2</name>
        <value>node-2</value>
    </property>

    <!-- 指定zk集群地址 -->
    <property>
        <name>yarn.resourcemanager.zk-address</name>
        <value>node-2:2181,node-3:2181,node-4:2181</value>
    </property>

    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>

</configuration>


# 配置workers节点
$ vim $HADOOP_HOME/etc/hadoop/workers

node-3
node-4

# 拷贝hadoop到其余节点（node-2,node-3,node-4）
$ scp -r hadoop-3.2.0 root@node-2:/xxx/xxx/


复制代码

hdfs HA 配置

# 启动zookeeper集群
$ $ZOOKEEPER_HOME/bin/zkServer.sh start

# 查看zookeeper状态
$ $ZOOKEEPER_HOME/bin/zkServer.sh status

# 启动 JournalNode 集群 分别在 node-三、node-4 上执行如下命令
$ hdfs --daemon start journalnode

# 格式化 ZooKeeper 集群
$ hdfs zkfc -formatZK

# 格式化集群的 NameNode (在 node-1 上执行)
$ hdfs namenode -format

# 启动刚格式化的 NameNode (在 node-1 上执行)
$ hdfs --daemon start namenode

# 同步 NameNode1 元数据到 NameNode2 上 (在 node-2 上执行) 
$ hdfs namenode -bootstrapStandby

# 启动 NameNode2 (在 node-2 上执行)
$ hdfs --daemon start namenode

# 启动集群中全部的DataNode (在 node-1 上执行) 
$ sbin/start-dfs.sh

# 启动 ZKFC 进程 (在 node-1 和 node-2 的主机上分别执行以下命令)
$ hdfs --daemon start zkfc

# 验证ha(在node-1节点停掉namenode进程)
$ hafs --daemon stop namenode


复制代码

resourceManager HA 配置

# 在 RM1 启动 YARN (在 node-1 上执行)
$ yarn --daemon start resourcemanager

# 在 RM2 启动 YARN (在 node-2 上执行)
$ yarn --daemon start resourcemanager

# 在任意节点执行获取resourceManager状态（active）
$ yarn rmadmin -getServiceState rm1

# 在任意节点执行获取resourceManager状态（standby）
$ yarn rmadmin -getServiceState rm2

# 验证 yarn 的 ha（在node-1节点执行）standby 的 resourcemanager 则会转换为 active
$ yarn --daemon stop resourcemanager

# 在任意节点执行获取resourceManager状态（active）
$ yarn rmadmin -getServiceState rm2

复制代码

总结

搭建hadoop HA 过程当中遇到了不少各类各样的问题上述步骤都是通过验证的如在安装过程当中遇到问题能够留言，谢谢！express

相关文章
相关标签/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公众号
   欢迎关注本站公众号,获取更多信息
联系我们 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程