MongoDB4.0.2集群搭建

时间 2019-11-06

标签 mongodb4.0.2 mongodb 集群搭建栏目 MongoDB 繁體版

原文原文链接

MongoDB4.0.2集群搭建

2018年09月30日 10:03:08 TOTORO_zip 阅读数 916更多linux

分类专栏： MangoDBnginx

https://www.jianshu.com/p/e1855a1d38b7mongodb

MongoDB4.0.2集群搭建

根据对象存储平台Django+MongoDB+Ceph的需求，现搭建部署一个十节点的MongoDB集群，主要如下关键点：数据库

根据最新版本MongoDB推荐，配置文件采用yaml方式来配置
一共10台服务器，即10个节点。对数据集进行分片，共分10个shard
每个shard都进行副本集配置，因为硬件磁盘已进行raid多副本备份，考虑到存储效率，本集群在副本集只须要一个备服务器，故采用1主+1备
+1仲裁（必须有仲裁节点，否则被服务器没法经过心跳机制升级为主服务器）的副本及配置方式

环境准备

系统：CentOS7.0 64bitvim

十台服务器：mongo00-mongo09服务器

服务器规划：app

mongo00	mongo01	mongo02	mongo03	mongo04	mongo05	mongo06	mongo07	mongo08	mongo09
mongos	mongos	mongos	mongos	mongos	mongos	mongos	config	config	config
shard0主	shard1主	shard2主	shard3主	shard4主	shard5主	shard6主	shard7主	shard8主	shard9主
shard9副	shard0副	shard1副	shard2副	shard3副	shard4副	shard5副	shard6副	shard7副	shard8副
shard8仲	shard9仲	shard0仲	shard1仲	shard2仲	shard3仲	shard4仲	shard5仲	shard6仲	shard7仲

端口分配：运维

mongos:20000linux运维
config:21000spa
shard0:27000
shard1:27001
shard2:27002
shard3:27003
shard4:27004
shard5:27005
shard6:27006
shard7:27007
shard8:27008
shard9:27009

MongoDB安装

下载、安装MongoDB

#官网下载
wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-4.0.2.tgz
#解压
tar -xzvf mongodb-linux-x86_64-rhel70-4.0.2.tgz -C /usr/local/
cd /use/local
mv mongodb-linux-x86_64-rhel70-4.0.2 mongodb

#mongo服务经过配置文件启动,存放配置文件目录/usr/local/mongodb/conf
#存放日志、进程管理信息的目录/var/log/nginx/mongodb
#根据服务器规划，在每台服务器建立该节点所含shard的数据存放目录/mnt/mongodata/shard0-9
#同时在mongo07-mongo09三台服务器建立存放config server数据的数据目录/mnt/mongodata/config
#mongo00:
mkdir -p /usr/local/mongodb/conf
mkdir -p /var/log/nginx/mongodb
mkdir -p /mnt/mongodata/shard0
mkdir -p /mnt/mongodata/shard9
mkdir -p /mnt/mongodata/shard8
#mongo01:
mkdir -p /usr/local/mongodb/conf
mkdir -p /var/log/nginx/mongodb
mkdir -p /mnt/mongodata/shard0
mkdir -p /mnt/mongodata/shard1
mkdir -p /mnt/mongodata/shard9
#mongo02:
mkdir -p /usr/local/mongodb/conf
mkdir -p /var/log/nginx/mongodb
mkdir -p /mnt/mongodata/shard0
mkdir -p /mnt/mongodata/shard1
mkdir -p /mnt/mongodata/shard2
#mongo03:
mkdir -p /usr/local/mongodb/conf
mkdir -p /var/log/nginx/mongodb
mkdir -p /mnt/mongodata/shard1
mkdir -p /mnt/mongodata/shard2
mkdir -p /mnt/mongodata/shard3
#mongo04:
mkdir -p /usr/local/mongodb/conf
mkdir -p /var/log/nginx/mongodb
mkdir -p /mnt/mongodata/shard2
mkdir -p /mnt/mongodata/shard3
mkdir -p /mnt/mongodata/shard4
#mongo05:
mkdir -p /usr/local/mongodb/conf
mkdir -p /var/log/nginx/mongodb
mkdir -p /mnt/mongodata/shard3
mkdir -p /mnt/mongodata/shard4
mkdir -p /mnt/mongodata/shard5
#mongo06:
mkdir -p /usr/local/mongodb/conf
mkdir -p /var/log/nginx/mongodb
mkdir -p /mnt/mongodata/shard4
mkdir -p /mnt/mongodata/shard5
mkdir -p /mnt/mongodata/shard6
#mongo07:
mkdir -p /usr/local/mongodb/conf
mkdir -p /var/log/nginx/mongodb
mkdir -p /mnt/mongodata/shard5
mkdir -p /mnt/mongodata/shard6
mkdir -p /mnt/mongodata/shard7
mkdir -p /mnt/mongodata/config
#mongo08:
mkdir -p /usr/local/mongodb/conf
mkdir -p /var/log/nginx/mongodb
mkdir -p /mnt/mongodata/shard6
mkdir -p /mnt/mongodata/shard7
mkdir -p /mnt/mongodata/shard8
mkdir -p /mnt/mongodata/config
#mongo09:
mkdir -p /usr/local/mongodb/conf
mkdir -p /var/log/nginx/mongodb
mkdir -p /mnt/mongodata/shard7
mkdir -p /mnt/mongodata/shard8
mkdir -p /mnt/mongodata/shard9
mkdir -p /mnt/mongodata/config

环境变量配置

vim /etc/profile
#加入如下内容
export MONGODB_HOME=/usr/local/mongodb
export PATH=$MONGODB_HOME/bin:$PATH
#当即生效
source /etc/profile

使用命令mongod -v输出信息版本信息验证环境变量是否配置成功

集群配置

一、config server配置服务器（副本集）

根据服务器规划，在mongo07-mongo09上部署三台config server副本集，在该三台服务器上分别添加如下配置文件：

vim /usr/local/mongodb/conf/config.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/config.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/config
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/configsrv.pid
# network interfaces
net:
port: 21000
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: configs
sharding:
clusterRole: configsvr

启动这三台服务器的config server

mongod -f /usr/local/mongodb/conf/config.conf

登录任意一台服务器，初始化副本集

mongo --port 21000
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致）
config = {
... _id : "configs",
... members : [
... {_id : 0, host : "mongo07:21000" },
... {_id : 1, host : "mongo08:21000" },
... {_id : 2, host : "mongo09:21000" }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

二、shard server分片服务器（副本集）

配置shard0副本集

在mongo00、mongo0一、mongo02服务器上作如下配置：

vim /usr/local/mongodb/conf/shard0.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/shard0.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/shard0
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/shard0.pid
# network interfaces
net:
port: 27000
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: shard0
sharding:
clusterRole: shardsvr

启动这三台服务器的shard0 server

mongod -f /usr/local/mongodb/conf/shard0.conf

登录任意一台服务器，初始化副本集

mongo --port 27000
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致,priority表明权重[1,100]，大的被分配为主服务器，0永久不会变为主服务器）
config = {
... _id : "shard0",
... members : [
... {_id : 0, host : "mongo00:27000", priority : 2 },
... {_id : 1, host : "mongo01:27000", priority : 1 },
... {_id : 2, host : "mongo02:27000", arbiterOnly :true }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

配置shard1副本集

在mongo0一、mongo0二、mongo03服务器上作如下配置：

vim /usr/local/mongodb/conf/shard1.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/shard1.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/shard1
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/shard1.pid
# network interfaces
net:
port: 27001
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: shard1
sharding:
clusterRole: shardsvr

启动这三台服务器的shard1 server

mongod -f /usr/local/mongodb/conf/shard1.conf

登录任意一台服务器，初始化副本集

mongo --port 27001
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致,priority表明权重[1,100]，大的被分配为主服务器，0永久不会变为主服务器）
config = {
... _id : "shard1",
... members : [
... {_id : 0, host : "mongo01:27001", priority : 2 },
... {_id : 1, host : "mongo02:27001", priority : 1 },
... {_id : 2, host : "mongo03:27001", arbiterOnly :true }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

配置shard2副本集

在mongo0二、mongo0三、mongo04服务器上作如下配置：

vim /usr/local/mongodb/conf/shard2.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/shard2.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/shard2
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/shard2.pid
# network interfaces
net:
port: 27002
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: shard2
sharding:
clusterRole: shardsvr

启动这三台服务器的shard2 server

mongod -f /usr/local/mongodb/conf/shard2.conf

登录任意一台服务器，初始化副本集

mongo --port 27002
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致,priority表明权重[1,100]，大的被分配为主服务器，0永久不会变为主服务器）
config = {
... _id : "shard2",
... members : [
... {_id : 0, host : "mongo02:27002", priority : 2 },
... {_id : 1, host : "mongo03:27002", priority : 1 },
... {_id : 2, host : "mongo04:27002", arbiterOnly :true }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

配置shard3副本集

在mongo0三、mongo0四、mongo05服务器上作如下配置：

vim /usr/local/mongodb/conf/shard3.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/shard3.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/shard3
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/shard3.pid
# network interfaces
net:
port: 27003
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: shard3
sharding:
clusterRole: shardsvr

启动这三台服务器的shard3 server

mongod -f /usr/local/mongodb/conf/shard3.conf

登录任意一台服务器，初始化副本集

mongo --port 27003
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致,priority表明权重[1,100]，大的被分配为主服务器，0永久不会变为主服务器）
config = {
... _id : "shard3",
... members : [
... {_id : 0, host : "mongo03:27003", priority : 2 },
... {_id : 1, host : "mongo04:27003", priority : 1 },
... {_id : 2, host : "mongo05:27003", arbiterOnly :true }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

配置shard4副本集

在mongo0四、mongo0五、mongo06服务器上作如下配置：

vim /usr/local/mongodb/conf/shard4.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/shard4.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/shard4
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/shard4.pid
# network interfaces
net:
port: 27004
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: shard4
sharding:
clusterRole: shardsvr

启动这三台服务器的shard4 server

mongod -f /usr/local/mongodb/conf/shard4.conf

登录任意一台服务器，初始化副本集

mongo --port 27004
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致,priority表明权重[1,100]，大的被分配为主服务器，0永久不会变为主服务器）
config = {
... _id : "shard4",
... members : [
... {_id : 0, host : "mongo04:27004", priority : 2 },
... {_id : 1, host : "mongo05:27004", priority : 1 },
... {_id : 2, host : "mongo06:27004", arbiterOnly :true }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

配置shard5副本集

在mongo0五、mongo0六、mongo07服务器上作如下配置：

vim /usr/local/mongodb/conf/shard5.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/shard5.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/shard5
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/shard5.pid
# network interfaces
net:
port: 27005
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: shard5
sharding:
clusterRole: shardsvr

启动这三台服务器的shard5 server

mongod -f /usr/local/mongodb/conf/shard5.conf

登录任意一台服务器，初始化副本集

mongo --port 27005
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致,priority表明权重[1,100]，大的被分配为主服务器，0永久不会变为主服务器）
config = {
... _id : "shard5",
... members : [
... {_id : 0, host : "mongo05:27005", priority : 2 },
... {_id : 1, host : "mongo06:27005", priority : 1 },
... {_id : 2, host : "mongo07:27005", arbiterOnly :true }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

配置shard6副本集

在mongo0六、mongo0七、mongo08服务器上作如下配置：

vim /usr/local/mongodb/conf/shard6.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/shard6.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/shard6
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/shard6.pid
# network interfaces
net:
port: 27006
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: shard6
sharding:
clusterRole: shardsvr

启动这三台服务器的shard6 server

mongod -f /usr/local/mongodb/conf/shard6.conf

登录任意一台服务器，初始化副本集

mongo --port 27006
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致,priority表明权重[1,100]，大的被分配为主服务器，0永久不会变为主服务器）
config = {
... _id : "shard6",
... members : [
... {_id : 0, host : "mongo06:27006", priority : 2 },
... {_id : 1, host : "mongo07:27006", priority : 1 },
... {_id : 2, host : "mongo08:27006", arbiterOnly :true }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

配置shard7副本集

在mongo0七、mongo0八、mongo09服务器上作如下配置：

vim /usr/local/mongodb/conf/shard7.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/shard7.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/shard7
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/shard7.pid
# network interfaces
net:
port: 27007
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: shard7
sharding:
clusterRole: shardsvr

启动这三台服务器的shard7 server

mongod -f /usr/local/mongodb/conf/shard7.conf

登录任意一台服务器，初始化副本集

mongo --port 27007
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致,priority表明权重[1,100]，大的被分配为主服务器，0永久不会变为主服务器）
config = {
... _id : "shard7",
... members : [
... {_id : 0, host : "mongo07:27007", priority : 2 },
... {_id : 1, host : "mongo08:27007", priority : 1 },
... {_id : 2, host : "mongo09:27007", arbiterOnly :true }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

配置shard8副本集

在mongo0八、mongo0九、mongo00服务器上作如下配置：

vim /usr/local/mongodb/conf/shard8.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/shard8.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/shard8
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/shard8.pid
# network interfaces
net:
port: 27008
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: shard8
sharding:
clusterRole: shardsvr

启动这三台服务器的shard8 server

mongod -f /usr/local/mongodb/conf/shard8.conf

登录任意一台服务器，初始化副本集

mongo --port 27008
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致,priority表明权重[1,100]，大的被分配为主服务器，0永久不会变为主服务器）
config = {
... _id : "shard8",
... members : [
... {_id : 0, host : "mongo08:27008", priority : 2 },
... {_id : 1, host : "mongo09:27008", priority : 1 },
... {_id : 2, host : "mongo00:27008", arbiterOnly :true }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

配置shard9副本集

在mongo0九、mongo00、mongo01服务器上作如下配置：

vim /usr/local/mongodb/conf/shard9.conf
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodb/shard9.log
# Where and how to store data.
storage:
dbPath: /mnt/mongodata/shard9
journal:
enabled: true
# how the process runs
processManagement:
fork: true
pidFilePath: /var/log/nginx/mongodb/shard9.pid
# network interfaces
net:
port: 27009
bindIp: 0.0.0.0
#operationProfiling:
replication:
replSetName: shard9
sharding:
clusterRole: shardsvr

启动这三台服务器的shard9 server

mongod -f /usr/local/mongodb/conf/shard9.conf

登录任意一台服务器，初始化副本集

mongo --port 27009
#定义副本集配置（键“_id”对应的值必须与配置文件中的replicaction.replSetName一致,priority表明权重[1,100]，大的被分配为主服务器，0永久不会变为主服务器）
config = {
... _id : "shard9",
... members : [
... {_id : 0, host : "mongo09:27009", priority : 2 },
... {_id : 1, host : "mongo00:27009", priority : 1 },
... {_id : 2, host : "mongo01:27009", arbiterOnly :true }
... ]
... }
#初始化副本集
rs.initiate(config)
#查看分区状态
rs.status();

三、mongos server路由服务器（在mongo00-mongo06上配置）

注意：启动mongodb时，先启动配置服务器和分片服务器，最后启动路由服务器

vim /usr/local/mongodb/conf/mongos.conf
systemLog:
destination: file
logAppend: true
path: /var/log/nginx/mongodbmongos.log
processManagement:
fork: true
# pidFilePath: /var/log/nginx/mongodbmongos.pid
# network interfaces
net:
port: 20000
bindIp: 0.0.0.0
#监听的配置服务器,只能有1个或者3个 configs为配置服务器的副本集名字
sharding:
configDB: configs/mongo07:21000,mongo08:21000,mongo09:21000

启动这七台服务器的mongos server

mongod -f /usr/local/mongodb/conf/mongos.conf

四、启用分片

目前已经搭建好配置服务器、数据分片服务器、路由服务器，下面进行分片启用，使得app链接到路由服务器时可使用分片机制

登陆任意一台mongos

mongo --port 20000
#使用admin数据库
use admin
#串联路由服务器与分片副本集
sh.addShard("shard0/mongo00:27000,mongo01:27000,mongo02:27000")
sh.addShard("shard1/mongo01:27001,mongo02:27001,mongo03:27001")
sh.addShard("shard2/mongo02:27002,mongo03:27002,mongo04:27002")
sh.addShard("shard3/mongo03:27003,mongo04:27003,mongo05:27003")
sh.addShard("shard4/mongo04:27004,mongo05:27004,mongo06:27004")
sh.addShard("shard5/mongo05:27005,mongo06:27005,mongo07:27005")
sh.addShard("shard6/mongo06:27006,mongo07:27006,mongo08:27006")
sh.addShard("shard7/mongo07:27007,mongo08:27007,mongo09:27007")
sh.addShard("shard8/mongo08:27008,mongo09:27008,mongo00:27008")
sh.addShard("shard9/mongo09:27009,mongo00:27009,mongo01:27009")
#查看集群状态
sh.status()

培训四个月的linux运维,能达到什么样的水平?