Mongodb分布式集群副本集+分片

时间 2020-12-23

标签 node linux mongodb 数据库 vim 服务器 session app 异步分布式栏目 MongoDB 繁體版

原文原文链接

简介

1. 副本集

开启复制集后，主节点会在 local 库下生成一个集合叫 oplog.rs，这是一个有限集合，也就是大小是固定的。其中记录的是整个mongod实例一段时间内数据库的全部变动（插入/更新/删除）操做，当空间用完时新记录自动覆盖最老的记录node

MongoDB复制集(副本集):由一组实列(进程)组成；包含一个Primary节点和多个Secondary节点，用户的全部写操做写入Primary ，Secondary经过oplog来同步primary的数据;能够经过心跳检测机制,一旦primary出现故障,则就会经过仲裁节点从secondary选取一个新的主节点linux

Primary:主节点，由选择产生,负责客户端的写操做,产生oplog日志文件mongodb

Secondary:从节点；负责客户端的读操做;数据库

Arbiter:仲裁节点;只参与选举的投票;不会成为Primary和secondary,任意节点宕机，复制集将不能提供服务了（没法选出Primary），这时能够给复制集添加一个Arbiter节点，即便有节点宕机，仍能选出Primary**vim

1.1 MongoDB选举的原理

MongoDB的节点分为三种类型，分别为标准节点（host）、被动节点（passive）和仲裁节点（arbiter）服务器

只有标准节点才有可能被选举为活跃节点（主节点），拥有选举权。被动节点有完整副本，不可能成为活跃节点，具备选举权。仲裁节点不复制数据，不可能成为活跃节点，只有选举权。说白了就是只有标准节点才有可能被选举为主节点，即便在一个复制集中说有的标准节点都宕机，被动节点和仲裁节点也不会成为主节点session

标准节点与被动节点的区别：priority值高者是标准节点，低者则为被动节点app

选举规则是票数高的获胜，priority是优先权0_{1000的值，至关于额外增长0}1000的票数。选举结果：票数高者获胜；若票数相同，数据新者获胜。异步

1.2 复制过程

客户端的数据进来；分布式
数据操做写入到日志缓冲；
数据写入到数据缓冲；
把日志缓冲中的操做日志放到OPLOG中来；
返回操做结果到客户端（异步）；
后台线程进行OPLOG复制到从节点，这个频率是很是高的，比日志刷盘频率还要高，从节点会一直监听主节点，OPLOG一有变化就会进行复制操做；
后台线程进行日志缓冲中的数据刷盘，很是频繁（默认100）毫秒，也可自行设置（30-60）；
后台线程进行数据缓冲中的数据刷盘，默认是60秒；

2. 分片技术

复制集主要用来实现自动故障转移从而达到高可用的目的，然而，随着业务规模的增加和时间的推移，业务数据量会愈来愈大，当前业务数据可能只有几百GB不到，一台DB服务器足以搞定全部的工做，而一旦业务数据量扩充大几个TB几百个TB时，就会产生一台服务器没法存储的状况，此时，须要将数据按照必定的规则分配到不一样的服务器进行存储、查询等，即为分片集群。分片集群要作到的事情就是数据分布式存储

存储方式：数据集被拆分红数据块（chunk），每一个数据块包含多个doc，数据块分布式存储在分片集群中。

2.1 角色

Config server：MongoDB负责追踪数据块在shard上的分布信息，每一个分片存储哪些数据块，叫作分片的元数据，保存在config server上的数据库 config中，通常使用3台config
server，全部config server中的config数据库必须彻底相同（建议将config server部署在不一样的服务器，以保证稳定性）；

Shard server：将数据进行分片，拆分红数据块（chunk），每一个trunk块的大小默认为64M，数据块真正存放的单位；

Mongos server：数据库集群请求的入口，全部的请求都经过mongos进行协调，查看分片的元数据，查找chunk存放位置，mongos本身就是一个请求分发中心，在生产环境一般有多mongos做为请求的入口，防止其中一个挂掉全部的mongodb请求都没有办法操做。

总结：应用请求mongos来操做mongodb的增删改查，配置服务器存储数据库元信息，而且和mongos作同步，数据最终存入在shard（分片）上，为了防止数据丢失，同步在副本集中存储了一份，仲裁节点在数据存储到分片的时候决定存储到哪一个节点。

2.2 分片的片键

概述：片键是文档的一个属性字段或是一个复合索引字段，一旦创建后则不可改变，片键是拆分数据的关键的依据，如若在数据极为庞大的场景下，片键决定了数据在分片的过程当中数据的存储位置，直接会影响集群的性能；

注：建立片键时，须要有一个支撑片键运行的索引；

2.3 片键分类

1.递增片键：使用时间戳，日期，自增的主键，ObjectId，_id等，此类片键的写入操做集中在一个分片服务器上，写入不具备分散性，这会致使单台服务器压力较大，但分割比较容易，这台服务器可能会成为性能瓶颈；

2.哈希片键：也称之为散列索引，使用一个哈希索引字段做为片键，优势是使数据在各节点分布比较均匀，数据写入可随机分发到每一个分片服务器上，把写入的压力分散到了各个服务器上。可是读也是随机的，可能会命中更多的分片，可是缺点是没法实现范围区分；

3.组合片键：数据库中没有比较合适的键值供片键选择，或者是打算使用的片键基数过小（即变化少如星期只有7天可变化），能够选另外一个字段使用组合片键，甚至能够添加冗余字段来组合；

4.标签片键：数据存储在指定的分片服务器上，能够为分片添加tag标签，而后指定相应的tag，好比让10...(T)出如今shard0000上，11...(Q)出如今shard0001或shard0002上，就可使用tag让均衡器指定分发；

环境介绍

分布式mongodb集群副本集+分片

CentOS Linux release 7.9.2009
Mongodb:4.0.21

IP	路由服务端口	配置服务端口	分片1端口	分片2端口	分片3端
172.16.245.102	27017	27018	27001	27002	27003
172.16.245.103	27017	27018	27001	27002	27003
172.16.245.104	27017	27018	27001	27002	27003

1.获取软件包

wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-4.0.21.tgz

2.建立路由、配置、分片等的相关目录与文件

三台服务器相同操做

mkdir -p /data/mongodb/conf
mkdir -p /data/mongodb/data/config
mkdir -p /data/mongodb/data/shard1
mkdir -p /data/mongodb/data/shard2
mkdir -p /data/mongodb/data/shard3
mkdir -p /data/mongodb/log/config.log
mkdir -p /data/mongodb/log/mongos.log
mkdir -p /data/mongodb/log/shard1.log
mkdir -p /data/mongodb/log/shard2.log
mkdir -p /data/mongodb/log/shard3.log
touch /data/mongodb/log/config.log/config.log
touch /data/mongodb/log/mongos.log/mongos.log
touch /data/mongodb/log/shard1.log/shard1.log
touch /data/mongodb/log/shard2.log/shard2.log
touch /data/mongodb/log/shard3.log/shard3.log

3. 配置服务器部署mongodb

3台服务器执行相同操做

[root@node5 conf]# vim /data/mongodb/conf/config.conf 
[root@node5 conf]# cat /data/mongodb/conf/config.conf 
dbpath=/data/mongodb/data/config
logpath=/data/mongodb/log/config.log/config.log
port=27018      #端口号
logappend=true
fork=true
maxConns=5000   
replSet=configs     #副本集名称
configsvr=true
bind_ip=0.0.0.0

4. 配置复本集

分别启动三台服务器的配置服务

[root@node5 conf]# /data/mongodb/bin/mongod -f /data/mongodb/conf/config.conf

链接mongo,只需在任意一台机器执行便可

[root@node5 conf]# /data/mongodb/bin/mongo --host 172.16.245.102 --port 27018

进入数据库之后切换数据库

use admin

初始化副本集

rs.initiate({_id:"configs",members:[{_id:0,host:"172.16.245.102:27018"},{_id:1,host:"172.16.245.103:27018"}, {_id:2,host:"172.16.245.104:27018"}]})

其中_id:"configs"的configs是上面config.conf配置文件里的复制集名称，把三台服务器的(指定相应的IP)配置服务组成复制集

查看状态

configs:PRIMARY> rs.status()
{
        "set" : "configs",                          #副本集名称
        "date" : ISODate("2020-12-22T06:39:04.184Z"),
        "myState" : 1,
        "term" : NumberLong(1),
        "syncingTo" : "",
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "configsvr" : true,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1608619142, 1),
                        "t" : NumberLong(1)
                },
                "readConcernMajorityOpTime" : {
                        "ts" : Timestamp(1608619142, 1),
                        "t" : NumberLong(1)
                },
                "appliedOpTime" : {
                        "ts" : Timestamp(1608619142, 1),
                        "t" : NumberLong(1)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1608619142, 1),
                        "t" : NumberLong(1)
                }
        },
        "lastStableCheckpointTimestamp" : Timestamp(1608619122, 1),
        "electionCandidateMetrics" : {
                "lastElectionReason" : "electionTimeout",
                "lastElectionDate" : ISODate("2020-12-22T05:31:42.975Z"),
                "electionTerm" : NumberLong(1),
                "lastCommittedOpTimeAtElection" : {
                        "ts" : Timestamp(0, 0),
                        "t" : NumberLong(-1)
                },
                "lastSeenOpTimeAtElection" : {
                        "ts" : Timestamp(1608615092, 1),
                        "t" : NumberLong(-1)
                },
                "numVotesNeeded" : 2,
                "priorityAtElection" : 1,
                "electionTimeoutMillis" : NumberLong(10000),
                "numCatchUpOps" : NumberLong(0),
                "newTermStartDate" : ISODate("2020-12-22T05:31:42.986Z"),
                "wMajorityWriteAvailabilityDate" : ISODate("2020-12-22T05:31:44.134Z")
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "172.16.245.102:27018",        #副本1
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 4383,
                        "optime" : {
                                "ts" : Timestamp(1608619142, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-12-22T06:39:02Z"),
                        "syncingTo" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1608615102, 1),
                        "electionDate" : ISODate("2020-12-22T05:31:42Z"),
                        "configVersion" : 1,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "172.16.245.103:27018",        #副本2
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 4052,
                        "optime" : {
                                "ts" : Timestamp(1608619142, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1608619142, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-12-22T06:39:02Z"),
                        "optimeDurableDate" : ISODate("2020-12-22T06:39:02Z"),
                        "lastHeartbeat" : ISODate("2020-12-22T06:39:02.935Z"),
                        "lastHeartbeatRecv" : ISODate("2020-12-22T06:39:03.044Z"),
                        "pingMs" : NumberLong(85),
                        "lastHeartbeatMessage" : "",
                        "syncingTo" : "172.16.245.102:27018",
                        "syncSourceHost" : "172.16.245.102:27018",
                        "syncSourceId" : 0,
                        "infoMessage" : "",
                        "configVersion" : 1
                },
                {
                        "_id" : 2,
                        "name" : "172.16.245.104:27018",        #副本3
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 4052,
                        "optime" : {
                                "ts" : Timestamp(1608619142, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1608619142, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-12-22T06:39:02Z"),
                        "optimeDurableDate" : ISODate("2020-12-22T06:39:02Z"),
                        "lastHeartbeat" : ISODate("2020-12-22T06:39:03.368Z"),
                        "lastHeartbeatRecv" : ISODate("2020-12-22T06:39:03.046Z"),
                        "pingMs" : NumberLong(85),
                        "lastHeartbeatMessage" : "",
                        "syncingTo" : "172.16.245.102:27018",
                        "syncSourceHost" : "172.16.245.102:27018",
                        "syncSourceId" : 0,
                        "infoMessage" : "",
                        "configVersion" : 1
                }
        ],
        "ok" : 1,
        "operationTime" : Timestamp(1608619142, 1),
        "$gleStats" : {
                "lastOpTime" : Timestamp(0, 0),
                "electionId" : ObjectId("7fffffff0000000000000001")
        },
        "lastCommittedOpTime" : Timestamp(1608619142, 1),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1608619142, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
configs:PRIMARY>

等几十秒左右，执行上面的命令查看状态，三台机器的配置服务就已造成复制集，其中1台为PRIMARY，其余2台为SECONDARY

5. 分片服务部署

3台服务器执行相同操做

在/data/mongodb/conf目录建立shard1.conf、shard2.conf、shard3.conf，内容以下

[root@node3 conf]# ls
config.conf  mongos.conf  shard1.conf  shard2.conf  shard3.conf
[root@node3 conf]# cat shard1.conf 
dbpath=/data/mongodb/data/shard1 
logpath=/data/mongodb/log/shard1.log/shard1.log 
port=27001
logappend=true
fork=true
maxConns=5000
storageEngine=mmapv1
shardsvr=true
replSet=shard1 
bind_ip=0.0.0.0
[root@node3 conf]# cat shard2.conf 
dbpath=/data/mongodb/data/shard2 
logpath=/data/mongodb/log/shard2.log/shard2.log
port=27002
logappend=true
fork=true
maxConns=5000
storageEngine=mmapv1
shardsvr=true
replSet=shard2 
bind_ip=0.0.0.0
[root@node3 conf]# cat shard3.conf 
dbpath=/data/mongodb/data/shard3 
logpath=/data/mongodb/log/shard3.log/shard3.log 
port=27003 
logappend=true
fork=true
maxConns=5000
storageEngine=mmapv1
shardsvr=true
replSet=shard3 
bind_ip=0.0.0.0

端口分别是2700一、2700二、27003，分别对应shard1.conf、shard2.conf、shard3.conf

在3台机器的相同端口造成一个分片的复制集，因为3台机器都须要这3个文件，因此根据这9个配置文件分别启动分片服务

三台机器都须要启动分片服务，节点1启动shard1 节点2启动shard1 节点2启动shard1 ....

[root@node3 conf]# /data/mongodb/bin/mongond -f /data/mongodb/conf/shard1.conf
[root@node3 conf]# /data/mongodb/bin/mongond -f /data/mongodb/conf/shard2.conf
[root@node3 conf]# /data/mongodb/bin/mongond -f /data/mongodb/conf/shard3.conf

6. 将分片配置为复制集

链接mongo，只需在任意一台机器执行便可

mongo --host 172.16.245.103  --port 27001 
 
 这里以shard1为例，其余两个分片则再需对应链接到2700二、27003的端口进行操做便可

进入数据库admin

use admin

初始化三个分片副本集集群

rs.initiate({_id:"shard1",members:[{_id:0,host:"172.16.245.102:27001"},{_id:1,host:"172.16.245.103:27001"},{_id:2,host:"172.16.245.104:27001"}]})

rs.initiate({_id:"shard2",members:[{_id:0,host:"172.16.245.102:27002"},{_id:1,host:"172.16.245.103:27002"},{_id:2,host:"172.16.245.104:27002"}]})

rs.initiate({_id:"shard3",members:[{_id:0,host:"172.16.245.102:27003"},{_id:1,host:"172.16.245.103:27003"},{_id:2,host:"172.16.245.104:27003"}]})

7. 路由服务部署

3台服务器执行相同操做

在/data/mongodb/conf目录建立mongos.conf，内容以下

[root@node4 conf]# cat mongos.conf 
logpath=/data/mongodb/log/mongos.log/mongos.log
logappend = true
port = 27017
fork = true
configdb = configs/172.16.245.102:27018,172.16.245.103:27018,172.16.245.104:27018
maxConns=20000
bind_ip=0.0.0.0

启动mongos
分别在三台服务器启动：

[root@node4 conf]# /data/mongodb/bin/mongos -f /data/mongodb/conf/mongos.conf

8. 启动分片功能

链接mongo

mongo --host 172.16.245.102 --port 27017

mongos>use admin

添加分片，只需在一台机器执行便可

mongos>sh.addShard("shard1/172.16.245.102:27001,172.16.245.103:27001,172.16.245.104:27001")
mongos>sh.addShard("shard2/172.16.245.102:27002,172.16.245.103:27002,172.16.245.104:27002")
mongos>sh.addShard("shard3/172.16.245.102:27003,172.16.245.103:27003,172.16.245.104:27003")


mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("5fe184bf29ea91799b557a8b")
  }
  shards:
        {  "_id" : "shard1",  "host" : "shard1/172.16.245.102:27001,172.16.245.103:27001,172.16.245.104:27001",  "state" : 1 }
        {  "_id" : "shard2",  "host" : "shard2/172.16.245.102:27002,172.16.245.103:27002,172.16.245.104:27002",  "state" : 1 }
        {  "_id" : "shard3",  "host" : "shard3/172.16.245.102:27003,172.16.245.103:27003,172.16.245.104:27003",  "state" : 1 }
  active mongoses:
        "4.0.21" : 3
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours: 
                No recent migrations
  databases:
        {  "_id" : "calon",  "primary" : "shard1",  "partitioned" : true,  "version" : {  "uuid" : UUID("2a4780da-8f33-4214-88f8-c9b1a3140299"),  "lastMod" : 1 } }
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                shard1  1
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) 
        {  "_id" : "test",  "primary" : "shard2",  "partitioned" : false,  "version" : {  "uuid" : UUID("d59549a4-3e68-4a7d-baf8-67a4d8372b76"),  "lastMod" : 1 } }
        {  "_id" : "ycsb",  "primary" : "shard3",  "partitioned" : true,  "version" : {  "uuid" : UUID("6d491868-245e-4c86-a5f5-f8fcd308b45e"),  "lastMod" : 1 } }
                ycsb.usertable
                        shard key: { "_id" : "hashed" }
                        unique: false
                        balancing: true
                        chunks:
                                shard1  2
                                shard2  2
                                shard3  2
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong("-6148914691236517204") } on : shard1 Timestamp(1, 0) 
                        { "_id" : NumberLong("-6148914691236517204") } -->> { "_id" : NumberLong("-3074457345618258602") } on : shard1 Timestamp(1, 1) 
                        { "_id" : NumberLong("-3074457345618258602") } -->> { "_id" : NumberLong(0) } on : shard2 Timestamp(1, 2) 
                        { "_id" : NumberLong(0) } -->> { "_id" : NumberLong("3074457345618258602") } on : shard2 Timestamp(1, 3) 
                        { "_id" : NumberLong("3074457345618258602") } -->> { "_id" : NumberLong("6148914691236517204") } on : shard3 Timestamp(1, 4) 
                        { "_id" : NumberLong("6148914691236517204") } -->> { "_id" : { "$maxKey" : 1 } } on : shard3 Timestamp(1, 5)

9.实现分片功能

设置分片chunk大小

mongos>use config
mongos>db.setting.save({"_id":"chunksize","value":1})  #设置块大小为1M是方便实验，否则须要插入海量数据

10. 启用数据库分片并进行测试

mongos> use shardbtest;
switched to db shardbtest
mongos> 
mongos> 
mongos> sh.enableSharding("shardbtest");
{
        "ok" : 1,
        "operationTime" : Timestamp(1608620190, 4),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1608620190, 4),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
mongos> sh.shardCollection("shardbtest.usertable",{"_id":"hashed"});   #为 shardbtest裤中的usertable表进行分片基于id的哈希分片
{
        "collectionsharded" : "shardbtest.usertable",
        "collectionUUID" : UUID("2b5a8bcf-6e31-4dac-831f-5fa414253655"),
        "ok" : 1,
        "operationTime" : Timestamp(1608620216, 36),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1608620216, 36),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
mongos> for(i=1;i<=3000;i++){db.usertable.insert({"id":i})}     #模拟插入3000条的数据
WriteResult({ "nInserted" : 1 })

11. 查看分片验证

mongos> db.usertable.stats();
{
        "sharded" : true,
        "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
        "userFlags" : 1,
        "capped" : false,
        "ns" : "shardbtest.usertable",
        "count" : 3000,                 #总3000
        "numExtents" : 9,
        "size" : 144096,
        "storageSize" : 516096,
        "totalIndexSize" : 269808,
        "indexSizes" : {
                "_id_" : 122640,
                "_id_hashed" : 147168
        },
        "avgObjSize" : 48,
        "maxSize" : NumberLong(0),
        "nindexes" : 2,
        "nchunks" : 6,
        "shards" : {
                "shard3" : {
                        "ns" : "shardbtest.usertable",
                        "size" : 48656,
                        "count" : 1013,                 #shard3写入1013
                        "avgObjSize" : 48,
                        "numExtents" : 3,
                        "storageSize" : 172032,
                        "lastExtentSize" : 131072,
                        "paddingFactor" : 1,
                        "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
                        "userFlags" : 1,
                        "capped" : false,
                        "nindexes" : 2,
                        "totalIndexSize" : 89936,
                        "indexSizes" : {
                                "_id_" : 40880,
                                "_id_hashed" : 49056
                        },
                        "ok" : 1,
                        "operationTime" : Timestamp(1608620309, 1),
                        "$gleStats" : {
                                "lastOpTime" : {
                                        "ts" : Timestamp(1608620272, 38),
                                        "t" : NumberLong(1)
                                },
                                "electionId" : ObjectId("7fffffff0000000000000001")
                        },
                        "lastCommittedOpTime" : Timestamp(1608620309, 1),
                        "$configServerState" : {
                                "opTime" : {
                                        "ts" : Timestamp(1608620307, 1),
                                        "t" : NumberLong(1)
                                }
                        },
                        "$clusterTime" : {
                                "clusterTime" : Timestamp(1608620309, 1),
                                "signature" : {
                                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                        "keyId" : NumberLong(0)
                                }
                        }
                },
                "shard2" : {
                        "ns" : "shardbtest.usertable",
                        "size" : 49232,
                        "count" : 1025,                      #shard2写入1025
                        "avgObjSize" : 48,
                        "numExtents" : 3,
                        "storageSize" : 172032,
                        "lastExtentSize" : 131072,
                        "paddingFactor" : 1,
                        "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
                        "userFlags" : 1,
                        "capped" : false,
                        "nindexes" : 2,
                        "totalIndexSize" : 89936,
                        "indexSizes" : {
                                "_id_" : 40880,
                                "_id_hashed" : 49056
                        },
                        "ok" : 1,
                        "operationTime" : Timestamp(1608620306, 1),
                        "$gleStats" : {
                                "lastOpTime" : {
                                        "ts" : Timestamp(1608620272, 32),
                                        "t" : NumberLong(1)
                                },
                                "electionId" : ObjectId("7fffffff0000000000000001")
                        },
                        "lastCommittedOpTime" : Timestamp(1608620306, 1),
                        "$configServerState" : {
                                "opTime" : {
                                        "ts" : Timestamp(1608620307, 1),
                                        "t" : NumberLong(1)
                                }
                        },
                        "$clusterTime" : {
                                "clusterTime" : Timestamp(1608620309, 1),
                                "signature" : {
                                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                        "keyId" : NumberLong(0)
                                }
                        }
                },
                "shard1" : {
                        "ns" : "shardbtest.usertable",
                        "size" : 46208,
                        "count" : 962,                  #shard1写入962
                        "avgObjSize" : 48,
                        "numExtents" : 3,
                        "storageSize" : 172032,
                        "lastExtentSize" : 131072,
                        "paddingFactor" : 1,
                        "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
                        "userFlags" : 1,
                        "capped" : false,
                        "nindexes" : 2,
                        "totalIndexSize" : 89936,
                        "indexSizes" : {
                                "_id_" : 40880,
                                "_id_hashed" : 49056
                        },
                        "ok" : 1,
                        "operationTime" : Timestamp(1608620308, 1),
                        "$gleStats" : {
                                "lastOpTime" : {
                                        "ts" : Timestamp(1608620292, 10),
                                        "t" : NumberLong(1)
                                },
                                "electionId" : ObjectId("7fffffff0000000000000001")
                        },
                        "lastCommittedOpTime" : Timestamp(1608620308, 1),
                        "$configServerState" : {
                                "opTime" : {
                                        "ts" : Timestamp(1608620307, 1),
                                        "t" : NumberLong(1)
                                }
                        },
                        "$clusterTime" : {
                                "clusterTime" : Timestamp(1608620309, 1),
                                "signature" : {
                                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                        "keyId" : NumberLong(0)
                                }
                        }
                }
        },
        "ok" : 1,
        "operationTime" : Timestamp(1608620309, 1),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1608620309, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}

11. 副本节点是否已同步数据

mongos> show dbs
admin       0.000GB
calon       0.078GB
config      0.235GB
shardbtest  0.234GB
test        0.078GB
ycsb        0.234GB
mongos> use shardbtest
switched to db shardbtest
mongos> db.usertable.stats();
{
        "sharded" : true,
        "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
        "userFlags" : 1,
        "capped" : false,
        "ns" : "shardbtest.usertable",
        "count" : 3000,
        "numExtents" : 9,
        "size" : 144096,
        "storageSize" : 516096,
        "totalIndexSize" : 269808,
        "indexSizes" : {
                "_id_" : 122640,
                "_id_hashed" : 147168
        },
        "avgObjSize" : 48,
        "maxSize" : NumberLong(0),
        "nindexes" : 2,
        "nchunks" : 6,
        "shards" : {
                "shard2" : {
                        "ns" : "shardbtest.usertable",
                        "size" : 49232,
                        "count" : 1025,
                        "avgObjSize" : 48,
                        "numExtents" : 3,
                        "storageSize" : 172032,
                        "lastExtentSize" : 131072,
                        "paddingFactor" : 1,
                        "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
                        "userFlags" : 1,
                        "capped" : false,
                        "nindexes" : 2,
                        "totalIndexSize" : 89936,
                        "indexSizes" : {
                                "_id_" : 40880,
                                "_id_hashed" : 49056
                        },
                        "ok" : 1,
                        "operationTime" : Timestamp(1608620886, 6),
                        "$gleStats" : {
                                "lastOpTime" : Timestamp(0, 0),
                                "electionId" : ObjectId("7fffffff0000000000000001")
                        },
                        "lastCommittedOpTime" : Timestamp(1608620886, 6),
                        "$configServerState" : {
                                "opTime" : {
                                        "ts" : Timestamp(1608620888, 1),
                                        "t" : NumberLong(1)
                                }
                        },
                        "$clusterTime" : {
                                "clusterTime" : Timestamp(1608620888, 1),
                                "signature" : {
                                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                        "keyId" : NumberLong(0)
                                }
                        }
                },
                "shard3" : {
                        "ns" : "shardbtest.usertable",
                        "size" : 48656,
                        "count" : 1013,
                        "avgObjSize" : 48,
                        "numExtents" : 3,
                        "storageSize" : 172032,
                        "lastExtentSize" : 131072,
                        "paddingFactor" : 1,
                        "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
                        "userFlags" : 1,
                        "capped" : false,
                        "nindexes" : 2,
                        "totalIndexSize" : 89936,
                        "indexSizes" : {
                                "_id_" : 40880,
                                "_id_hashed" : 49056
                        },
                        "ok" : 1,
                        "operationTime" : Timestamp(1608620889, 1),
                        "$gleStats" : {
                                "lastOpTime" : Timestamp(0, 0),
                                "electionId" : ObjectId("7fffffff0000000000000001")
                        },
                        "lastCommittedOpTime" : Timestamp(1608620889, 1),
                        "$configServerState" : {
                                "opTime" : {
                                        "ts" : Timestamp(1608620888, 1),
                                        "t" : NumberLong(1)
                                }
                        },
                        "$clusterTime" : {
                                "clusterTime" : Timestamp(1608620889, 1),
                                "signature" : {
                                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                        "keyId" : NumberLong(0)
                                }
                        }
                },
                "shard1" : {
                        "ns" : "shardbtest.usertable",
                        "size" : 46208,
                        "count" : 962,
                        "avgObjSize" : 48,
                        "numExtents" : 3,
                        "storageSize" : 172032,
                        "lastExtentSize" : 131072,
                        "paddingFactor" : 1,
                        "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
                        "userFlags" : 1,
                        "capped" : false,
                        "nindexes" : 2,
                        "totalIndexSize" : 89936,
                        "indexSizes" : {
                                "_id_" : 40880,
                                "_id_hashed" : 49056
                        },
                        "ok" : 1,
                        "operationTime" : Timestamp(1608620888, 1),
                        "$gleStats" : {
                                "lastOpTime" : Timestamp(0, 0),
                                "electionId" : ObjectId("7fffffff0000000000000001")
                        },
                        "lastCommittedOpTime" : Timestamp(1608620888, 1),
                        "$configServerState" : {
                                "opTime" : {
                                        "ts" : Timestamp(1608620888, 1),
                                        "t" : NumberLong(1)
                                }
                        },
                        "$clusterTime" : {
                                "clusterTime" : Timestamp(1608620888, 1),
                                "signature" : {
                                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                        "keyId" : NumberLong(0)
                                }
                        }
                }
        },
        "ok" : 1,
        "operationTime" : Timestamp(1608620889, 1),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1608620889, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
mongos>

以上就实现了mongodb复制集的高可用以及分片