Paeony：开源Etcdv3-cpp-api 库入门介绍

时间  2020-12-07
标签 c++ git github vim api 网络 多线程 负载均衡 线程 栏目 C&C++ 繁體版
原文   原文链接
日期 做者 版本 备注
2020-12-7 dingbin v1.0

Paeony：Etcdv3-cpp-api 库简介

Paeony是做者开源的用C++语言实现成熟的etcd v3版本客户端API库。Github地址是：https://github.com/apollo008/paeony 。它达到稳定可靠的企业级应用效果。它在原始单纯etcdv3-cpp-api基础上，封装了不少面向负载均衡和链接重试的特性。它具备如下优点：c++

  1. 重试机制: 当etcd服务端不可链接或网络出现问题后，Paeony客户端会自动作出合理时间间隔的屡次链接重试，而且兼顾从新链接的负载均衡，直到再次链接成功为止，保证系统不由于一个etcd server链接不可用而宕机；链接重试过程同步输出详细日志，帮助用户迅速跟踪定位Etcd服务链接情况异常缘由，以快速解决etcd服务链接问题。整个重连过程用户无感知。
  2. 链接状态监控: Paeony初始化以后会一直对etcd链接进行监听，一旦发现链接状态发生变化将会做出相应的处理。
  3. Etcd客户端链接实例管理:Paeony会对etcdv3客户端到server集群的链接进行管理，并在须要的时候重建etcd链接实例，保证与etcd集群链接的可靠性。
  4. 采用做者开源成熟稳定的日志系统 tulip-log , Github地址：https://github.com/apollo008/tulip-log ,同步输出丰富的etcdv3客户端操做状态和链接状态日志信息，便于排查线上问题。
  5. 对etcdv3 客户端的watch和leaseKeepAlive 过程作了尤为加强支持，使得该etcdv3 客户端不会由于某个etcd server不可用或网络抖动致使watch失效或关键心跳节点丢失等严重问题。顽强稳定地支持适用各类网络环境和etcd servers 链接异常情况的watch 和leaseKeepAlive表现。
  6. 客户端所有接口都支持多线程同步访问，保证多线程操做访问下结果也能正确可靠。
  7. 支持Etcdv3版本事务Transaction操做。
  8. 结合demo示例：src/paeony/paeony/core/examples/hello_paeony.cpp, 接口设计便捷易用。

基于Paeony的Etcd v3 c++ api基本用法

项目的src/paeony/paeony/core/examples目录给出了经常使用的etcdv3 client使用示例。git

具体来讲，demo使用入口见 src/paeony/paeony/core/examples/hello_paeony.cpp代码示例：github

#include "paeony/core/examples/paeony_demo.h"

PAEONY_USE_NAMESPACE;
STD_USE_NAMESPACE;

int main(int argc, char** argv) {
    TLOG_CONFIG("logger.conf");
    TLOG_DECLARE_AND_SETUP_LOGGER(HELLOPAEONY, MAIN);

    EtcdDemoConnInfo info;
    PaeonyDemo demo(info);

    demo.DemoEtcdv3ClientGetPut();
    demo.DemoEtcdv3Lease();
    demo.DemoEtcdv3Watch();
    demo.DemoEtcdv3Transaction();

    TLOG_LOG_FLUSH();
    TLOG_LOG_SHUTDOWN();

    return 0;
}

具体的4种主要的Etcdv3 client 接口Demo参考文件：src/paeony/paeony/core/examples/paeony_demo.cppvim

  • Etcdv3Client/Get/Put

/**
 *@brief     Method to demo etcdv3 client and Get/Put method
 *@author    dingbinthu@163.com
 *@date      2020/12/7, 上午1:14
 */
void PaeonyDemo::DemoEtcdv3ClientGetPut() {
    Etcdv3ClientPtr client = std::make_shared<Etcdv3Client>(m_info.m_certificateContents,
                                                            m_info.m_user,
                                                            m_info.m_passwd,
                                                            m_info.m_allEndpoints,
                                                            time(NULL),
                                                            m_info.m_etctTestKey);

    try {
        while (true) {
            paeony::EtcdRangeResponse getResponse = client->Get(m_info.m_etctTestKey, true, false,
                                                               false, 0, 3,
                                                               2, 50);
            if (getResponse.IsOk()) {
                TLOG_LOG(INFO,"Successfully get, value=[%s].",
                (getResponse.m_rpcResp.kvs_size()>0?getResponse.m_rpcResp.kvs(0).value().c_str(): "") );
            }
            else {
                TLOG_LOG(ERROR,"Failed to get,error_code is:[%d],operation failed,detail message:[%s]",
                                      getResponse.GetErrorCode(), getResponse.GetErrorMsg().c_str() );

            }
            EtcdPutResponse putResponse = client->Put(m_info.m_etctTestKey + "/123","hello,paeony!!!",
                                                                                  0, false, 5, 2, 50);
            if (putResponse.IsOk()) {
                TLOG_LOG(INFO,"Successfully put key:[%s]", m_info.m_etctTestKey.c_str());
            }
            else {
                TLOG_LOG(ERROR,"Failed to put, error_code is:[%d],operation failed,detail message:[%s]",
                                       putResponse.GetErrorCode(),putResponse.GetErrorMsg().c_str() );
            }
            sleep(3);
        }
    }
    catch (std::exception const & ex)
    {
        TLOG_LOG(ERROR,"Communication problem,details:[%s]", ex.what());
    }
}

  • Watch

/**
 *@brief     Method used to demo Etcdv3 Watch interface
 *@author    dingbinthu@163.com
 *@date      2020/12/7, 上午1:15
 */
void PaeonyDemo::DemoEtcdv3Watch() {

    Etcdv3ClientPtr client = std::make_shared<Etcdv3Client>(m_info.m_certificateContents,
                                                            m_info.m_user,
                                                            m_info.m_passwd,
                                                            m_info.m_allEndpoints,
                                                            time(NULL),
                                                            m_info.m_etctTestKey);

    string key = "/paeony/TestWatch";
    client ->Put(key,"0");

    function<void(EtcdWatchResponse)> watchCallback = [](EtcdWatchResponse response) {
        PaeonyDemo::_logger->Log(tulip::TLOG_LEVELNO_ERROR,__FILE__,__LINE__,__FUNCTION__,
        "got watch response, with response's events size:[%d] and if for the firstResponseCallback:[%s]",
        response.m_rpcResp.events_size(), (response.m_bFirstResponse ? "true":"false"));
    };

    vector<WatchCreateRequest::FilterType>  filtersVec;
    EtcdWatchWorkerPtr watchWorker = make_shared<EtcdWatchWorker>(client,key,true,true,true,filtersVec,
                                                                 watchCallback,100,30,4 * 1024 * 1024);

    watchWorker->Start();
    while(!watchWorker->IsStarted()){
        TLOG_LOG(INFO, "wait EtcdWatchWorker start...");
        usleep(10);
    }

    TLOG_LOG(INFO,"EtcdWatchWorker started......");
    sleep(180);
    TLOG_LOG(INFO, "=======================now terminate EtcdWatchWorker...");
    watchWorker->Terminate();
    TLOG_LOG(INFO,"EtcdWatchWorker terminate, wait done");
    watchWorker->Join();
    TLOG_LOG(INFO,"EtcdWatchWorker done.");
    int remainTimeSecond = 10;
    int t = 0;
    while (remainTimeSecond - t * 2 > 0) {
        TLOG_LOG(INFO,"Wait 10 seconds to exit Function:[%s], remain [%d] second..",
        __FUNCTION__ , remainTimeSecond - t * 2);
        ++t;
        sleep(2);
    }
    TLOG_LOG(INFO,"now exit function:[%s]", __FUNCTION__ );
}

  • Lease

/**
 *@brief     Method used to demo Etcdv3 Lease interface
 *@author    dingbinthu@163.com
 *@date      2020/12/7, 上午1:15
 */
void PaeonyDemo::DemoEtcdv3Lease() {
    Etcdv3ClientPtr client = std::make_shared<Etcdv3Client>(m_info.m_certificateContents,
                                                            m_info.m_user,
                                                            m_info.m_passwd,
                                                            m_info.m_allEndpoints,
                                                            time(NULL),
                                                            m_info.m_etctTestKey);
    int64_t ttl = 60;
    EtcdLeaseGrantResponse lgr;
    do {
        lgr = client ->LeaseGrant(ttl,0,3,2,50);
        if (lgr.IsOk()) {
            break;
        }
        else {
        TLOG_LOG(ERROR,"Lease grant errCode:[%d],errMsg:[%s]",
                       lgr.GetErrorCode(),lgr.GetErrorMsg().c_str());
        }
        sleep(1);
    } while(true);

    client ->Put("/paeony/TestLeaseKeepAlive","0",lgr.m_rpcResp.id());

    function<void(LeaseGrantResponse)> reLeaseGrantCallback = [client](LeaseGrantResponse response) {
        time_t tm = time(NULL);
        ostringstream oss;
        oss <<tm;
        client ->Put("/paeony/TestLeaseKeepAlive",oss.str(),response.id());
        PaeonyDemo::_logger->Log(tulip::TLOG_LEVELNO_INFO,__FILE__,__LINE__,__FUNCTION__,
                    "got reLeaseGrant response, with id:[%ld] and ttl:[%ld]", response.id(),response.ttl());
    };

    EtcdLeaseKeepAliveWorkerPtr leaseKeepAliveWorker = make_shared<EtcdLeaseKeepAliveWorker> (client,
               lgr.m_rpcResp.id(),lgr.m_rpcResp.ttl(),reLeaseGrantCallback,100,ttl + 2,2 * 1024 * 1024);

    leaseKeepAliveWorker->Start();
    while(!leaseKeepAliveWorker->IsStarted()){
        TLOG_LOG(INFO,"wait LeaseKeepAliveWorker start");
        usleep(10);
    }
    TLOG_LOG(INFO,"LeaseKeepAliveWorker started");
    sleep(60 * 60);

    TLOG_LOG(INFO,"=======================now terminate LeaseKeepAliveWorker...");
    leaseKeepAliveWorker->Terminate();
    TLOG_LOG(INFO,"LeaseKeepAliveWorker terminate, wait done");
    leaseKeepAliveWorker->Join();
    TLOG_LOG(INFO,"LeaseKeepAliveWorker done.");

    int remainTimeSecond = 3 * 60;
    int t = 0;
    while (remainTimeSecond - t * 5 > 0) {
        TLOG_LOG(INFO,"Wait 3 minute to exit Function:[%s], remain [%d] second...",
                                             __FUNCTION__,remainTimeSecond - t * 5);
        ++t;
        sleep(5);
    }
    TLOG_LOG(INFO,"now exit function:[%s]",__FUNCTION__);
}

  • Transaction

/**
 *@brief     Method used to demo Etcdv3 transaction interface
 *@author    dingbinthu@163.com
 *@date      2020/12/7, 上午1:15
 */
void PaeonyDemo::DemoEtcdv3Transaction() {
    string key = "/paeony/TestTxn";

    Compare compare;
    compare.set_result(etcdserverpb::Compare::CompareResult::Compare_CompareResult_EQUAL);
    compare.set_target(etcdserverpb::Compare::CompareTarget::Compare_CompareTarget_VERSION);
    compare.set_key(key);
    compare.set_version(0);

    TransactionRequestPtrVec successTxnRequestsVec;
    TransactionRequestPtr put1 = 
                    make_shared<TransactionPutRequest>(Etcdv3Client::s_ConstructPutRequest(key,"1"));
    successTxnRequestsVec.push_back(put1);
    TransactionRequestPtr range1 = 
                    make_shared<TransactionRangeRequest>(Etcdv3Client::s_ConstructRangeRequest(key));
    successTxnRequestsVec.push_back(range1);
    TransactionRequestPtr put2 = 
                   make_shared<TransactionPutRequest>(Etcdv3Client::s_ConstructPutRequest(key + "1","2"));
    successTxnRequestsVec.push_back(put2);
    TransactionRequestPtr range2 = 
                   make_shared<TransactionRangeRequest>(Etcdv3Client::s_ConstructRangeRequest(key + "1"));
    successTxnRequestsVec.push_back(range2);


    TransactionRequestPtrVec  failureTxnRequestsVec;
    TransactionRequestPtr put1_ = 
                       make_shared<TransactionPutRequest>(Etcdv3Client::s_ConstructPutRequest(key,"1_"));
    failureTxnRequestsVec.push_back(put1_);
    TransactionRequestPtr range1_ = 
                       make_shared<TransactionRangeRequest>(Etcdv3Client::s_ConstructRangeRequest(key));
    failureTxnRequestsVec.push_back(range1_);
    TransactionRequestPtr put2_ = 
                      make_shared<TransactionPutRequest>(Etcdv3Client::s_ConstructPutRequest(key+"2","2_"));
    failureTxnRequestsVec.push_back(put2_);
    TransactionRequestPtr range2_ = 
                      make_shared<TransactionRangeRequest>(Etcdv3Client::s_ConstructRangeRequest(key +"2"));
    failureTxnRequestsVec.push_back(range2_);

    Etcdv3ClientPtr client = std::make_shared<Etcdv3Client>(m_info.m_certificateContents,
                                                            m_info.m_user,
                                                            m_info.m_passwd,
                                                            m_info.m_allEndpoints,
                                                            time(NULL),
                                                            m_info.m_etctTestKey);
    EtcdTxnResponse txnResponse = 
                         client->Transaction(compare,successTxnRequestsVec,failureTxnRequestsVec,8,3,50);

    if (txnResponse.IsOk()) {
        TLOG_LOG(INFO, "Succeed in executing transaction operation whose txnResponse's succeeded is:[%s]",
                                                    (txnResponse.m_rpcResp.succeeded() ? "true":"false"));
    }
    else {
        TLOG_LOG(ERROR,"Failed to execute transaction operation with errorCode:[%d] and errorMsg:[%s]",
                                         txnResponse.GetErrorCode(), txnResponse.GetErrorMsg().c_str() );
    }
}

Tulip-log的配置文件示例参考：src/paeony/paeony/core/examples/logger.confapi

编译和安装

目前支持类Unix环境下编译安装，Paeony项目依赖的第三方库有：网络

  • protobuf
  • grpc
  • openssl
  • tulip-log

编译安装paeony以前先要安装以上4个依赖库。多线程

具体编译和安装方法以下：负载均衡

git clone https://github.com/apollo008/paeony.git paeony.git
cd paeony.git
vim CMakeLists.txt 修改第8行：
原来是：set(PAEONY_DEPEND_PREFIX_DIR /path/to/install/share)
将/path/to/install/share 替换为 安装上面4个目标依赖库的路径，好比${HOME}/local,注意确保该路径有写权限。
修改以后后续不要再改变该路径。

mkdir build-dir
cd build-dir
#安装依赖
cmake -DENABLE_BUILD_SHARE=ON ../src
执行完以上这一步便可完成依赖库。注意不须要再执行make 和make install了。

接下来安装paeony
#安装paeony
首先保持根目录下CMakeLists.txt第8行修改的内容再也不改变；
cd build-dir
rm -rf *
cmake -DCMAKE_INSTALL_PREFIX=/path/to/install  ../src
make -j10
make install

执行完以上，便可在/path/to/install目录下生成bin、lib、include3个目录。其中lib目录是libpaeony.so; bin目录下是hello_paeony可执行程序； include目录是paeony库的头文件。ui

注意：运行demo程序hello_paeony时，须要在当前目录放置tulip-log的日志配置文件logger.conf ，同时该目录下要已经建立好logs 目录供输出文件名滚动的日志文件。线程

其它

相关细节或其它未尽事宜，可联系 dingbinthu@163.com 探讨咨询。

相关文章
相关标签/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公众号
   欢迎关注本站公众号,获取更多信息
联系我们 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程