OpenStack 实现技术分解 (5) 应用开发 — 使用 OpenStackClients 进行二次开发

时间 2019-11-18

标签 openstack 实现技术分解应用开发使用 openstackclients 进行二次开发繁體版

原文原文链接

前文列表

OpenStack 实现技术分解 (1) 开发环境 — Devstack 部署案例详解
 OpenStack 实现技术分解 (2) 虚拟机初始化工具 — Cloud-Init & metadata & userdata
OpenStack 实现技术分解 (3) 开发工具 — VIM & dotfiles
OpenStack 实现技术分解 (4) 通用技术 — TaskFlowcss

参考阅读

Openstack API 类型 & REST 风格
 OpenStackClients
Python bindings to the OpenStack Identity API
Python Bindings for the OpenStack Images API
Python bindings to the OpenStack Nova API
Cinder Python API
Python bindings to the OpenStack Networking APIhtml

前言

OpenStack 为用户提供了三种操做方式, Web界面/CLI/RESTAPI, 实际上前二者是对 RESTAPI 作了两种不一样形式的包装, 使用户能够经过网页或者指令行的方式来调用 RESTAPI 接口. python

本篇博文主要记录了使用 OpenStackClients (OSC 命令行客户端) 项目所提供了Python Bindings API 来进行二次开发的技巧, 以及实现一个启动虚拟机并部署 Workpass+MySQL 自动化脚本的 Demo. 源码详见 GitHub: openstackclient-api-demomysql

在介绍 OpenStackClients 以前, 咱们能够尝试直接使用 curl 指令来查看一个 tenant 所含有的虚拟机列表.git

Step 1: 获取帐户 admin 的身份验证 Temporary token

curl -k -X 'POST' -v http://200.21.18.2:5000/v2.0/tokens -d '{"auth":{"passwordCredentials":{"username": "admin", "password":"fanguiju"}}}' -H 'Content-type: application/json' | python -mjson.tool

Response:github

{
    "access": { "metadata": { "is_admin": 0, "roles": [] }, "serviceCatalog": [], "token": { "audit_ids": [ "AOMhHXq_Qx2Nz41RVoUy7g" ], "expires": "2017-03-19T05:41:20Z", "id": "16ae22b6c36f4ebc97938f51b7d0631b", "issued_at": "2017-03-19T04:41:20.039145" }, "user": { "id": "135b2cb86962401c82044fd4ca9daae4", "name": "admin", "roles": [], "roles_links": [], "username": "admin" } } }

获取到 Temporary token: 16ae22b6c36f4ebc97938f51b7d0631b, 表示咱们的帐户信息经过了验证流程. sql

Step 2: 使用 Temporary token 来获取 tenants list

curl -X 'GET' -H  "X-Auth-Token:16ae22b6c36f4ebc97938f51b7d0631b" -v http://200.21.18.2:5000/v2.0/tenants | python -mjson.tool

Response:json

{
    "tenants": [ { "description": "", "enabled": true, "id": "6c4e4d58cb9d4451b36e774b348e8813", "name": "admin" }, { "description": "", "enabled": true, "id": "ad9a69f3da8f4aa280389fcdf855aeb5", "name": "demo" } ],
    "tenants_links": [] }

能够看出 admin 帐户含有 admin tenant 和 demo tenant. ubuntu

Step 3: 获取 admin tenant 的 token 信息

curl -k -X 'POST' -v http://200.21.18.2:5000/v2.0/tokens -d '{"auth":{"passwordCredentials":{"username": "admin", "password":"fanguiju"},"tenantId":"6c4e4d58cb9d4451b36e774b348e8813"}}' -H 'Content-type: application/json' | python -mjson.tool

Response:vim

{
    "access": { "metadata": { "is_admin": 0, "roles": [ "14a6da35e3ef4e47a540c6608aa00ca7" ] }, "serviceCatalog": [ { "endpoints": [ { "adminURL": "http://200.21.18.2:8774/v2.1/6c4e4d58cb9d4451b36e774b348e8813", "id": "705f599f3bae42ceb4a70616d9663ad8", "internalURL": "http://200.21.18.2:8774/v2.1/6c4e4d58cb9d4451b36e774b348e8813", "publicURL": "http://200.21.18.2:8774/v2.1/6c4e4d58cb9d4451b36e774b348e8813", "region": "RegionOne" } ], "endpoints_links": [], "name": "nova", "type": "compute" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:8760/v1.1/6c4e4d58cb9d4451b36e774b348e8813", "id": "39ceecd18b754c9495834d0155fe91bf", "internalURL": "http://200.21.18.2:8760/v1.1/6c4e4d58cb9d4451b36e774b348e8813", "publicURL": "http://200.21.18.2:8760/v1.1/6c4e4d58cb9d4451b36e774b348e8813", "region": "RegionOne" } ], "endpoints_links": [], "name": "egis", "type": "recovery" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:8776/v2/6c4e4d58cb9d4451b36e774b348e8813", "id": "218769a91d0943ff8db44887645ec0ff", "internalURL": "http://200.21.18.2:8776/v2/6c4e4d58cb9d4451b36e774b348e8813", "publicURL": "http://200.21.18.2:8776/v2/6c4e4d58cb9d4451b36e774b348e8813", "region": "RegionOne" } ], "endpoints_links": [], "name": "cinderv2", "type": "volumev2" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:9292", "id": "7f2f8036b0194ea0bd5231710b2cddf4", "internalURL": "http://200.21.18.2:9292", "publicURL": "http://200.21.18.2:9292", "region": "RegionOne" } ], "endpoints_links": [], "name": "glance", "type": "image" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813", "id": "054567bc62ce4b4fbdbdcd7c3a23748e", "internalURL": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813", "publicURL": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813", "region": "RegionOne" } ], "endpoints_links": [], "name": "nova_legacy", "type": "compute_legacy" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:8776/v1/6c4e4d58cb9d4451b36e774b348e8813", "id": "2eefe27748774693b635bf48f486f225", "internalURL": "http://200.21.18.2:8776/v1/6c4e4d58cb9d4451b36e774b348e8813", "publicURL": "http://200.21.18.2:8776/v1/6c4e4d58cb9d4451b36e774b348e8813", "region": "RegionOne" } ], "endpoints_links": [], "name": "cinder", "type": "volume" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:8773/", "id": "4d8f727748924cdf9d23591bad2bbd19", "internalURL": "http://200.21.18.2:8773/", "publicURL": "http://200.21.18.2:8773/", "region": "RegionOne" } ], "endpoints_links": [], "name": "ec2", "type": "ec2" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:35357/v2.0", "id": "16e2a0df7fa64c8cbcdb5936e23b19cc", "internalURL": "http://200.21.18.2:5000/v2.0", "publicURL": "http://200.21.18.2:5000/v2.0", "region": "RegionOne" } ], "endpoints_links": [], "name": "keystone", "type": "identity" } ], "token": { "audit_ids": [ "4zrwvCd7TySk7jJKuO4G1Q" ], "expires": "2017-03-19T05:48:41Z", "id": "74e396f8202b481a9cbd95b319a4314b", "issued_at": "2017-03-19T04:48:42.002243", "tenant": { "description": "", "enabled": true, "id": "6c4e4d58cb9d4451b36e774b348e8813", "name": "admin" } }, "user": { "id": "135b2cb86962401c82044fd4ca9daae4", "name": "admin", "roles": [ { "name": "admin" } ], "roles_links": [], "username": "admin" } } }

须要注意的是, 这一步骤所获取的 Tenant token 是区别于 Temporary token 的, Temporary token 做为临时 token 是为了实现多租户的场景所提供的鉴权条件(外部鉴权). 而 Tenant token 才是联系不一样 OpenStack Project 间的认证通行证(内部鉴权). 从这一步骤能够看出想要获取 Tenant token 就须要同时向 Keystone 提供帐户信息和 tenant_id, 此时用户不只获得了 Tenant token 还获取了相应的 endpoints list. 而且用户可以经过 endpints list 进一步的去访问注册在 Keystone 中的其余 OpenStack 组件.

Step 4: 使用 Tenant token 和 tenant_id 获取 admin tenant 的虚拟机列表

curl -v -H "X-Auth-Token:74e396f8202b481a9cbd95b319a4314b" http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813/servers

Response:

{
    "servers": [ { "id": "138ecea2-1656-46bd-aefd-39449e11c356", "links": [ { "href": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813/servers/138ecea2-1656-46bd-aefd-39449e11c356", "rel": "self" }, { "href": "http://200.21.18.2:8774/6c4e4d58cb9d4451b36e774b348e8813/servers/138ecea2-1656-46bd-aefd-39449e11c356", "rel": "bookmark" } ], "name": "aju_test_dvs" }, { "id": "42da5d12-a470-4193-8410-0209c04f333a", "links": [ { "href": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813/servers/42da5d12-a470-4193-8410-0209c04f333a", "rel": "self" }, { "href": "http://200.21.18.2:8774/6c4e4d58cb9d4451b36e774b348e8813/servers/42da5d12-a470-4193-8410-0209c04f333a", "rel": "bookmark" } ], "name": "TestVMwareInterface" } ] }

最终, 咱们从 Response 中获得了 admin tenant 所具备的两台虚拟机的信息.

完整的 RESTAPI 请求流程图片以下:

显然, 使用 curl 请求 RESTAPI 的方式过于繁复, 不能知足用户对 OpenStack 多方位的应用需求(e.g. 实现 OpenStack 的自动化操做脚本). 对此, OpenStack 为用户提供了更高级别的 RESTAPI 调用封装 — OpenStackClients

OpenStackClients

(摘自 OpenStackClients 官方文档)Each OpenStack project has a related client project that includes Python API bindings and a CLI.
每个 OpenStack 项目都具备一个包含了 Python API bindings 和 CLI 相关的 client 项目. 如图:

有图可见, OpenStackClients 项目主要实现了将 OpenStack 计算(Compute)、身份识别(Keystone)、镜像(Glance)、网络(Neutron)、对象存储(Swift)和卷存储(Cinder) 等核心组件所提供出来的 REST API 整合封装为具备统一指令结构的 CLI. 简而言之, 就是 OpenStackClients 项目使得用户可以经过 CLI 的形式调用以上组件提供的 REST API, 从而实现操做. 而且咱们也能够从代码的层面直接导入 OpenStackClients, 更加便于开发者对 OpenStack 功能模块的调用.

使用 OpenStackClients 获取 project_client object 的 demo

vim openstack_clients.py

#!/usr/bin/env python
#encoding=utf8

from openstackclient.identity.client import identity_client_v2
from keystoneclient import session as identity_session
import glanceclient
import novaclient.client as novaclient
import cinderclient.client as cinderclient

# 定义 project_client version
NOVA_CLI_VER = 2
GLANCE_CLI_VER = 2
CINDER_CLI_VER = 2


class OpenstackClients(object):
    """Clients generator of openstack."""

    def __init__(self, auth_url, username, password, tenant_name):

        ### Identity authentication via keystone v2
        # An authentication plugin to authenticate the session with.
        # 经过身份验证信息获取 keystone 的 auth object
        # Keystoneclient v2 的详细使用介绍请浏览 https://docs.openstack.org/developer/python-keystoneclient/using-api-v2.html
        auth = identity_client_v2.v2_auth.Password(
            auth_url=auth_url,          # http://200.21.18.3:35357/v2.0/
            username=username,          # admin
            password=password,          # fanguiju
            tenant_name=tenant_name)    # admin
        try:
            # 经过 auth object 获取 Keystone 的 session object
            self.session = identity_session.Session(auth=auth)
        except Exception as err:
            raise

        # Return a token as provided by the auth plugin.
        # 经过 session object 获取 Tenant token
        self.token = self.session.get_token()

    def get_glance_client(self, interface='public'):
        """Get the glance-client object."""

        # Get an endpoint as provided by the auth plugin.
        # 默认获取 glance project 的 public endpoint
        glance_endpoint = self.session.get_endpoint(service_type="image",
                                                    interface=interface)
        # Client for the OpenStack Images API.
        # 经过 glance endpoint 和 token 获取 glance_client object
        # 而后就可使用 glance_client 调用其实例方法来实现对 glance project 的操做了
        # glanceclient v2 所提供的实例方法列表请浏览 https://docs.openstack.org/developer/python-glanceclient/ref/v2/images.html
        glance_client = glanceclient.Client(GLANCE_CLI_VER,
                                            endpoint=glance_endpoint,
                                            token=self.token)
        return glance_client

    def get_nova_client(self):
        """Get the nova-client object."""

        # Initialize client object based on given version. Don't need endpoint.
        # 也能够 不指定 endpoint 的类型, 仅使用 session object 来获取 nove_client
        # novaclient v2 的实例方法列表请浏览 https://docs.openstack.org/developer/python-novaclient/api.html#usage 
        nova_client = novaclient.Client(NOVA_CLI_VER, session=self.session)
        return nova_client

    def get_cinder_client(self, interface='public'):
        """Get the cinder-client object."""

        cinder_endpoint = self.session.get_endpoint(service_type='volume',
                                                    interface=interface)
        # cinder_client v2 的实例方法列表请查看 https://docs.openstack.org/developer/python-cinderclient/
        cinder_client = cinderclient.Client(CINDER_CLI_VER, session=self.session)
        return cinder_client

调用 project_client object 实例方法实现对 project 操做的 demo

vim auto_dep.py

#!/usr/bin/env python
#encoding=utf8


import os
from os import path
import time

import openstack_clients as os_cli


# FIXME(Fan Guiju): Using oslo_config and logging
AUTH_URL = 'http://200.21.18.3:35357/v2.0/'
USERNAME = 'admin'
PASSWORD = 'fanguiju'
PROJECT_NAME = 'admin'

DISK_FORMAT = 'qcow2'
IMAGE_NAME = 'ubuntu_server_1404_x64'
IMAGE_PATH = path.join(path.curdir, 'images',
                       '.'.join([IMAGE_NAME, DISK_FORMAT]))

MIN_DISK_SIZE_GB = 20

KEYPAIR_NAME = 'jmilkfan-keypair'
KEYPAIT_PUB_PATH = '/home/stack/.ssh/id_rsa.pub'

DB_NAME = 'blog'
DB_USER = 'wordpress'
DB_PASS = 'fanguiju'
DB_BACKUP_SIZE = 5
DB_VOL_NAME = 'mysql-volume'
DB_INSTANCE_NAME = 'AUTO-DEP-DB'
MOUNT_POINT = '/dev/vdb'

BLOG_INSTANCE_NAME = 'AUTO-DEP-BLOG'

TIMEOUT = 60


class AutoDep(object):
    def __init__(self, auth_url, username, password, tenant_name):
        # 实例化上述的 openstack_client.OpenstackClients 的对象
        openstack_clients = os_cli.OpenstackClients(
            auth_url,
            username,
            password,
            tenant_name)

        # 经过 openstack_clients 的实例方法获取 project_client 对象
        self._glance = openstack_clients.get_glance_client()
        self._nova = openstack_clients.get_nova_client()
        self._cinder = openstack_clients.get_cinder_client()

    def _wait_for_done(self, objs, target_obj_name):
        """Wait for action done."""
        count = 0
        while count <= TIMEOUT:
            for obj in objs.list():
                if obj.name == target_obj_name:
                    return
            time.sleep(3)
            count += 3
        raise

    def upload_image_to_glance(self):
        images = self._glance.images.list()
        for image in images:
            if image.name == IMAGE_NAME:
                return image
        # 调用 glanceclient.images.create method 建立一个 image object.
        new_image = self._glance.images.create(name=IMAGE_NAME,
                                               disk_format=DISK_FORMAT,
                                               container_format='bare',
                                               min_disk=MIN_DISK_SIZE_GB,
                                               visibility='public')
        # Open image file with read+binary.
        # 调用 glanceclient.images.upload method 上传一个 image
        self._glance.images.upload(new_image.id, open(IMAGE_PATH, 'rb'))
        self._wait_for_done(objs=self._glance.images,
                            target_obj_name=IMAGE_NAME)
        image = self._glance.images.get(new_image.id)
        return image

    def create_volume(self):
        # 调用 cinderclient.volumes.list method 获取 volumes 的列表
        volumes = self._cinder.volumes.list()
        for volume in volumes:
            if volume.name == DB_VOL_NAME:
                return volume
        # cinderclient.v2.volumes:VolumeManager
        # 调用 minderclient.volumes.create method 建立一个 volume
        new_volume = self._cinder.volumes.create(
            size=DB_BACKUP_SIZE,
            name=DB_VOL_NAME,
            volume_type='lvmdriver-1',
            availability_zone='nova',
            description='backup volume of mysql server.')
        if new_volume:
            return new_volume
        else:
            raise

    def get_flavor_id(self):
        # 调用 novaclient.flavors.list method 获取全部 flavors 的列表
        flavors = self._nova.flavors.list()
        for flavor in flavors:
            if flavor.disk == MIN_DISK_SIZE_GB:
                return flavor.id

    def _get_ssh_pub_key(self):
        if not path.exists(KEYPAIT_PUB_PATH):
            raise
        return open(KEYPAIT_PUB_PATH, 'rb').read()

    def import_keypair_to_nova(self):
        # 调用 novaclient.keypairs.list method 获取 keypairs 的列表
        keypairs = self._nova.keypairs.list()
        for keypair in keypairs:
            if keypair.name == KEYPAIR_NAME:
                return None
        keypair_pub = self._get_ssh_pub_key()
        # 调用 nova client.keypairs.create method 建立 keypair
        self._nova.keypairs.create(KEYPAIR_NAME, public_key=keypair_pub)

    def nova_boot(self, image, volume):
        flavor_id = self.get_flavor_id()
        self.import_keypair_to_nova()
        db_instance = False

        # 调用 novaclient.servers.list method 获取 servers 的列表
        servers = self._nova.servers.list()
        server_names = []
        for server in servers:
            server_names.append(server.name)
            if server.name == DB_INSTANCE_NAME:
                db_instance = server

        if not db_instance:
            # Create the mysql server
            db_script_path = path.join(path.curdir, 'scripts/db_server.txt')
            db_script = open(db_script_path, 'r').read()
            db_script = db_script.format(DB_NAME, DB_USER, DB_PASS)
            # 经过 nova client.servers.create method 建立一个 server
            # 这里由于但愿建立 server 并对其进行预设置, 因此使用了 userdata 参数
            # userdata 参数会接收一个 script 文件, 并在 server 第一次启动的时候执行
            db_instance = self._nova.servers.create(
                # FIXME(Fan Guiju): Using the params `block_device_mapping` to attach the volume.
                DB_INSTANCE_NAME,
                image.id,
                flavor_id,
                key_name=KEYPAIR_NAME,
                userdata=db_script)
            # 经过 novaclient.server.get method 和 server_id 来获取单个 server 的详细信息
            if not self._nova.server.get(db_instance.id):
                self._wait_for_done(objs=self._nova.servers,
                                    target_obj_name=DB_INSTANCE_NAME)
        # Attach the mysql-vol to mysql server, device type is `vd`.
        # 经过 cinderclient.volumes.attach method 挂在一个 volume 到 server 上
        # mountpoint 参数执行了挂载到 server 的设备路径, e.g. /dev/vdb
        self._cinder.volumes.attach(volume=volume,
                                    instance_uuid=db_instance.id,
                                    mountpoint=MOUNT_POINT)
        time.sleep(5)

        if BLOG_INSTANCE_NAME not in server_names:
            # Create the wordpress blog server
            # Nova-Network
            db_instance_ip = self._nova.servers.\
                get(db_instance.id).networks['private'][0]
            blog_script_path = path.join(path.curdir, 'scripts/blog_server.txt')
            blog_script = open(blog_script_path, 'r').read()
            blog_script = blog_script.format(DB_NAME,
                                             DB_USER,
                                             DB_PASS,
                                             db_instance_ip)
            self._nova.servers.create(BLOG_INSTANCE_NAME,
                                      image.id,
                                      flavor_id,
                                      key_name=KEYPAIR_NAME,
                                      userdata=blog_script)
            self._wait_for_done(objs=self._nova.servers,
                                target_obj_name=BLOG_INSTANCE_NAME)

        servers = self._nova.servers.list(search_opts={'all_tenants': True})
        return servers


def main():
    """FIXME(Fan Guiju): Operation manual."""
    os.environ['LANG'] = 'en_US.UTF8'

    deploy = AutoDep(auth_url=AUTH_URL,
                     username=USERNAME,
                     password=PASSWORD,
                     tenant_name=PROJECT_NAME)
    image = deploy.upload_image_to_glance()
    volume = deploy.create_volume()
    deploy.nova_boot(image, volume)

if __name__ == '__main__':
    main()

最后

上面给出了一个自动化运行 OpenStack Project 功能模块的脚本, 但实际上, 咱们可以使用 OpenStackClients 进行更加复杂的工做, 例如: 自定义一个新的 OpenStack Project, 并使之与 OpenStack 的原生 Project 进行互动, 这才是真正意义上的二次开发.