Nova AZ与Cinder AZ同步的问题

1.背景

在使用OpenStack云平台时，有时因业务架构须要拆分不一样的Availability Zone可用域，包括计算Nova和存储Cinder，并指望能够将两个AZ对应起来以便起到隔离的效果。目前多数OpenStack平台或多或少采用了Ceph做为Cinder存储后端（以及Glance、Swift），并在建立虚拟机时选择从镜像建立块存储。咱们但愿能够经过建立不一样的Cinder AZ配置不一样的Ceph pool，每一个pool关联不一样的osd，已便达到不一样用户建立的虚拟机操做系统磁盘在Ceph集群中是彻底隔离的。

2.问题

首先咱们启动多个cinder-volume实例，配置不一样的storage_availalibility_zone，指望效果是选择虚拟机的AZ时能够匹配Cinder里的AZ，例如：

storage_availability_zone=AZ1

然而实际状况并不理想，在建立虚拟机时，选择了Nova的AZ（好比AZ1），建立出来的卷却在Cinder的nova可用域里，由于Cinder的默认域是nova，最终结果并无匹配上。实际上，nova在调用cinder的时候并未把虚拟机实例的availalibility_zone的值传过去。

3.解决

查看源码/usr/lib/python2.7/site-packages/nova/conf/cinder.py中的设定，发现一个关键参数cross_az_attach，默认值为True，这意味着虚拟机的磁盘能够跨域绑定。

cfg.BoolOpt('cross_az_attach',
            default=True,
            help="""
Allow attach between instance and volume in different availability zones.

If False, volumes attached to an instance must be in the same availability
zone in Cinder as the instance availability zone in Nova.
This also means care should be taken when booting an instance from a volume
where source is not "volume" because Nova will attempt to create a volume using
the same availability zone as what is assigned to the instance.
If that AZ is not in Cinder (or allow_availability_zone_fallback=False in
cinder.conf), the volume create request will fail and the instance will fail
the build request.
By default there is no availability zone restriction on volume attach.
"""),
]

顺藤摸瓜，继续查看/usr/lib/python2.7/site-packages/nova/virt/block_device.py，发现若是cross_az_attach为True，则传给Cinder的availability_zone为空！！！若是cross_az_attach为False，那么nova会给Cinder传递实例的availability_zone。

def _get_volume_create_az_value(instance):
    """Determine az to use when creating a volume

    Uses the cinder.cross_az_attach config option to determine the availability
    zone value to use when creating a volume.

    :param nova.objects.Instance instance: The instance for which the volume
        will be created and attached.
    :returns: The availability_zone value to pass to volume_api.create
    """
    # If we're allowed to attach a volume in any AZ to an instance in any AZ,
    # then we don't care what AZ the volume is in so don't specify anything.
    if CONF.cinder.cross_az_attach:
        return None
    # Else the volume has to be in the same AZ as the instance otherwise we
    # fail. If the AZ is not in Cinder the volume create will fail. But on the
    # other hand if the volume AZ and instance AZ don't match and
    # cross_az_attach is False, then volume_api.check_attach will fail too, so
    # we can't really win. :)
    # TODO(mriedem): It would be better from a UX perspective if we could do
    # some validation in the API layer such that if we know we're going to
    # specify the AZ when creating the volume and that AZ is not in Cinder, we
    # could fail the boot from volume request early with a 400 rather than
    # fail to build the instance on the compute node which results in a
    # NoValidHost error.
    return instance.availability_zone

一切都清楚了，接下来配置nova.conf文件，在Cinder部分添加参数

cross_az_attach = False

再次建立虚拟机，发现Nova的AZ和Cinder的AZ对应上啦，解决！

4.More..

在Cinder中还有个参数是allow_availability_zone_fallback，目的是为了防止建立虚拟机时的Nova AZ在Cinder中不存在时不报错，而是使用Cinder中default_availability_zone或者storage_availability_zone进行建立。代码在/usr/lib/python2.7/site-packages/cinder/volume/flows/api/create_volume.py中

if availability_zone not in self.availability_zones:
            if CONF.allow_availability_zone_fallback:
                original_az = availability_zone
                availability_zone = (
                    CONF.default_availability_zone or
                    CONF.storage_availability_zone)
                LOG.warning(_LW("Availability zone '%(s_az)s' "
                                "not found, falling back to "
                                "'%(s_fallback_az)s'."),
                            {'s_az': original_az,
                             's_fallback_az': availability_zone})
            else:
                msg = _("Availability zone '%(s_az)s' is invalid.")
                msg = msg % {'s_az': availability_zone}
                raise exception.InvalidInput(reason=msg)

可是目测这个参数和cross_az_attach有冲突，由于使用cross_az_attach的状况下选择一个Cinder里没有的可是Nova中含有的AZ，建立结果仍是失败的，最终会以cross_az_attach做为判断。

5.参考文献

【1】https://www.mirantis.com/blog...
【2】https://wenku.baidu.com/view/...