Docker私有仓库Harbor介绍和部署记录

时间  2019-11-11
标签 docker 私有 仓库 harbor 介绍 部署 记录 栏目 Docker 繁體版
原文   原文链接

Docker容器应用的开发和运行离不开可靠的镜像管理，虽然Docker官方也提供了公共的镜像仓库，可是从安全和效率等方面考虑，部署咱们私有环境内的Registry
也是很是必要的。以前介绍了Docker私有仓库Registry，这里介绍另外一款企业级Docker镜像仓库Harbor的部署和使用，在Kubernetes集群中，推荐使用Harbor仓库环境。html

1、Harbor仓库介绍前端

咱们在平常Docker容器使用和管理过程当中，渐渐发现部署企业私有仓库每每是颇有必要的, 它能够帮助你管理企业的一些敏感镜像, 同时因为Docker Hub的下载速度和GFW的缘由, 每每须要将一些没法直接下载的镜像导入本地私有仓库. 而Harbor就是部署企业私有仓库的一个不二之选。Harbor是由VMware公司开源的企业级的Docker Registry管理项目，Harbor主要提供Dcoker Registry管理UI，提供的功能包括：基于角色访问的控制权限管理(RBAC)、AD/LDAP集成、日志审核、管理界面、自我注册、镜像复制和中文支持等。Harbor的目标是帮助用户迅速搭建一个企业级的Docker registry服务。它以Docker公司开源的registry为基础，额外提供了以下功能:
->  基于角色的访问控制(Role Based Access Control)
->  基于策略的镜像复制(Policy based image replication)
->  镜像的漏洞扫描(Vulnerability Scanning)
->  AD/LDAP集成(LDAP/AD support)
->  镜像的删除和空间清理(Image deletion & garbage collection)
->  友好的管理UI(Graphical user portal)
->  审计日志(Audit logging)
->  RESTful API
->  部署简单(Easy deployment)node

Harbor的全部组件都在Dcoker中部署，因此Harbor可以使用Docker Compose快速部署。须要特别注意：因为Harbor是基于Docker Registry V2版本，因此docker必须大于等于1.10.0版本，docker-compose必需要大于1.6.0版本！python

2、Harbor仓库结构mysql

Harbor的每一个组件都是以Docker容器的形式构建的，可使用Docker Compose来进行部署。若是环境中使用了kubernetes，Harbor也提供了kubernetes的配置文件。Harbor大概须要如下几个容器组成：ui(Harbor的核心服务)、log(运行着rsyslog的容器，进行日志收集)、mysql(由官方mysql镜像构成的数据库容器)、Nginx(使用Nginx作反向代理)、registry(官方的Docker registry)、adminserver(Harbor的配置数据管理器)、jobservice(Harbor的任务管理服务)、redis(用于存储session)。linux

Harbor是一个用于存储和分发Docker镜像的企业级Registry服务器，总体架构仍是很清晰的。下面借用了网上的架构图:nginx

===========================================================================git

Harbor依赖的外部组件
-> Nginx(即Proxy代理层): Nginx前端代理，主要用于分发前端页面ui访问和镜像上传和下载流量; Harbor的registry,UI,token等服务，经过一个前置的反向代理统一接收浏览器、Docker客户端的请求，并将请求转发给后端不一样的服务。
-> Registry v2: 镜像仓库，负责存储镜像文件; Docker官方镜像仓库, 负责储存Docker镜像，并处理docker push/pull命令。因为咱们要对用户进行访问控制，即不一样用户对Docker image有不一样的读写权限，Registry会指向一个token服务，强制用户的每次docker pull/push请求都要携带一个合法的token, Registry会经过公钥对token进行解密验证。
-> Database(MySQL或Postgresql)：为core services提供数据库服务，负责储存用户权限、审计日志、Docker image分组信息等数据。github

Harbor自有组件
-> Core services(Admin Server): 这是Harbor的核心功能，主要提供如下服务： 
    -> UI：提供图形化界面，帮助用户管理registry上的镜像（image）, 并对用户进行受权。
    -> webhook：为了及时获取registry 上image状态变化的状况， 在Registry上配置webhook，把状态变化传递给UI模块。
    -> Auth服务：负责根据用户权限给每一个docker push/pull命令签发token. Docker 客户端向Regiøstry服务发起的请求,若是不包含token，会被重定向到这里，得到token后再从新向Registry进行请求。
    -> API: 提供Harbor RESTful API
-> Replication Job Service：提供多个 Harbor 实例之间的镜像同步功能。
-> Log collector：为了帮助监控Harbor运行，负责收集其余组件的log，供往后进行分析。web

再来仔细看下Harbor主要组件和数据流走向：

 

-> proxy，它是一个nginx前端代理，主要是分发前端页面ui访问和镜像上传和下载流量，上图中经过深蓝色先标识；
-> ui提供了一个web管理页面，固然还包括了一个前端页面和后端API，底层使用mysql数据库；
-> registry是镜像仓库，负责存储镜像文件，当镜像上传完毕后经过hook通知ui建立repository，上图经过红色线标识，固然registry的token认证也是经过ui组件完成；
-> adminserver是系统的配置管理中心附带检查存储用量，ui和jobserver启动时候回须要加载adminserver的配置，经过灰色线标识；
-> jobsevice是负责镜像复制工做的，他和registry通讯，从一个registry pull镜像而后push到另外一个registry，并记录job_log，上图经过紫色线标识；
-> log是日志汇总组件，经过docker的log-driver把日志汇总到一块儿，经过浅蓝色线条标识。

Harbor的误区
误区一: Harbor是负责存储容器镜像的 (Harbor是镜像仓库，那么它就应当是存储镜像的)
其实关于镜像的存储，Harbor使用的是官方的docker registry服务去完成，至于registry是用本地存储或者s3都是能够的，Harbor的功能是在此之上提供用户权限管理、镜像复制等功能，提升使用的registry的效率。

误区二：Harbor镜像复制是存储直接复制 (镜像的复制，不少人觉得应该是镜像分层文件的直接拷贝)
其实Harbor镜像复制采用了一个更加通用、高屋建瓴的作法，经过docker registry 的API去拷贝，这不是省事，这种作法屏蔽了繁琐的底层文件操做、不只能够利用现有docker registry功能没必要重复造轮子，并且能够解决冲突和一致性的问题。

Harbor的部署
这里不建议使用kubernetes来部署, 缘由是镜像仓库很是重要, 尽可能保证部署和维护的简洁性, 所以这里直接使用compose的方式进行部署。官方提供3种部署Harbor的方式:
1）在线安装: 从Docker Hub下载Harbor的镜像来安装, 因为Docker Hub比较慢, 建议Docker配置好加速器。
2）离线安装: 这种方式应对与部署主机没联网的状况使用。须要提早下载离线安装包: harbor-offline-installer-.tgz 到本地
3）OVA安装: 这个主要用vCentor环境是使用

后面部署时会为Docker配置镜像加速器, 所以会采用在线部署的方式, 部署步骤以下：
-> 下载Harbor最新的在线安装包
-> 配置Harbor (harbor.cfg)
-> 运行install.sh来安装和启动Harbor
-> Harbor的日志路径:/var/log/harbor

Harbor仓库部署的官方要求的最小系统配置
-> 2个cpu
-> 4g内存
-> 40g硬盘，由于是存储镜像的因此推荐硬盘大点。

参考
Harbor GitHub
Harbor 安装手册
Harbor 用户手册

3、Harbor仓库环境部署记录

3.1) 环境要求和准备工做

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Harbor以容器的形式进行部署, 所以能够被部署到任何支持Docker的Linux发行版, 要使用Harbor，须要安装docker和docker-compose编排工具，而且具有以下环境:
Python2.7+
Docker Engine 1.10+
Docker Compose 1.6.0+
 
这里测试环境部署到Centos7.5机器上，以下：
[root@harbor-node ~] # cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)
 
[root@harbor-node ~] # setenforce 0
[root@harbor-node ~] # cat /etc/sysconfig/selinux
...........
SELINUX=disabled
 
[root@harbor-node ~] # systemctl stop firewalld
[root@harbor-node ~] # systemctl disable firewalld
[root@harbor-node ~] # firewall-cmd --state
not running
 
centos7自带的python版本就是2.7.5
[root@harbor-node ~] # python --version
Python 2.7.5

3.2) 安装Docker

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
更新yum包
[root@harbor-node ~] # yum update
   
卸载旧版本 Docker
[root@harbor-node ~] # yum remove docker docker-common docker-selinux docker-engine
   
安装软件包
[root@harbor-node ~] # yum install -y yum-utils device-mapper-persistent-data lvm2
   
添加 Docker yum源
[root@harbor-node ~] # yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Loaded plugins: fastestmirror
adding repo from: https: //download .docker.com /linux/centos/docker-ce .repo
grabbing  file  https: //download .docker.com /linux/centos/docker-ce .repo to  /etc/yum .repos.d /docker-ce .repo
repo saved to  /etc/yum .repos.d /docker-ce .repo
   
安装 Docker （直接yum按照docker-ce便可）
随着Docker的不断流行与发展，docker公司（或称为组织）也开启了商业化之路，Docker 从 17.03版本以后分为 CE（Community Edition） 和 EE（Enterprise Edition）:
1) Docker EE由公司支持，可在通过认证的操做系统和云提供商中使用，并可运行来自Docker Store的、通过认证的容器和插件。
2) Docker CE是免费的Docker产品的新名称，Docker CE包含了完整的Docker平台，很是适合开发人员和运维团队构建容器APP。
    事实上，Docker CE 17.03，可理解为Docker 1.13.1的Bug修复版本。所以，从Docker 1.13升级到Docker CE 17.03风险相对是较小的。
   
[root@harbor-node ~] # yum -y install docker-ce
   
启动 Docker
[root@harbor-node ~] # systemctl start docker
[root@harbor-node ~] # systemctl enable docker
[root@harbor-node ~] # systemctl status docker
● docker.service - Docker Application Container Engine
    Loaded: loaded ( /usr/lib/systemd/system/docker .service; enabled; vendor preset: disabled)
    Active: active (running) since Sun 2019-05-26 22:15:34 CST; 27min ago
      Docs: https: //docs .docker.com
  Main PID: 15260 (dockerd)
    CGroup:  /system .slice /docker .service
            └─15260  /usr/bin/dockerd  -H fd: //  --containerd= /run/containerd/containerd .sock
 
May 26 22:15:33 harbor-node dockerd[15260]:  time = "2019-05-26T22:15:33.570826805+08:00"  level=info msg= "pickfirstBalancer: HandleSubConnStateChange: 0xc420175490, READY"  module=grpc
May 26 22:15:33 harbor-node dockerd[15260]:  time = "2019-05-26T22:15:33.570899114+08:00"  level=info msg= "pickfirstBalancer: HandleSubConnStateChange: 0xc42006de20, READY"  module=grpc
May 26 22:15:33 harbor-node dockerd[15260]:  time = "2019-05-26T22:15:33.665440742+08:00"  level=info msg= "Graph migration to content-addressability took 0.00 seconds"
May 26 22:15:33 harbor-node dockerd[15260]:  time = "2019-05-26T22:15:33.666111994+08:00"  level=info msg= "Loading containers: start."
May 26 22:15:33 harbor-node dockerd[15260]:  time = "2019-05-26T22:15:33.913110547+08:00"  level=info msg= "Default bridge (docker0) is assigned with an IP address 172.17.0.0/16... IP address"
May 26 22:15:34 harbor-node dockerd[15260]:  time = "2019-05-26T22:15:34.088687650+08:00"  level=info msg= "Loading containers: done."
May 26 22:15:34 harbor-node dockerd[15260]:  time = "2019-05-26T22:15:34.128885651+08:00"  level=info msg= "Docker daemon"  commit=481bc77 graphdriver(s)=overlay2 version=18.09.6
May 26 22:15:34 harbor-node dockerd[15260]:  time = "2019-05-26T22:15:34.129073367+08:00"  level=info msg= "Daemon has completed initialization"
May 26 22:15:34 harbor-node dockerd[15260]:  time = "2019-05-26T22:15:34.223886566+08:00"  level=info msg= "API listen on /var/run/docker.sock"
May 26 22:15:34 harbor-node systemd[1]: Started Docker Application Container Engine.
Hint: Some lines were ellipsized, use -l to show  in  full.
   
查看 Docker 版本号
[root@harbor-node ~] # docker --version
Docker version 18.09.6, build 481bc77156
  
[root@harbor-node ~] # docker version
Client:
  Version:           18.09.6
  API version:       1.39
  Go version:        go1.10.8
  Git commit:        481bc77156
  Built:             Sat May  4 02:34:58 2019
  OS /Arch :           linux /amd64
  Experimental:       false
  
Server: Docker Engine - Community
  Engine:
   Version:          18.09.6
   API version:      1.39 (minimum version 1.12)
   Go version:       go1.10.8
   Git commit:       481bc77
   Built:            Sat May  4 02:02:43 2019
   OS /Arch :          linux /amd64
   Experimental:      false

3.3) 安装Docker Compose

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Docker Compose 是 Docker 容器进行编排的工具，定义和运行多容器的应用，能够一条命令启动多个容器。
 
安装 epel-release
[root@harbor-node ~] # yum install epel-release
   
安装 python-pip
[root@harbor-node ~] # yum install -y python-pip
   
安装 docker-compose
[root@harbor-node ~] # pip install docker-compose
.........
Successfully installed asn1crypto-0.24.0 bcrypt-3.1.6 cached-property-1.5.1 certifi-2019.3.9 cffi-1.12.3 chardet-3.0.4 cryptography-2.6.1 docker-3.7.2 docker-compose-1.24.0 docker-pycreds-0.4.0 dockerpty-0.4.1 docopt-0.6.2 enum34-1.1.6 functools32-3.2.3.post2 idna-2.7 jsonschema-2.6.0 paramiko-2.4.2 pyasn1-0.4.5 pycparser-2.19 pynacl-1.3.0 requests-2.20.1 texttable-0.9.1 urllib3-1.24.3 websocket-client-0.56.0
You are using pip version 8.1.2, however version 19.1.1 is available.
You should consider upgrading via the  'pip install --upgrade pip'  command .
   
查看 docker-compose 版本号
[root@harbor-node ~] # docker-compose -version
docker-compose version 1.24.0, build 0aa5906
  
[root@harbor-node ~] # pip freeze | grep compose
You are using pip version 8.1.2, however version 19.1.1 is available.
You should consider upgrading via the  'pip install --upgrade pip'  command .
docker-compose==1.24.0
   
安装 git
[root@harbor-node ~] # yum install git

3.4) 为Docker配置加速器, 方便经过国内镜像服务器快速拉取Docker Hub提供的镜像

1
2
3
4
5
[root@harbor-node ~] # mkdir -p /etc/docker
[root@harbor-node ~] # cat /etc/docker/daemon.json
{
   "registry-mirrors" : [ "https://v5d7kh0f.mirror.aliyuncs.com" ]
}

3.5) 下载Harbor安装包，配置Harbor
到Harbor的GitHub仓库的Release页面, 下载最新的在线安装包(若是下载不了, 请从这里下载百度网盘Harbor   提取码：xu3j）

这里分为在线和离线的版本，我下载的是1.8.0在线的版本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
[root@harbor-node ~] # ll harbor-online-installer-v1.8.0.tgz
-rw-r--r-- 1 root root 7954 May 26 22:45 harbor-online-installer-v1.8.0.tgz
 
[root@harbor-node ~] # tar -zvxf harbor-online-installer-v1.8.0.tgz
[root@harbor-node ~] # cd harbor
[root@harbor-node harbor] # ls
harbor.yml   install .sh  LICENSE  prepare
 
下载下来以后解压缩，目录下会有harbor.yaml (新版本是.yaml文件，以前版本是.conf 或者 .cfg文件)，就是Harbor的配置文件了。
[root@harbor-node harbor] # cp harbor.yml harbor.yml.bak
[root@harbor-node harbor] # vim harbor.yml
[root@harbor-node harbor] # cat harbor.yml |grep -v "#"|grep -v "^$"
hostname : 172.16.60.213
http:
   port: 80
harbor_admin_password: kevin@BO123             
database:
   password: root123
data_volume:  /data
clair:
   updaters_interval: 12
   http_proxy:
   https_proxy:
   no_proxy: 127.0.0.1,localhost,core,registry
jobservice:
   max_job_workers: 10
chart:
   absolute_url: disabled
log:
   level: info
   rotate_count: 50
   rotate_size: 200M
   location:  /var/log/harbor
_version: 1.8.0
 
配置解释
hostname : 修改为Harbao部署机自身的ip地址
db_password: 这是postgresql数据库root密码
harbor_admin_password: harbor初始管理员密码为Harbor12345, 这里最好修改为本身的密码，默认密码至少8位，最好是大小写、数字和特殊字符。
 
配置完Harbor以后，接着进行安装启动Harbor，Harbor目录下有一个 install .sh, 执行它来进行安装
[root@harbor-node harbor] # ./install.sh
...........
...........
✔ ----Harbor has been installed and started successfully.----
 
Now you should be able to visit the admin portal at http: //172 .16.60.213.
For  more  details, please visit https: //github .com /goharbor/harbor  .
 
安装完成后，会发现解压目录harbor下面多了一个docker-compose.yml文件，里面包含了harbor依赖的镜像和对应容器建立的信息
 
查看harbor对应容器信息（还能够执行 "docker images" 和 "docker ps" 查看harbor的镜像和容器状况）
[root@harbor-node harbor] # docker-compose ps       #"注意docker-compose"命令只能在当前harbor目录下使用（由于该目录下有harbor配置文件）
       Name                     Command                  State                 Ports         
---------------------------------------------------------------------------------------------
harbor-core          /harbor/start .sh                 Up (healthy)                           
harbor-db            /entrypoint .sh postgres          Up (healthy)   5432 /tcp                
harbor-jobservice    /harbor/start .sh                 Up                                     
harbor-log           /bin/sh  -c  /usr/local/bin/  ...   Up (healthy)   127.0.0.1:1514->10514 /tcp
harbor-portal       nginx -g daemon off;             Up (healthy)   80 /tcp                  
nginx               nginx -g daemon off;             Up (healthy)   0.0.0.0:80->80 /tcp      
redis               docker-entrypoint.sh redis ...   Up             6379 /tcp                
registry             /entrypoint .sh  /etc/regist  ...   Up (healthy)   5000 /tcp                
registryctl          /harbor/start .sh                 Up (healthy) 
 
而后就能够访问harbor了,访问地址为：http: //172 .16.60.213
用户名为admin，密码为配置文件中定义的 "kevin@BO123"

Harbor 服务的关闭和启动

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
1) Harbor的日志路径：var /log/harbor
[root@harbor-node harbor] # cat harbor.yml|grep log
log:
   # Log files are rotated log_rotate_count times before being removed. If count is 0, old versions are removed rather than rotated.
   # Log files are rotated only if they grow bigger than log_rotate_size bytes. If size is followed by k, the size is assumed to be in kilobytes.
   # The directory on your host that store log
   location:  /var/log/harbor
    
[root@harbor-node harbor] # ls /var/log/harbor/
core.log  jobservice.log  portal.log  postgresql.log  proxy.log  redis.log  registryctl.log  registry.log
    
2) 中止和关闭harbor命令:  "docker-compose down -v"
[root@harbor-node harbor] # docker-compose down -v
Stopping nginx             ...  done
Stopping harbor-jobservice ...  done
Stopping harbor-portal     ...  done
Stopping harbor-core       ...  done
Stopping redis             ...  done
Stopping registryctl       ...  done
Stopping registry          ...  done
Stopping harbor-db         ...  done
Stopping harbor-log        ...  done
Removing nginx             ...  done
Removing harbor-jobservice ...  done
Removing harbor-portal     ...  done
Removing harbor-core       ...  done
Removing redis             ...  done
Removing registryctl       ...  done
Removing registry          ...  done
Removing harbor-db         ...  done
Removing harbor-log        ...  done
Removing network harbor_harbor
    
[root@harbor-node harbor] # docker-compose ps
Name   Command   State   Ports
------------------------------
    
[root@harbor-node harbor] # docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    
能够修改harbor配置文件，好比这里修改harbor的web登陆端口，由80端口修改成8080端口
[root@harbor-node harbor] # vim harbor.yml
.........
http:
   # port for http, default is 80. If https enabled, this port will redirect to https port
   port: 8080
    
而后将harbor修改的配置更新到 docker-compose.yml 文件
[root@harbor-node harbor] # ./prepare
prepare base  dir  is  set  to  /root/harbor
Clearing the configuration  file :  /config/log/logrotate .conf
Clearing the configuration  file :  /config/nginx/nginx .conf
Clearing the configuration  file :  /config/core/env
Clearing the configuration  file :  /config/core/app .conf
Clearing the configuration  file :  /config/registry/config .yml
Clearing the configuration  file :  /config/registry/root .crt
Clearing the configuration  file :  /config/registryctl/env
Clearing the configuration  file :  /config/registryctl/config .yml
Clearing the configuration  file :  /config/db/env
Clearing the configuration  file :  /config/jobservice/env
Clearing the configuration  file :  /config/jobservice/config .yml
Generated configuration  file :  /config/log/logrotate .conf
Generated configuration  file :  /config/nginx/nginx .conf
Generated configuration  file :  /config/core/env
Generated configuration  file :  /config/core/app .conf
Generated configuration  file :  /config/registry/config .yml
Generated configuration  file :  /config/registryctl/env
Generated configuration  file :  /config/db/env
Generated configuration  file :  /config/jobservice/env
Generated configuration  file :  /config/jobservice/config .yml
loaded secret from  file :  /secret/keys/secretkey
Generated configuration  file :  /compose_location/docker-compose .yml
Clean up the input  dir
    
防止容器进程没有权限读取生成的配置
[root@harbor-node harbor] # ll common/
total 0
drwxr-xr-x 9 root root 105 May 26 23:10 config
    
[root@harbor-node harbor] # chmod -R 777 common
    
[root@harbor-node harbor] # ll common/
total 0
drwxrwxrwx 9 root root 105 May 27 00:41 config
    
=======================================================================================================================================
特别注意： 这里的common权限若是设置过小，可能会致使harbor启动后，报下面的错
发现启动harbor后，如上有些服务，如nginx，registry状态一直是 "Restarting" ，这时须要查看日志：
[root@harbor-node harbor] # tail -100 /var/log/harbor/registry.log |grep error
May 27 01:01:18 172.19.0.1 registry[2960]: configuration error:  open  /etc/registry/config .yml: permission denied
May 27 01:01:21 172.19.0.1 registry[2960]: configuration error:  open  /etc/registry/config .yml: permission denied
May 27 01:01:23 172.19.0.1 registry[2960]: configuration error:  open  /etc/registry/config .yml: permission denied
May 27 01:01:27 172.19.0.1 registry[2960]: configuration error:  open  /etc/registry/config .yml: permission denied
=======================================================================================================================================
    
最后再次启动 harbor
[root@harbor-node harbor] # docker-compose up -d
Creating network  "harbor_harbor"  with the default driver
Creating harbor-log ...  done
Creating registryctl ...  done
Creating registry    ...  done
Creating redis       ...  done
Creating harbor-db   ...  done
Creating harbor-core ...  done
Creating harbor-jobservice ...  done
Creating harbor-portal     ...  done
Creating nginx             ...  done
    
查看服务
[root@harbor-node harbor] # docker-compose  ps
       Name                     Command                       State                     Ports      
------------------------------------------------------------------------------------------------------
harbor-core          /harbor/start .sh                 Up (health: starting)                        
harbor-db            /entrypoint .sh postgres          Up (health: starting)   5432 /tcp             
harbor-jobservice    /harbor/start .sh                 Up                                           
harbor-log           /bin/sh  -c  /usr/local/bin/  ...   Up (health: starting)   127.0.0.1:1514->10514 /tcp
harbor-portal       nginx -g daemon off;             Up (health: starting)   80 /tcp               
nginx               nginx -g daemon off;             Up (health: starting)   0.0.0.0:8080->80 /tcp 
redis               docker-entrypoint.sh redis ...   Up                      6379 /tcp             
registry             /entrypoint .sh  /etc/regist  ...   Up (health: starting)   5000 /tcp             
registryctl          /harbor/start .sh                 Up (health: starting) 
    
而后访问http: //172 .16.60.213:8080，便可访问harbor的web界面
 
###############################################################################################
要是想修改harbor的登录用户密码，则最好在harbor web界面里直接修改，这样是最保险的！
 
若是是想经过修改harbar.yaml文件来重置harbor用户密码，则不能单纯的修改后就执行 "./prepare" 和重启docker-compose，这样是不能修改harbor用户密码的！
这时由于harbor在这里用的是postgresql数据库，以pdkdf2算法保存的秘文密码！须要先进入 "harbor-db" 容器内部,执行相关postgresql命令行。
并且postgresql的用户密码采用的是pbkdf2算法，须要提早计算好新密码的密钥值，pdkdf2算法须要 "盐值" ， "迭代次数" 和密钥长度int型等，特别麻烦!!
 
因此若是忘记harbor的web密码或者是admin密码须要重置，而且对于postgresql数据库 或者 pbkdf2算法操做不熟悉的话，建议删除data源数据的database，从新部署！
作法以下：
# docker-compose down -v
# rm -rf /data/database
# vim harbor.yaml           #在这里重置或修改密码
# docker-compose up -d
 
这样就可使用重置后的新密码登录harbor web界面了，可是以前建立的用户和项目就都删除了。
这种状况最好适用于刚建立初期。
 
###############################################################################################
docker-compose up -d           # 后台启动，若是容器不存在根据镜像自动建立
docker-compose down - v         # 中止容器并删除容器
docker-compose start           # 启动容器，容器不存在就没法启动，不会自动建立镜像
docker-compose stop            # 中止容器
   
须要注意：
其实上面是中止docker-compose.yml中定义的全部容器，默认状况下docker-compose就是操做同目录下的docker-compose.yml文件。
若是使用其余yml文件，可使用-f本身指定。

->  登陆Harbor web界面，在"系统管理"->"配置管理"->"认证模式"->"容许自注册"这一项的对勾去掉，则登陆的时候就不会有"用户注册"这一功能了。
->  能够在"配置管理"这一项进行认证模式，邮箱，标签等设置。

3.6） 使用Harbor私有仓库　　

3.6.1）harbor的login登陆

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
1）在harbor远程别的客户机上登陆
[root@docker-client ~] # docker login 172.16.60.213
Username: admin
Password:
Error response from daemon: Get https: //172 .16.60.213 /v1/users/ : dial tcp 172.16.60.213:443: connect: connection refused
  
在进行harbor登陆或上传代码时，会报出上面错误！
这是由于docker1.3.2版本开始默认docker registry使用的是https，而Harbor默认设置的是http方式而不是https，因此当执行用docker login、pull、push等
命令操做非https的docker regsitry的时就会报错。
  
解决办法：
以下，在 /etc/docker/daemon .json文件里添加 "insecure-registries" 配置
[root@docker-client ~] # vim /etc/docker/daemon.json
{
     "insecure-registries" : [
         "172.16.60.213"
       ]
}
  
而后重启docker服务
[root@docker-client ~] # systemctl restart docker
  
接着再次验证harbor登陆，发现就能登陆上了
[root@docker-client ~] # docker login 172.16.60.213      #或者直接执行"docker login -u admin -p kevin@BO123 172.16.60.213"命令登录
Username: admin
Password:
Login Succeeded
[root@docker-client ~] #
  
2）若是是在harbor本机登陆，出现上面的报错：
[root@harbor-node harbor] # docker login 172.16.60.213
Username: admin
Password:
Error response from daemon: Get https: //172 .16.60.213 /v1/users/ : dial tcp 172.16.60.213:443: connect: connection refused
  
解决办法：
在 /etc/docker/daemon .json 文件里添加 "insecure-registries" 配置 （第一行是以前添加的docker加速配置），注意两行之间有一个 "," 逗号隔开
[root@harbor-node harbor] # vim /etc/docker/daemon.json
{
   "registry-mirrors" : [ "https://v5d7kh0f.mirror.aliyuncs.com" ],
   "insecure-registries" : [ "172.16.60.213" ]
}
  
修改事后重启docker, 而后重启Harbor服务
[root@harbor-node harbor] # systemctl restart docker
[root@harbor-node harbor] # docker-compose stop
[root@harbor-node harbor] # docker-compose start
  
而后再测试再harbor本机登陆
[root@harbor-node harbor] # docker login 172.16.60.213
Username: admin
Password:
WARNING! Your password will be stored unencrypted  in  /root/ .docker /config .json.
Configure a credential helper to remove this warning. See
https: //docs .docker.com /engine/reference/commandline/login/ #credentials-store
  
Login Succeeded
[root@harbor-node harbor] #
  
登陆的帐号信息都保存到 /root/ .docker /config .json文件里了
[root@harbor-node harbor] # cat /root/.docker/config.json
{
         "auths" : {
                 "172.16.60.213" : {
                         "auth" :  "YWRtaW46a2V2aW5AQk8xOTg3"
                 }
         },
         "HttpHeaders" : {
                 "User-Agent" :  "Docker-Client/18.09.6 (linux)"
         }
  
只要 /root/ .docker /config .json里的信息不删除，后续再次登陆的时候，就不用输入用户名和密码了
[root@harbor-node ~] # docker login 172.16.60.213
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted  in  /root/ .docker /config .json.
Configure a credential helper to remove this warning. See
https: //docs .docker.com /engine/reference/commandline/login/ #credentials-store
  
Login Succeeded

===============================================================
注意事项总结：
harbor支持http和https，但若是使用http的话，在拉取镜像的时候，会抛出仓库不受信任的异常。
须要在全部的docker客户端的docker配置文件/etc/docker/daemon.json中添加以下配置:

1
2
3
{
     "insecure-registries" : [ "https://*.*.*.*" ]
}

若是使用自签名的https证书，仍然会提示证书不受信任的问题。须要将自签名的ca证书发送到全部的docker客户端的指定目录。
关于使用自签名证书配置harbor的具体过程能够参考: https://github.com/goharbor/harbor/blob/master/docs/configure_https.md

3.6.2）harbor仓库的使用

1
2
3
4
5
6
7
8
#镜像打标签的命令
# docker tag 镜像名:标签 私服地址/仓库项目名/镜像名:标签
 
#推送到私服的命令
#docker push 私服地址/仓库项目名/镜像名:标签
 
#从私服拉取镜像的命令
#docker pull 私服地址/仓库项目名/镜像名:标签

首先在Harbor web界面里最好建立一个本身须要的"项目" (或者使用默认的"library"项目),项目公开和私有：
-  Public: 全部用户对于公开项目都有读权限,这种方式对于你想把一些仓库分享给其余人的时候,是很是方便的.
-  Private: 私有项目只能被有特定用户权限的人去访问。这种方式对于内部团队来讲共享也是比较方便的

好比建立一个公开项目"kevin_bo"，点击进去能够看到推送镜像的信息提示

而后就能够在Harbor服务器的终端命令行里进行镜像推送到Harbor仓库的操做了:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
在进行harbor镜像推送和拉取操做前，须要事先login登陆到harbor仓库里，这样才有harbor镜像的推送和拉取的权限！！
[root@harbor-node ~] # docker login 172.16.60.213
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted  in  /root/ .docker /config .json.
Configure a credential helper to remove this warning. See
https: //docs .docker.com /engine/reference/commandline/login/ #credentials-store
 
Login Succeeded
 
先查看本机有哪些镜像
[root@harbor-node ~] # docker images
REPOSITORY                    TAG                        IMAGE ID            CREATED             SIZE
goharbor /redis-photon          v1.8.0                     66d7402d2770        10 days ago         103MB
goharbor /harbor-registryctl    v1.8.0                     0ca3e2b624f5        10 days ago         96.2MB
goharbor /registry-photon       v2.7.1-patch-2819-v1.8.0   1e7d99ccba24        10 days ago         81.3MB
goharbor /nginx-photon          v1.8.0                     4a4b48b32ae4        10 days ago         36MB
goharbor /harbor-log            v1.8.0                     e718bdc405a3        10 days ago         81.5MB
goharbor /harbor-jobservice     v1.8.0                     d47940dd883f        10 days ago         118MB
goharbor /harbor-core           v1.8.0                     b07a1a4be17f        10 days ago         135MB
goharbor /harbor-portal         v1.8.0                     76298a1ef089        10 days ago         42.9MB
goharbor /harbor-db             v1.8.0                     d1e0b3df3e95        10 days ago         140MB
goharbor /prepare               v1.8.0                     769ca785dab0        10 days ago         139MB
 
好比推送其中的goharbor /redis-photon :v1.8.0镜像到Harbor仓库的 "kevin_bo" 项目里
[root@harbor-node ~] # docker tag goharbor/redis-photon:v1.8.0 172.16.60.213/kevin_bo/redis-photon:v1.0
[root@harbor-node ~] # docker push 172.16.60.213/kevin_bo/redis-photon:v1.0
The push refers to repository [172.16.60.213 /kevin_bo/redis-photon ]
8864c4b9ac3d: Pushed
420b26399278: Pushed
4433bcd802e7: Pushed
268091c30a67: Pushed
23d9f72a5270: Pushed
v1.0: digest: sha256:1e2ce8e6a852713d789c6315642d1483d1efdb4acee4699817810bef219ec93d size: 1366
 
查看本机的images，发现多了一个上面制做的镜像，就是原来goharbor /redis-photon :v1.8.0的tag，能够选择删除
[root@harbor-node ~] # docker rmi 172.16.60.213/kevin_bo/redis-photon:v1.0
Untagged: 172.16.60.213 /kevin_bo/redis-photon :v1.0
Untagged: 172.16.60.213 /kevin_bo/redis-photon @sha256:1e2ce8e6a852713d789c6315642d1483d1efdb4acee4699817810bef219ec93d
 
[root@harbor-node ~] # docker images
REPOSITORY                    TAG                        IMAGE ID            CREATED             SIZE
goharbor /redis-photon          v1.8.0                     66d7402d2770        10 days ago         103MB
goharbor /harbor-registryctl    v1.8.0                     0ca3e2b624f5        10 days ago         96.2MB
goharbor /registry-photon       v2.7.1-patch-2819-v1.8.0   1e7d99ccba24        10 days ago         81.3MB
goharbor /nginx-photon          v1.8.0                     4a4b48b32ae4        10 days ago         36MB
goharbor /harbor-log            v1.8.0                     e718bdc405a3        10 days ago         81.5MB
goharbor /harbor-jobservice     v1.8.0                     d47940dd883f        10 days ago         118MB
goharbor /harbor-core           v1.8.0                     b07a1a4be17f        10 days ago         135MB
goharbor /harbor-portal         v1.8.0                     76298a1ef089        10 days ago         42.9MB
goharbor /harbor-db             v1.8.0                     d1e0b3df3e95        10 days ago         140MB
goharbor /prepare               v1.8.0                     769ca785dab0        10 days ago         139MB
 
同理，推送其余镜像是一样的操做，好比再推送goharbor /harbor-core :v1.8.0到harbor仓库的 "kevin_bo" 项目里
[root@harbor-node ~] # docker tag goharbor/harbor-core:v1.8.0 172.16.60.213/kevin_bo/goharbor/harbor-core:v1.0
[root@harbor-node ~] # docker push 172.16.60.213/kevin_bo/goharbor/harbor-core:v1.0
The push refers to repository [172.16.60.213 /kevin_bo/goharbor/harbor-core ]
5385ffb8451e: Pushed
36e1cb2d6ffa: Pushed
452d238b3e48: Pushed
af3a6f89469a: Pushed
05bc5efb1724: Pushed
23d9f72a5270: Mounted from kevin_bo /redis-photon
v1.0: digest: sha256:7899f284617bb051180adf6c3aedd140a519d9092b8986dd9058d4dcec0d31de size: 1580
 
[root@harbor-node ~] # docker images
REPOSITORY                                    TAG                        IMAGE ID            CREATED             SIZE
goharbor /redis-photon                          v1.8.0                     66d7402d2770        10 days ago         103MB
goharbor /harbor-registryctl                    v1.8.0                     0ca3e2b624f5        10 days ago         96.2MB
goharbor /registry-photon                       v2.7.1-patch-2819-v1.8.0   1e7d99ccba24        10 days ago         81.3MB
goharbor /nginx-photon                          v1.8.0                     4a4b48b32ae4        10 days ago         36MB
goharbor /harbor-log                            v1.8.0                     e718bdc405a3        10 days ago         81.5MB
goharbor /harbor-jobservice                     v1.8.0                     d47940dd883f        10 days ago         118MB
goharbor /harbor-core                           v1.8.0                     b07a1a4be17f        10 days ago         135MB
172.16.60.213 /kevin_bo/goharbor/harbor-core    v1.0                       b07a1a4be17f        10 days ago         135MB
goharbor /harbor-portal                         v1.8.0                     76298a1ef089        10 days ago         42.9MB
goharbor /harbor-db                             v1.8.0                     d1e0b3df3e95        10 days ago         140MB
goharbor /prepare                               v1.8.0                     769ca785dab0        10 days ago         139MB
[root@harbor-node ~] # docker rmi 172.16.60.213/kevin_bo/goharbor/harbor-core:v1.0
Untagged: 172.16.60.213 /kevin_bo/goharbor/harbor-core :v1.0
Untagged: 172.16.60.213 /kevin_bo/goharbor/harbor-core @sha256:7899f284617bb051180adf6c3aedd140a519d9092b8986dd9058d4dcec0d31de
 
[root@harbor-node ~] # docker images
REPOSITORY                    TAG                        IMAGE ID            CREATED             SIZE
goharbor /redis-photon          v1.8.0                     66d7402d2770        10 days ago         103MB
goharbor /harbor-registryctl    v1.8.0                     0ca3e2b624f5        10 days ago         96.2MB
goharbor /registry-photon       v2.7.1-patch-2819-v1.8.0   1e7d99ccba24        10 days ago         81.3MB
goharbor /nginx-photon          v1.8.0                     4a4b48b32ae4        10 days ago         36MB
goharbor /harbor-log            v1.8.0                     e718bdc405a3        10 days ago         81.5MB
goharbor /harbor-jobservice     v1.8.0                     d47940dd883f        10 days ago         118MB
goharbor /harbor-core           v1.8.0                     b07a1a4be17f        10 days ago         135MB
goharbor /harbor-portal         v1.8.0                     76298a1ef089        10 days ago         42.9MB
goharbor /harbor-db             v1.8.0                     d1e0b3df3e95        10 days ago         140MB
goharbor /prepare               v1.8.0                     769ca785dab0        10 days ago         139MB

而后登陆到Harbor web 界面里，就能够看到"kevin_bo"项目里就有了上面推送的两个镜像了，点击到对应的镜像了，还能够对镜像进行打标签，复制镜像等操做：

好比将"kevin_bo"项目里上面的kevin_bo/goharbor/harbor-core镜像复制到"library"项目里

而后到"library"项目里就能看到上面从"kevin_bo"项目里复制过来的镜像了

注意： harbor私仓的相关容器映射到主机的volumes数据卷的空间要有保证，最好是单独的分区空间。
上面测试harbor容器经过volumes映射到主机的目录是/data, 能够到这里查看harbor推送的镜像:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[root@harbor-node repositories] # pwd
/data/registry/docker/registry/v2/repositories
 
能够查看两个 "项目"
[root@harbor-node repositories] # ll
total 0
drwxr-xr-x 4 10000 10000 42 May 27 14:01 kevin_bo
drwxr-xr-x 3 10000 10000 27 May 27 14:08 library
 
[root@harbor-node repositories] # ll kevin_bo/
total 0
drwxr-xr-x 3 10000 10000 25 May 27 14:01 goharbor
drwxr-xr-x 5 10000 10000 55 May 27 13:58 redis-photon
 
[root@harbor-node repositories] # ll library/
total 0
drwxr-xr-x 3 10000 10000 21 May 27 14:08 172.16.60.213
 
[root@harbor-node repositories] # ll library/172.16.60.213/
total 0
drwxr-xr-x 3 10000 10000 25 May 27 14:08 library

========测试下在harbor客户端下载harbor仓库里的镜像=======

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
[root@client ~] # docker login 172.16.60.213
Username: admin
Password:
Login Succeeded
 
[root@client ~] # docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
 
从harbor仓库拉取镜像
[root@client ~] # docker pull 172.16.60.213/kevin_bo/goharbor/harbor-core:v1.0
Trying to pull repository 172.16.60.213 /kevin_bo/goharbor/harbor-core  ...
v1.0: Pulling from 172.16.60.213 /kevin_bo/goharbor/harbor-core
4e360eca2e60: Pull complete
c066267eb2b9: Pull complete
932afda2a169: Pull complete
7ed16fb7e79a: Pull complete
d09137d80617: Pull complete
588769341947: Pull complete
Digest: sha256:7899f284617bb051180adf6c3aedd140a519d9092b8986dd9058d4dcec0d31de
 
Status: Downloaded newer image  for  172.16.60.213 /kevin_bo/goharbor/harbor-core :v1.0
[root@client ~] # docker pull 172.16.60.213/kevin_bo/redis-photon:v1.0
Trying to pull repository 172.16.60.213 /kevin_bo/redis-photon  ...
v1.0: Pulling from 172.16.60.213 /kevin_bo/redis-photon
4e360eca2e60: Already exists
b08cc3be5c43: Pull complete
a750a309c85d: Pull complete
49b2d8335a1a: Pull complete
31e8f89dc042: Pull complete
Digest: sha256:1e2ce8e6a852713d789c6315642d1483d1efdb4acee4699817810bef219ec93d
Status: Downloaded newer image  for  172.16.60.213 /kevin_bo/redis-photon :v1.0
 
[root@client ~] # docker images
REPOSITORY                                    TAG                 IMAGE ID            CREATED             SIZE
172.16.60.213 /kevin_bo/redis-photon            v1.0                66d7402d2770        10 days ago         103 MB
172.16.60.213 /kevin_bo/goharbor/harbor-core    v1.0                b07a1a4be17f        10 days ago         135 MB

能够在登陆Harbor web界面以后，修改相关用户的密码。在不一样用户帐号下建立项目，以及推送和拉取harbor镜像操做。

3.6.3）Harbor的https证书启用
经过上面可知，harbor默认安装后采用的是http方式，后面使用的时候可能会发现不少不方面。由于Docker客户端登陆harbor进行镜像推送或拉取时默认是https方式！因此http方式下，须要在每一台harbor客户端机器上都要设置"insecure-registries", 感受很麻烦！因此最好仍是将harbor默认的http方式改成https方式！另外，从安全角度考虑，容器的仓库在生产环境中每每也是须要被设定为https的方式，而harbor将这些证书的建立和设定都进行了简单的集成，下面来看一下在harbor下如何使用https的方式。配置记录以下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
在建立证书以前，为了方面验证，须要将前面在客户端机器上 /etc/docker/daemon .json文件里添加 "insecure-registries" 配置删除
[root@docker-client ~] # vim /etc/docker/daemon.json
{}
  
[root@docker-client ~] # rm -rf /root/.docker
    
重启docker服务
[root@docker-client ~] # systemctl restart docker
  
将harbor部署机自身的 /etc/docker/daemon .json文件里添加 "insecure-registries" 配置也删除
[root@harbor-node ~] # vim /etc/docker/daemon.json
{
   "registry-mirrors" : [ "https://v5d7kh0f.mirror.aliyuncs.com" ]
}
  
[root@harbor-node ~] # rm -rf /root/.docker
  
而后重启docker和docker-compose
[root@harbor-node ~] # systemctl restart docker
[root@harbor-node ~] # docker-compose down -t
[root@harbor-node ~] # docker-compose up -d
  
1）建立CA
[root@harbor-node harbor] # pwd
/root/harbor
[root@harbor-node harbor] # mkdir ssl
[root@harbor-node harbor] # cd ssl/
[root@harbor-node ssl] # pwd
/root/harbor/ssl
  
[root@harbor-node ssl] # openssl req  -newkey rsa:4096 -nodes -sha256 -keyout ca.key -x509 -days 365 -out ca.crt
Generating a 4096 bit RSA private key
......................................++
...................................................................................................................................................................................................................................................................++
writing new private key to  'ca.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter  '.' , the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:beijing
Locality Name (eg, city) [Default City]:beijing
Organization Name (eg, company) [Default Company Ltd]:DevOps               
Organizational Unit Name (eg, section) []:Tec
Common Name (eg, your name or your server's  hostname ) []:172.16.60.213
Email Address []:wangshibo@kevin.com
  
2) 建立证书请求文件csr
[root@harbor-node ssl] # openssl req -newkey rsa:4096 -nodes -sha256 -keyout 172.16.60.213.key  -out 172.16.60.213.csr
Generating a 4096 bit RSA private key
.++
..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................++
writing new private key to  '172.16.60.213.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter  '.' , the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:beijing
Locality Name (eg, city) [Default City]:beijing
Organization Name (eg, company) [Default Company Ltd]:DevOps
Organizational Unit Name (eg, section) []:Tec
Common Name (eg, your name or your server's  hostname ) []:172.16.60.213
Email Address []:wangshibo@kevin.com
  
Please enter the following  'extra'  attributes
to be sent with your certificate request
A challenge password []:123456
An optional company name []:DevOps
  
3) 建立证书
[root@harbor-node ssl] # echo subjectAltName = IP:172.16.60.213 > extfile.cnf
[root@harbor-node ssl] # openssl x509 -req -days 365 -in 172.16.60.213.csr -CA ca.crt -CAkey ca.key -CAcreateserial -extfile extfile.cnf -out 172.16.60.213.crt
Signature ok
subject= /C =CN /ST =beijing /L =beijing /O =DevOps /OU =Tec /CN =172.16.60.213 /emailAddress =wangshibo@kevin.com
Getting CA Private Key
  
4) 设定证书 & 修改
查看证书所在路径, 后面将harbor.yaml文件中的路径也一样设定
[root@harbor-node ssl] # pwd
/root/harbor/ssl
[root@harbor-node ssl] # ll
total 28
-rw-r--r-- 1 root root 2033 May 28 01:16 172.16.60.213.crt
-rw-r--r-- 1 root root 1809 May 28 01:15 172.16.60.213.csr
-rw-r--r-- 1 root root 3272 May 28 01:15 172.16.60.213.key
-rw-r--r-- 1 root root 2114 May 28 01:13 ca.crt
-rw-r--r-- 1 root root 3268 May 28 01:13 ca.key
-rw-r--r-- 1 root root   17 May 28 01:16 ca.srl
-rw-r--r-- 1 root root   34 May 28 01:16 extfile.cnf
  
5) 修改harbor.yaml文件
先关闭docker-compose
[root@harbor-node harbor] # pwd
/root/harbor
[root@harbor-node harbor] # docker-compose down -v
Stopping nginx             ...  done
Stopping harbor-jobservice ...  done
Stopping harbor-portal     ...  done
Stopping harbor-core       ...  done
Stopping harbor-db         ...  done
Stopping registryctl       ...  done
Stopping redis             ...  done
Stopping registry          ...  done
Stopping harbor-log        ...  done
Removing nginx             ...  done
Removing harbor-jobservice ...  done
Removing harbor-portal     ...  done
Removing harbor-core       ...  done
Removing harbor-db         ...  done
Removing registryctl       ...  done
Removing redis             ...  done
Removing registry          ...  done
Removing harbor-log        ...  done
Removing network harbor_harbor
  
[root@harbor-node harbor] # docker-compose ps
Name   Command   State   Ports
------------------------------
  
[root@harbor-node harbor] # vim harbor.yml
.................
.................
  
# http related config
#http:
   # port for http, default is 80. If https enabled, this port will redirect to https port
   #port: 80
  
# https related config
https:
#   # https port for harbor, default is 443
     port: 443
#   # The path of cert and key files for nginx
     certificate:  /root/harbor/ssl/172 .16.60.213.crt
     private_key:  /root/harbor/ssl/172 .16.60.213.key
.................
.................
  
================================================================================================================
特别注意：
上面harbor.yaml文件中修改的配置格式必定要正确！ "https" 要顶格写， "port:443"  和  "certificate" 、 "private_key" 保持缩进一致！
不然在下面执行 "./prepare" 更新命令时，会报错：
[root@harbor-node harbor] # ./prepare
..........
   File  "/usr/lib/python3.6/site-packages/yaml/composer.py" , line 84,  in  compose_node
     node = self.compose_mapping_node(anchor)
   File  "/usr/lib/python3.6/site-packages/yaml/composer.py" , line 127,  in  compose_mapping_node
     while  not self.check_event(MappingEndEvent):
   File  "/usr/lib/python3.6/site-packages/yaml/parser.py" , line 98,  in  check_event
     self.current_event = self.state()
   File  "/usr/lib/python3.6/site-packages/yaml/parser.py" , line 439,  in  parse_block_mapping_key
     "expected <block end>, but found %r"  % token. id , token.start_mark)
yaml.parser.ParserError:  while  parsing a block mapping
   in  "/input/harbor.yml" , line 15, column 4
expected <block end>, but found  '<block mapping start>'
   in  "/input/harbor.yml" , line 17, column 5
   
上面的报错，就是因为harbor.yaml文件配置格式不正确致使的！！！！
================================================================================================================
  
接着执行prepare脚本，将harbor修改的配置更新到 docker-compose.yml 文件
[root@harbor-node harbor] # ./prepare
prepare base  dir  is  set  to  /root/harbor
Clearing the configuration  file :  /config/log/logrotate .conf
Clearing the configuration  file :  /config/nginx/nginx .conf
Clearing the configuration  file :  /config/core/env
Clearing the configuration  file :  /config/core/app .conf
Clearing the configuration  file :  /config/registry/config .yml
Clearing the configuration  file :  /config/registry/root .crt
Clearing the configuration  file :  /config/registryctl/env
Clearing the configuration  file :  /config/registryctl/config .yml
Clearing the configuration  file :  /config/db/env
Clearing the configuration  file :  /config/jobservice/env
Clearing the configuration  file :  /config/jobservice/config .yml
Generated configuration  file :  /config/log/logrotate .conf
Generated configuration  file :  /config/nginx/nginx .conf
Generated configuration  file :  /config/core/env
Generated configuration  file :  /config/core/app .conf
Generated configuration  file :  /config/registry/config .yml
Generated configuration  file :  /config/registryctl/env
Generated configuration  file :  /config/db/env
Generated configuration  file :  /config/jobservice/env
Generated configuration  file :  /config/jobservice/config .yml
loaded secret from  file :  /secret/keys/secretkey
Generated configuration  file :  /compose_location/docker-compose .yml
Clean up the input  dir
  
查看一下docker-compose.yml文件，发现已经将新配置的443端口的https信息更新到docker-compose.yml文件里了
以下80端口和443端口都配置了，因此harbor访问时是http强转到https的
[root@harbor-node harbor] # cat docker-compose.yml |grep 443 -C3
     dns_search: .
     ports:
       - 80:80
       - 443:443
     depends_on:
       - postgresql
       - registry
  
重启docker-compose
[root@harbor-node harbor] # docker-compose up -d
Creating network  "harbor_harbor"  with the default driver
Creating harbor-log ...  done
Creating registry    ...  done
Creating harbor-db   ...  done
Creating registryctl ...  done
Creating redis       ...  done
Creating harbor-core ...  done
Creating harbor-jobservice ...  done
Creating harbor-portal     ...  done
Creating nginx             ...  done
  
[root@harbor-node harbor] # docker-compose ps
       Name                     Command                  State                        Ports                
------------------------------------------------------------------------------------------------------------
harbor-core          /harbor/start .sh                 Up (healthy)                                         
harbor-db            /entrypoint .sh postgres          Up (healthy)   5432 /tcp                              
harbor-jobservice    /harbor/start .sh                 Up                                                   
harbor-log           /bin/sh  -c  /usr/local/bin/  ...   Up (healthy)   127.0.0.1:1514->10514 /tcp             
harbor-portal       nginx -g daemon off;             Up (healthy)   80 /tcp                                
nginx               nginx -g daemon off;             Up (healthy)   0.0.0.0:443->443 /tcp , 0.0.0.0:80->80 /tcp
redis               docker-entrypoint.sh redis ...   Up             6379 /tcp                              
registry             /entrypoint .sh  /etc/regist  ...   Up (healthy)   5000 /tcp                              
registryctl          /harbor/start .sh                 Up (healthy)
  
在harbor部署机本机确认login登录 (使用80端口或443端口均可以，自动跳转的)
[root@harbor-node harbor] # docker login -u admin -p kevin@Bo1987 172.16.60.213
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https: //172 .16.60.213 /v2/ : x509: certificate signed by unknown authority
  
[root@harbor-node harbor] # docker login -u admin -p kevin@Bo1987 172.16.60.213:443
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https: //172 .16.60.213:443 /v2/ : x509: certificate signed by unknown authority
  
以上出现报错，解决办法：
此种状况多发生在自签名的证书，报错含义是签发证书机构未经认证，没法识别。
 
解决办法：
[root@harbor-node harbor] # chmod 644 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
[root@harbor-node harbor] # cat /root/harbor/ssl/172.16.60.213.crt >> /etc/pki/tls/certs/ca-bundle.crt
[root@harbor-node harbor] # chmod 444 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
 
因为证书是docker的daemon须要用到的，因此须要重启docker服务，进而也要重启docker-compose
[root@harbor-node harbor] # systemctl restart docker
[root@harbor-node harbor] # docker-compose down -v
[root@harbor-node harbor] # docker-compose up -d
 
而后再次尝试在harbor本机登录, 发现就能够正常登录了！！
[root@harbor-node harbor] # docker login -u admin -p kevin@Bo1987 172.16.60.213
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted  in  /root/ .docker /config .json.
Configure a credential helper to remove this warning. See
https: //docs .docker.com /engine/reference/commandline/login/ #credentials-store
 
Login Succeeded
 
登录的权限信息保存到 /root/ .docker /config .json 文件里了，只要这个文件不删除，下次就能够不须要输入用户名和密码直接登录了！
[root@harbor-node harbor] # cat /root/.docker/config.json
{
         "auths" : {
                 "172.16.60.213" : {
                         "auth" :  "YWRtaW46a2V2aW5AQk9CTzEyMw=="
                 }
         },
         "HttpHeaders" : {
                 "User-Agent" :  "Docker-Client/18.09.6 (linux)"
         }
}
 
[root@harbor-node harbor] # docker login 172.16.60.213
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted  in  /root/ .docker /config .json.
Configure a credential helper to remove this warning. See
https: //docs .docker.com /engine/reference/commandline/login/ #credentials-store
 
Login Succeeded
 
上面是使用80端口登录的，后面加上443端口也是能够登录的
[root@harbor-node harbor] # docker login 172.16.60.213:443
Username: admin
Password:
WARNING! Your password will be stored unencrypted  in  /root/ .docker /config .json.
Configure a credential helper to remove this warning. See
https: //docs .docker.com /engine/reference/commandline/login/ #credentials-store
 
Login Succeeded
 
[root@harbor-node harbor] # docker login 172.16.60.213:443
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted  in  /root/ .docker /config .json.
Configure a credential helper to remove this warning. See
https: //docs .docker.com /engine/reference/commandline/login/ #credentials-store
 
Login Succeeded
 
[root@harbor-node ssl] # docker login -u admin -p kevin@Bo1987 172.16.60.213:443
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted  in  /root/ .docker /config .json.
Configure a credential helper to remove this warning. See
https: //docs .docker.com /engine/reference/commandline/login/ #credentials-store
 
Login Succeeded
 
==========================================================================================
上面是在harbor本机尝试的登录，如今在远程客户机上(这里客户机为172.16.60.214)测试harbor登录：
 
首先很重要的一步,这一步极其关键！必定不要忘记操做！！
就是须要将harbor服务端生成的CA证书拷贝到每一个远程客户机的 "/etc/docker/certs.d/harbor服务器的域名或ip/"  目录下
[root@client ~] # mkdir /etc/docker/certs.d/172.16.60.213/
 
接着在harbor服务器将CA证书拷贝过来
[root@harbor-node ssl] # rsync -e "ssh -p22" -avpgolr ./* root@172.16.60.214:/etc/docker/certs.d/172.16.60.213/
 
而后在客户机上查看是否拷贝过来了harbor服务端的CA证书
[root@client 172.16.60.213] # pwd
/etc/docker/certs .d /172 .16.60.213
[root@client 172.16.60.213] # ls
172.16.60.213.crt  172.16.60.213.csr  172.16.60.213.key  ca.crt  ca.key  ca.srl  extfile.cnf
 
进行一样的受权操做，
[root@client 172.16.60.213] # chmod 644 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
[root@client 172.16.60.213] # cat /etc/docker/certs.d/172.16.60.213/172.16.60.213.crt >> /etc/pki/tls/certs/ca-bundle.crt
[root@client 172.16.60.213] # chmod 444 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
 
重启docker服务
[root@client 172.16.60.213] # systemctl restart docker
 
最后进行harbor登录，就能够正常登录了！
[root@client 172.16.60.213] # docker login -u admin -p kevin@Bo1987 172.16.60.213:443
Login Succeeded
 
[root@client 172.16.60.213] # cat /root/.docker/config.json                          
{
         "auths" : {
                 "172.16.60.213:443" : {
                         "auth" :  "YWRtaW46a2V2aW5AQk9CTzEyMw=="
                 }
         }
}
 
上面是使用443端口登录harbor仓库是正常的，若是此时使用80端口登录，则出现以下报错：
[root@client 172.16.60.213] # docker login -u admin -p kevin@Bo1987 172.16.60.213
Error response from daemon: Missing client certificate 172.16.60.213.cert  for  key 172.16.60.213.key
 
因此在客户端就使用443端口来登录harbor仓库了！

3.6.4) Harbor私仓的高可用
经过三个harbor完成高可用部署，前面经过负载均衡器对外提供服务。共享数据库与缓存。结构以下 ：

还有一种比较简单的方法：部署2台harbor服务器，配置部署同样，实现原理:

建立docker harbor主主复制就是在两个harbor私仓的web界面里建立相互之间的镜像同步关系，同步关系能够选择相同用户或不一样用户之间。这样就能够保证harbor私仓的双机热备关系了。即：1) "系统管理"->"仓库管理"->"新建目标", 填写对端harbor信息2) "系统管理"->"同步管理"->"新建规则", 规则里会引用目的Registry，也就是上面一步建立的目标。同步模式有Push-based，Pull-based；触发模式有自动和定时。

相关文章
相关标签/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公众号
   欢迎关注本站公众号,获取更多信息
联系我们 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程