题记:搞定了就是故事,搞不定就是事故。html
条件有限,开局只有一台开发机,要跑5套子系统组成的项目群。还要有一些辅助和验证的系统要跑在这上面。从配置来看,要想顺畅开心地完成项目群基础环境支撑,考虑引入资源消耗1/200的神器Docker:更可贵是丝绝不用担忧环境污染、版本冲突、以及忽然开发服务器搞崩溃了;java
安装很成功,按官方教程启动服务时,崩溃到怀疑人生:node
Docker 官方支招:本身编译带 systemctl 服务的镜像 参考连接,python
再三协调以后,放弃等待甲方老师的支援。决定仍是本身搞。不就是一些一级依赖么,补全就是!linux
主要是我方火力太猛:c++
搞定。想来想去,仍是毛选里的那句话支撑了我:git
自力更生,艰苦奋斗!github
最近在看一行禅师的《佛陀传》,但愿能从中汲取心灵的力量。《天龙八部》里扫地僧也曾说过:docker
只有佛法越高,慈悲之念越盛,武功绝技才能练得越我
或许我辈须工做中修行,也是这个道理!windows
对,就是采坑。如下是从草稿中摘取的部分尝试:
编号 | 容器 |
---|---|
1 | docker run -it –name dc7 ailyfeng/centos7.2.1511 /bin/bash |
2 | docker run -it –name dc88 sssllc/centos7.2-jdk1.8 /bin/bash |
3 | docker run -it –name ct7 centos /bin/bash |
4 | docker run -it –privileged –name dc99 sssllc/centos7.2-jdk1.8 |
5 | docker build –rm -t centos:systemd . && docker run -it –name dr7 centos:systemd |
6 | docker run -it –name dr18 yjjy0921/redhat7.2 /bin/bash |
ldd 应该是 linux 通用的命令;rpm 和 repotrack 貌似是 CentoOS 的专有命令;
编号 | 命令 | 举例 | 做用 |
---|---|---|---|
1 | ldd | ldd wkhtmltopdf | not found的就是对应的依赖包不存在,适用于二进制包(wkhtmltopdf)的命令;而须要安装的命令(docker-ce)只能经过安装报错来捕获缺失的依赖了 |
2 | rpm -ql | rpm -ql docker-ce | 查看安装的时候有哪些命令在PATH下,用这些命令去启动 |
3 | repotrack | repotrack -a x86_64 -p /usr/local/yumrepo | 打包下载指定架构(X86_63)全部的依赖到指定目录 |
要安装 Docker 基础环境,必须知足:
[root@pms tmp] uname -r 3.10.0-327.e17.x86_64 [root@pms tmp] ls -l /sys/class/misc/device-mapper lrwxrwxrwx. 1 root root 0 May 20 16:17 /sys/class/misc/device-mapper -> ../../devices/virtual/misc/device-mapper
MacBook 上,起 CentOS7.2 容器,来验证思路;
ChinaDreams:~ kangcunhua$ docker run -it --name dc88 sssllc/centos7.2-jdk1.8 /bin/bash [root@7d935562e0ae /]# java -version java version "1.8.0_111" Java(TM) SE Runtime Environment (build 1.8.0_111-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode) [root@7d935562e0ae /]# yum repo list Loaded plugins: fastestmirror, ovl No such command: repo. Please use /usr/bin/yum --help [root@7d935562e0ae /]# [root@7d935562e0ae /]# yum repolist ... [root@7d935562e0ae home]# cd /usr/local/ root@7d935562e0ae local]# mkdir yumrepo
[root@7d935562e0ae local]# cd yumrepo/ [root@7d935562e0ae yumrepo]# yum install --downloadonly --downloaddir=/usr/local/yumrepo/ docker
[root@7d935562e0ae yum.repos.d]# yum install createrepo -y
[root@7d935562e0ae yum.repos.d]# pwd /etc/yum.repos.d [root@7d935562e0ae yum.repos.d]# vi docker.repo
docker.repo
[docker-yum] name=dockeryum baseurl=file:///usr/local/yumrepo enable=1 gpgcheck=0
主要是将索引生成在/usr/local/yumrepo/repodata目录
[root@7d935562e0ae yum.repos.d]# createrepo /usr/local/yumrepo/ [root@7d935562e0ae yum.repos.d]# cd /usr/local/yumrepo/ [root@7d935562e0ae yumrepo]# ls ..... docker-client-1.13.1-53.git774336d.el7.centos.x86_64.rpm repodata .....
[root@7d935562e0ae yumrepo]# yum --disablerepo=\* --enablerepo=docker-yum install docker -y [root@7d935562e0ae yumrepo]# docker -v Docker version 1.13.1, build 774336d/1.13.1
此时会报错:提示 docker daemon 没有启动;
[root@7d935562e0ae /]# docker run hello-world /usr/bin/docker-current: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?. See '/usr/bin/docker-current run --help'.
使用sysytemctl启动服务。此时会报错,是CentOS7.2容器的“安全”考虑,没有启动D-Bus:虽然不那么友好;
[root@7d935562e0ae /]# systemctl start docker Failed to get D-Bus connection: Operation not permitted
在能查到的资料里,即便是官方,也没有说明 Docker Daemon 的守护命令放置在哪儿,以便咱们能直接运行;
后续在安装 Docker-CE 时我找到了这个命令:使用rpm -ql docker 查找该软件安装时在 PATH 下有哪些命令能够运行,成功找到了 Docker Daemon 所在:/usr/bin/dockerd 。详见文末参考;
[root@7d935562e0ae /]# /usr/bin/dockerd INFO[2018-05-22T11:55:22.811053980Z] libcontainerd: started new docker-containerd process pid=53 INFO[0000] starting containerd module=containerd revision=773c489c9c1b21a6d78b5c538cd395416ec50f88 version=v1.0.3 ERRO[0000] failed to change OOM score to -500 error="write /proc/53/oom_score_adj: permission denied" module=containerd ....
仍旧报错,但至少验证了离线安装的思路是正确的。只是在 CentOS7.2 容器中没法启动 Docker 守护进程;
通过数轮的尝试,咱们能够得出结论:
梳理思路,重来
ChinaDreams:docker-systemctl kangcunhua$ docker run -it --name dc18 sssllc/centos7.2-jdk1.8 /bin/bash
[root@557a4e0c3e7e /]#
ChinaDreams:docker-systemctl kangcunhua$ docker run -it --name dc18 sssllc/centos7.2-jdk1.8 /bin/bash [root@557a4e0c3e7e /]#
若是不配置 docker-ce 源,默认安装的是 docker1.13。大概是两年前的版本了。后续 docker 官方将社区版本命名为 docker-ce。因此安装新版本,仍是要配置下 yum 源的:这里强烈建议配置国内的,速度快。
[root@557a4e0c3e7e /]# yum-config-manager --add-repo https://mirrors.ustc.edu.cn/docker-ce/linux/centos/docker-ce.repo
配置后, yum search docker-ce 能够搜索到了。这时候就能够下载到指定位置:为了后续离线安装;
[root@557a4e0c3e7e /]# cd /usr/local [root@557a4e0c3e7e local]# mkdir yumrepo [root@557a4e0c3e7e local]# yum install --downloadonly --downloaddir=/usr/local/yumrepo/ docker-ce
先安装本地 repo 索引建立工具,经过这个工具,创建索引:就是本地安装包所在目录下的 repodata 目录;
[root@557a4e0c3e7e yumrepo]# yum install createrepo -y [root@557a4e0c3e7e yumrepo]# createrepo /usr/local/yumrepo/
[root@557a4e0c3e7e yumrepo]# cd /etc/yum.repos.d [root@557a4e0c3e7e yum.repos.d]# vi docker.repo [root@557a4e0c3e7e yum.repos.d]# more docker.repo [docker-yum] name=dockeryum baseurl=file:///usr/local/yumrepo enable=1 gpgcheck=0
yum --disablerepo=\* --enablerepo=docker-yum install docker-ce -y
[root@557a4e0c3e7e yum.repos.d]# docker -v Docker version 18.03.1-ce, build 9ee9f40
这个是由于容器中又启动了 Docker daemon。部分报错日志节选:
[root@557a4e0c3e7e /]# docker -v Docker version 18.03.1-ce, build 9ee9f40 [root@557a4e0c3e7e /]# /usr/bin/dockerd ... ERRO[0000] failed to change OOM score to -500 error="write /proc/53/oom_score_adj: permission denied" module=containerd ... address="/var/run/docker/containerd/docker-containerd.sock" module="containerd/grpc" INFO[0000] containerd successfully booted in 0.029058s module=containerd ERRO[2018-05-22T11:55:23.265326880Z] 'overlay2' is not supported over aufs ... WARN[2018-05-22T11:55:23.353577680Z] Running modprobe xt_conntrack failed with message: ``, error: exit status 1 Error starting daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables v1.4.21: can't initialize iptables table `nat': Permission denied (you must be root) Perhaps iptables or your kernel needs to be upgraded. (exit status 3)
ChinaDreams:Desktop kangcunhua$ docker cp dc18:/usr/local/yumrepo . ChinaDreams:Desktop kangcunhua$ docker cp dc18:/etc/yum.repos.d/docker.repo .
copy 资源到内网,经过 ssh 将依赖包传到服务器;
[root@pma03 ~]# yum repolist #能够成功查看到docker-yum
[root@pma03 ~]# yum --disablerepo=\* --enablerepo=docker-yum install docker-ce -y
报错,缺依赖包;docker-ce相关依赖包已经全了,只是有一些CentOS依赖的包版本比docker-ce低,内网系统可能优化精简“过”了,或基础软件不完整,致使这些低版本和高版本不能和谐共处;只能根据报错信息,去笔记本下载依赖;报错信息和分析详见文末:“附:依赖包冲突和解决”;
依旧是回到我的笔记本,联网下载依赖包:
ChinaDreams:~ kangcunhua$ docker start dc7 dc7 [root@ce27b30d0d9e /]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo [root@ce27b30d0d9e /]# yum clean all [root@ce27b30d0d9e /]# yum makecache
共计尝试两次,使用以下命令,下载补全系统依赖包:
编号 | 命令 |
---|---|
1 | repotrack -a x86_64 -p /usr/local/yumrepo docker-ce |
2 | repotrack -a x86_64 -p /usr/local/yumrepo glibc-2.17-105.e |
3 | repotrack -a x86_64 -p /usr/local/yumrepo systemd-sysv |
4 | repotrack -a x86_64 -p /usr/local/yumrepo dracut-network |
5 | repotrack -a x86_64 -p /usr/local/yumrepo libgudev1 |
6 | repotrack -a x86_64 -p /usr/local/yumrepo dracut-config-rescue |
7 | repotrack -a x86_64 -p /usr/local/yumrepo systemd-python |
8 | repotrack -a x86_64 -p /usr/local/yumrepo libstdc++ |
9 | repotrack -a x86_64 -p /usr/local/yumrepo glibc-headers |
10 | repotrack -a x86_64 -p /usr/local/yumrepo pcre-devel |
11 | repotrack -a x86_64 -p /usr/local/yumrepo gcc-c++ |
12 | repotrack -a x86_64 -p /usr/local/yumrepo glibc-devel |
13 | repotrack -a x86_64 -p /usr/local/yumrepo libtool-ltdl |
14 | repotrack -a x86_64 -p /usr/local/yumrepo libselinux-devel |
15 | repotrack -a x86_64 -p /usr/local/yumrepo libsepol-devel |
保证这些依赖包,不会致使安装失败。
[root@ce27b30d0d9e /]# yum --disablerepo=\* --enablerepo=docker-yum install docker -y
将依赖包上传到内网服务器指定目录/usr/local/yumrepo ;
[root@pma03 ~]# yum --disablerepo=\* --enablerepo=docker-yum install docker-ce -y
没有报错!
[root@pma03 ~]# docker -v Docker version 18.03.1-ce, build 9ee9f40
[root@pma03 ~]# systemctl start docker
搞定!
添加一个用户dev,专门管理docker。千万不要图省事,服务器上直接用root管理docker;容器能够;
[root@pma03 ~]# cat /etc/group [root@pma03 ~]# cat /etc/group | grep docker [root@pma03 ~]# more /etc/passwd [root@pma03 ~]# gpasswd -a dev docker [root@pma03 ~]# useradd -g docker dev [root@pma03 ~]# passwd dev [root@pma03 ~]# chmod -v u+w /etc/sudoers [root@pma03 ~]# vi /etc/sudoers [root@pma03 ~]# chomod -v u-w /etc/sudoers [root@pma03 ~]# systemctl restart docker [root@pma03 ~]# docker -v [root@pma03 ~]# docker info [root@pma03 ~]# su dev [dev@pma03 ~]#
又被坑了:docker for Mac、docker for windows 安装完都自带 docker-compose,linux 版本的 docker-ce 竟然不带。想起来离线安装各类坑就头大,网上查了半天资料,竟然是个 python 工具,要先安装 pip,o No!只好翻到 github,看看能不能源码编译安装。结果看到有 release 的下载,猜测能够直接使用。后来用下载后的文件百度,果真翻到一篇指南,能够这样搞:
方法四:离线安装下载 docker-compose-Linux-x86_64,而后从新命名添加可执行权限便可:
ChinaDreams:Desktop kangcunhua$ docker cp ./docker-compose-Linux-x86_64.dms dc18:/usr/local/bin/docker-compose
[root@557a4e0c3e7e /]# cd /usr/local/bin/ [root@557a4e0c3e7e bin]# ls docker-compose [root@557a4e0c3e7e bin]# chmod +x /usr/local/bin/docker-compose [root@557a4e0c3e7e bin]# docker-compose -v docker-compose version 1.21.2, build a133471
一样把安装包 copy 进内网,ssh 上传到服务器
[root@pma03 ~]# cd /usr/local/bin/ [root@pma03 bin]# cp /home/dev/docker-images/docker-compose-Linux-x86_64.dms docker-compose [root@pma03 bin]# chmod +x ./docker-compose [root@pma03 bin]# docker-compose -v [root@pma03 bin]# chown -R dev:docker ./docker-compose [root@pma03 bin]# su dev [dev@pma03 ~]# docker-compose -v
搞定!
后续要作的事情就简单了:
尽情 happy 吧!
内网离线安装碰上的依赖包问题和分析
yun install docker-ce 时,提示有依赖包版本冲突 or 缺失;
Error: Package: systemd-sysv-219-19.el7.x86_64 (@anaconda) Requires: systemd = 219-19.el7 Removing: systemd-219-19.el7.x86_64 (@anaconda) systemd = 219-19.el7 Updated By: systemd-219-42.el7_4.4.x86_64 (localyum) systemd = 219-42.el7_4.4 Error: Package: dracut-network-033-359.el7.x86_64 (@anaconda) Requires: dracut = 033-359.el7 Removing: dracut-033-359.el7.x86_64 (@anaconda) dracut = 033-359.el7 Updated By: dracut-033-502.el7.x86_64 (localyum) dracut = 033-502.el7 Error: Package: libgudev1-219-19.el7.x86_64 (@anaconda) Requires: systemd-libs = 219-19.el7 Removing: systemd-libs-219-19.el7.x86_64 (@anaconda) systemd-libs = 219-19.el7 Updated By: systemd-libs-219-42.el7_4.4.x86_64 (localyum) systemd-libs = 219-42.el7_4.4 Error: Package: dracut-config-rescue-033-359.el7.x86_64 (@anaconda) Requires: dracut = 033-359.el7 Removing: dracut-033-359.el7.x86_64 (@anaconda) dracut = 033-359.el7 Updated By: dracut-033-502.el7.x86_64 (localyum) dracut = 033-502.el7 Error: Package: systemd-python-219-19.el7.x86_64 (@anaconda) Requires: systemd = 219-19.el7 Removing: systemd-219-19.el7.x86_64 (@anaconda) systemd = 219-19.el7 Updated By: systemd-219-42.el7_4.4.x86_64 (localyum) systemd = 219-42.el7_4.4 You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles --nodigest
看到报错的时候,心里是崩溃的。最后仍是收拾心情,逐一解决了。
网上惟一能够找到的资料就是这篇离线安装docker包冲突,报错信息如出一辙,可是帖子没有给出肯定的缘由和解决办法;可是给予了我信心,确实不是由于CentOS7.2和Redhat7.2有底层不一样,致使的这些错误,这样是无力解决的;后续解决完依赖后,我猜想缘由是,甲方提供的Redhat或CentOS,默认软件包安装的有问题:作了一些精简“优化”处理不了部分依赖包冲突;
后续找到了这篇文章:CentOS7.2离线安装docker-ce最新版,文中指出Docker须要的部分软件包版本要高于CentOS7;就是说一样的软件包,Docker依赖的高版本的,可是CentOS依赖低版本的;解决办法就是找到他们,所有安装上去;
下载 libgudev1 和 systemd-sysv,是由于 centos7.2 的 libgudev1 和 systemd-sysv 依赖 systemd-219-19.el7.x86_64,而 docker-ce 须要 systemd-219-30el7.x86_64.
repotrack -a x86_64 解决上述依赖后(命令详见正文:解决依赖),copy 进内网,再次尝试安装 Docker-CE。
Error: Package: systemd-sysv-219-19.el7.x86_64 (@anaconda) Requires: systemd = 219-19.el7 Removing: systemd-219-19.el7.x86_64 (@anaconda) systemd = 219-19.el7 Updated By: systemd-219-42.el7_4.4.x86_64 (localyum) systemd = 219-42.el7_4.4 Error: Package: dracut-network-033-359.el7.x86_64 (@anaconda) Requires: dracut = 033-359.el7 Removing: dracut-033-359.el7.x86_64 (@anaconda) dracut = 033-359.el7 Updated By: dracut-033-502.el7.x86_64 (localyum) dracut = 033-502.el7 Error: Package: libgudev1-219-19.el7.x86_64 (@anaconda) Requires: systemd-libs = 219-19.el7 Removing: systemd-libs-219-19.el7.x86_64 (@anaconda) systemd-libs = 219-19.el7 Updated By: systemd-libs-219-42.el7_4.4.x86_64 (localyum) systemd-libs = 219-42.el7_4.4 Error: Package: dracut-config-rescue-033-359.el7.x86_64 (@anaconda) Requires: dracut = 033-359.el7 Removing: dracut-033-359.el7.x86_64 (@anaconda) dracut = 033-359.el7 Updated By: dracut-033-502.el7.x86_64 (localyum) dracut = 033-502.el7 Error: Package: systemd-python-219-19.el7.x86_64 (@anaconda) Requires: systemd = 219-19.el7 Removing: systemd-219-19.el7.x86_64 (@anaconda) systemd = 219-19.el7 Updated By: systemd-219-42.el7_4.4.x86_64 (localyum) systemd = 219-42.el7_4.4
解决办法就是继续回到笔记本联网下载这些依赖包;详见:“解决依赖”;
直接安装 rpm 包能够的,直接敲 “rpm -ivh 包名”可是有些rpm包是有依赖性的,能够在命令尾端添加 “–force
–nodeps”,“–force”指强制“,”–nodeps“指不查找依赖性好比”rpm -ivh *.rpm –force
—nodeps“,同时,也可使用yum命令,会自动解决包依赖的关系,能便于管理大量系统的更新问题,建议使用
yum
Runtime privilege, Linux capabilities, and LXC configuration–cap-add : Add Linux capabilities
–cap-drop : Drop Linux capabilities
–privileged=false : Give extended privileges to this container
–device=[] : Allows you to run devices inside the container without
the –privileged flag.–lxc-conf=[] : (lxc exec-driver only) Add custom lxc options
–lxc-conf=”lxc.cgroup.cpuset.cpus = 0,1”默认状况下,Docker的容器是没有特权的,例如不能在容器中再启动一个容器。这是由于默认状况下容器是不能访问任何其它设备的。可是经过”privileged”,容器就拥有了访问任何其它设备的权限。
ChinaDreams:~ kangcunhua$ docker run -it --privileged --name dc99 sssllc/centos7.2-jdk1.8 /usr/sbin/init Welcome to CentOS Linux 7 (Core)! [ OK ] Reached target Swap. [ OK ] Created slice Root Slice. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Delayed Shutdown Socket. [ OK ] Listening on /dev/initctl Compatibility Named Pipe. [ OK ] Reached target Remote File Systems. [ OK ] Created slice User and Session Slice. [ OK ] Created slice System Slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. Mounting Debug File System... Starting Journal Service... Mounting FUSE Control File System... Starting Apply Kernel Variables... Starting Create Static Device Nodes in /dev... Mounting Huge Pages File System... Starting Setup Virtual Console... [ OK ] Created slice system-getty.slice. Starting Remount Root and Kernel File Systems... [ OK ] Reached target Paths. [ OK ] Mounted FUSE Control File System. [ OK ] Mounted Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Started Journal Service. [ OK ] Started udev Kernel Device Manager. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Reached target Local File Systems (Pre). Starting Rebuild Hardware Database... Starting Load/Save Random Seed... Starting Flush Journal to Persistent Storage... [ OK ] Reached target Local File Systems. Starting Rebuild Journal Catalog... [ OK ] Started Flush Journal to Persistent Storage. Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Rebuild Journal Catalog. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Hardware Database. Starting udev Coldplug all Devices... Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting LSB: Supports the direct execution of binary formats.... Starting Permit User Sessions... [ OK ] Started D-Bus System Message Bus. Starting D-Bus System Message Bus... Starting Login Service... [ OK ] Started Permit User Sessions. Starting Cleanup of Temporary Directories... [ OK ] Started Getty on tty1. Starting Getty on tty1... [ OK ] Started Cleanup of Temporary Directories. [ OK ] Started Login Service. [ OK ] Started LSB: Supports the direct execution of binary formats.. [ TIME ] Timed out waiting for device dev-ttyS0.device. [DEPEND] Dependency failed for Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Reached target Multi-User System. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. ^C^C^C^C^C^Cc\c/
参考连接
$ sudo yum-config-manager \ --add-repo \ https://mirrors.ustc.edu.cn/docker-ce/linux/centos/docker-ce.repo $ sudo yum-config-manager --enable docker-ce-edge $ sudo yum makecache fast $ sudo yum install docker-ce
rpm -ql 软件包 查看安装的时候有哪些命令在 PATH 下,用这些命令去启动,这个是一种解决的方法
[root@9c676d901d7b bin]# rpm -ql docker-ce /etc/udev/rules.d/80-docker.rules /usr/bin/docker /usr/bin/docker-containerd /usr/bin/docker-containerd-ctr /usr/bin/docker-containerd-shim /usr/bin/docker-init /usr/bin/docker-proxy /usr/bin/docker-runc /usr/bin/dockerd /usr/lib/systemd/system/docker.service /usr/share/bash-completion/completions/docker
本文由做者:蛮大人 受权发布
连接:https://opsdev.fun/2018/05/09...著做权归做者全部。转载请联系做者得到受权