Docker Swarm 搭建

Docker Swarm 搭建

1. 环境介绍

操作系统	Centos 7	Centos 7	Centos 7
内核版本	Linux 3.10.0-957.el7.x86_64	Linux 3.10.0-957.el7.x86_64	Linux 3.10.0-957.el7.x86_64
主机名称	swarm-manager	swarm-worker1	swarm-worker2
IP	192.168.1.100	192.168.1.200	192.168.1.250
Docker Domain	20.10.15	20.10.15	20.10.15
containerd	1.6.33	1.6.33	1.6.33

2. 安装前准备工作

# 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld# 修改 hostname
hostnamectl set-hostname xxxx  # 修改后退出当前终端重新连接即可# 更新下软件源
# 地址 https://developer.aliyun.com/mirror/# 时区调整，时间校准
date -R
timedatectl set-timezone Asia/Shanghai
yum -y install ntp
ntpdate ntp1.aliyun.com# 关闭 selinux： 
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0vi /etc/sysctl.conf# 网络转发
net.ipv4.ip_forward = 1sysctl -p

3. 安装Docker

# 安装docker 需要的依赖 要去操作系统必须能访问外网yum install -y yum-utils device-mapper-persistent-data lvm2 bash-completion# 添加阿里云docker-ce 软件源
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo# 查看版本
yum list docker-ce --showduplicates | sort -r
yum list containerd.io --showduplicates | sort -r# 注意 如果要安装 指定版本的docker 比如 18.09
yum install -y docker-ce-20.10.15-3.el7 docker-ce-cli-20.10.15-3.el7 containerd.io# 启动docker
systemctl start docker# 设置开机自启动
systemctl enable docker# 查看docker 状态
systemctl status docker# 配置镜像
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{"registry-mirrors": ["https://dockerhub.icu","https://docker.chenby.cn","https://docker.1panel.live","https://docker.aws19527.cn","https://docker.anyhub.us.kg","https://dhub.kubesre.xyz"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

4. 初始化Docker Swarm

4.1 swarm-manager 主机 初始化

# 初始化集群
docker swarm init[root@swarm-manager ~]# docker swarm init
Swarm initialized: current node (5nod1t171e1kfv6bsf4dvj3ci) is now a manager.To add a worker to this swarm, run the following command:docker swarm join --token SWMTKN-1-6031unqtu39ma1mbtqtiov8i8beikzimm5j5e4jds6k2jy246i-c95gqxwhp5kvj6kh1df2tsi6z 192.168.1.100:2377To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.# 群初始化:当前节点(5nod1t171e1kfv6bsf4dvj3ci)现在是一个管理器。# 要向集群中添加一个工作节点，运行以下命令:docker swarm join --token SWMTKN-1-6031unqtu39ma1mbtqtiov8i8beikzimm5j5e4jds6k2jy246i-c95gqxwhp5kvj6kh1df2tsi6z 192.168.1.100:2377
# 要向集群中添加管理器，请运行` docker swarm join-token manager `并按照说明操作。# 查看状态
docker info -f '{{.Swarm}}'[root@swarm-manager ~]# docker info -f '{{.Swarm}}'
{5nod1t171e1kfv6bsf4dvj3ci 192.168.1.100 active true  [{5nod1t171e1kfv6bsf4dvj3ci 192.168.1.100:2377}] 1 1 0xc000178c60 []}# token 忘记咋办
docker swarm join-token workerTo add a worker to this swarm, run the following command:docker swarm join --token SWMTKN-1-6031unqtu39ma1mbtqtiov8i8beikzimm5j5e4jds6k2jy246i-c95gqxwhp5kvj6kh1df2tsi6z 192.168.1.100:2377

4.2 加入集群

# swarm-worker1 和 swarm-worker2 节点加入
docker swarm join --token SWMTKN-1-6031unqtu39ma1mbtqtiov8i8beikzimm5j5e4jds6k2jy246i-c95gqxwhp5kvj6kh1df2tsi6z 192.168.1.100:2377# swarm-worker1 节点 
[root@swarm-worker1 ~]# docker swarm join --token SWMTKN-1-6031unqtu39ma1mbtqtiov8i8beikzimm5j5e4jds6k2jy246i-c95gqxwhp5kvj6kh1df2tsi6z 192.168.1.100:2377
This node joined a swarm as a worker. # 此节点以 worker 身份加入群。# swarm-worker2 节点
[root@swarm-worker2 ~]# docker swarm join --token SWMTKN-1-6031unqtu39ma1mbtqtiov8i8beikzimm5j5e4jds6k2jy246i-c95gqxwhp5kvj6kh1df2tsi6z 192.168.1.100:2377
This node joined a swarm as a worker. # 此节点以 worker 身份加入群

4.3 查看状态

# swarm-manager 主机查看docker info 
[root@swarm-manager ~]# docker info 
Client:Context:    defaultDebug Mode: falsePlugins:app: Docker App (Docker Inc., v0.9.1-beta3)buildx: Docker Buildx (Docker Inc., v0.8.2-docker)scan: Docker Scan (Docker Inc., v0.23.0)Server:Containers: 0Running: 0Paused: 0Stopped: 0Images: 0Server Version: 20.10.15Storage Driver: overlay2Backing Filesystem: extfsSupports d_type: trueNative Overlay Diff: trueuserxattr: falseLogging Driver: json-fileCgroup Driver: cgroupfsCgroup Version: 1Plugins:Volume: localNetwork: bridge host ipvlan macvlan null overlayLog: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslogSwarm: active # 当前状态活跃NodeID: 5nod1t171e1kfv6bsf4dvj3ciIs Manager: trueClusterID: oe5ugn0ujh4i2bj0w5kutv2gzManagers: 1  # 计算节点Nodes: 3   # 工作节点Default Address Pool: 10.0.0.0/8  # 默认地址池SubnetSize: 24Data Path Port: 4789 # 数据通路端口Orchestration:Task History Retention Limit: 5Raft:Snapshot Interval: 10000Number of Old Snapshots to Retain: 0Heartbeat Tick: 1Election Tick: 10Dispatcher:Heartbeat Period: 5 seconds  # 心跳周期CA Configuration:Expiry Duration: 3 monthsForce Rotate: 0Autolock Managers: falseRoot Rotation In Progress: falseNode Address: 192.168.1.100 # 节点地址Manager Addresses:192.168.1.100:2377 # 控制节点地址Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runcDefault Runtime: runcInit Binary: docker-initcontainerd version: d2d58213f83a351ca8f528a95fbd145f5654e957runc version: v1.1.12-0-g51d5e94init version: de40ad0Security Options:seccompProfile: defaultKernel Version: 3.10.0-957.el7.x86_64Operating System: CentOS Linux 7 (Core)OSType: linuxArchitecture: x86_64CPUs: 4Total Memory: 3.683GiBName: swarm-managerID: IRIN:WS4R:MACN:UNOC:TJHS:GLAB:E2ER:LC6H:D6HJ:T5I4:MOWK:XDIDDocker Root Dir: /var/lib/dockerDebug Mode: falseRegistry: https://index.docker.io/v1/Labels:Experimental: falseInsecure Registries:127.0.0.0/8Registry Mirrors:https://ovvphjcn.mirror.aliyuncs.com/Live Restore Enabled: false# 查看节点
docker node ls[root@swarm-manager ~]# docker node ls
ID                            HOSTNAME        STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
5nod1t171e1kfv6bsf4dvj3ci *   swarm-manager   Ready     Active         Leader           20.10.15
itsuvlblh71h0kuv602pgd6xd     swarm-worker1   Ready     Active                          20.10.15
qhw9dxwqo0goh1nyiaq6mfhkw     swarm-worker2   Ready     Active                          20.10.15# 更改节点availablity状态
# swarm集群中node的availability状态可以为 active或者drain，其中：
# active状态下，node可以接受来自manager节点的任务分派；
# drain状态下，node节点会结束task，且不再接受来自manager节点的任务分派（也就是下线节点）。docker node update --availbility drain itsuvlblh71h0kuv602pgd6xd [root@swarm-manager ~]# docker node ls
ID                            HOSTNAME        STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
5nod1t171e1kfv6bsf4dvj3ci *   swarm-manager   Ready     Active         Leader           20.10.15
itsuvlblh71h0kuv602pgd6xd     swarm-worker1   Ready     Drain                           20.10.15
qhw9dxwqo0goh1nyiaq6mfhkw     swarm-worker2   Ready     Active                          20.10.15# 当swarm-worker1的状态改为drain后，那么该节点就不会接受task任务分发，就算之前已经接受的任务也会转移到别的节点上。# 再次修改为active状态（及将下线的节点再次上线）
docker node update --availbility active itsuvlblh71h0kuv602pgd6xd [root@swarm-manager ~]# docker node update --availability active itsuvlblh71h0kuv602pgd6xd
itsuvlblh71h0kuv602pgd6xd
[root@swarm-manager ~]# docker node ls
ID                            HOSTNAME        STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
5nod1t171e1kfv6bsf4dvj3ci *   swarm-manager   Ready     Active         Leader           20.10.15
itsuvlblh71h0kuv602pgd6xd     swarm-worker1   Ready     Active                          20.10.15
qhw9dxwqo0goh1nyiaq6mfhkw     swarm-worker2   Ready     Active                          20.10.15# 如果需要删除一个节点
docker node rm --force itsuvlblh71h0kuv602pgd6xd[root@swarm-manager ~]# docker node rm -f itsuvlblh71h0kuv602pgd6xd
itsuvlblh71h0kuv602pgd6xd
[root@swarm-manager ~]# docker node ls
ID                            HOSTNAME        STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
5nod1t171e1kfv6bsf4dvj3ci *   swarm-manager   Ready     Active         Leader           20.10.15
qhw9dxwqo0goh1nyiaq6mfhkw     swarm-worker2   Ready     Active                          20.10.15# 从新把swarm-worker1 加入集群
docker swarm join-token workerdocker swarm join --token SWMTKN-1-6031unqtu39ma1mbtqtiov8i8beikzimm5j5e4jds6k2jy246i-c95gqxwhp5kvj6kh1df2tsi6z 192.168.1.100:2377[root@swarm-worker1 ~]# docker swarm join --token SWMTKN-1-6031unqtu39ma1mbtqtiov8i8beikzimm5j5e4jds6k2jy246i-c95gqxwhp5kvj6kh1df2tsi6z 192.168.1.100:2377
This node joined a swarm as a worker.# 再次查看节点
docker node ls
[root@swarm-manager ~]# docker node ls
ID                            HOSTNAME        STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
5nod1t171e1kfv6bsf4dvj3ci *   swarm-manager   Ready     Active         Leader           20.10.15
gz0ep0xbi5v1an7ulf0lmo1wx     swarm-worker1   Ready     Active                          20.10.15
qhw9dxwqo0goh1nyiaq6mfhkw     swarm-worker2   Ready     Active                          20.10.15

4.4 管理node

# 将 node 提升为 manager   docker node promote 节点id
docker node promote itsuvlblh71h0kuv602pgd6xd[root@swarm-manager ~]# docker node promote itsuvlblh71h0kuv602pgd6xd
Node itsuvlblh71h0kuv602pgd6xd promoted to a manager in the swarm.# 查看节点
docker node ls[root@swarm-manager ~]# docker node ls
ID                            HOSTNAME        STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
5nod1t171e1kfv6bsf4dvj3ci *   swarm-manager   Ready     Active         Leader           20.10.15
itsuvlblh71h0kuv602pgd6xd     swarm-worker1   Ready     Active         Reachable        20.10.15
qhw9dxwqo0goh1nyiaq6mfhkw     swarm-worker2   Ready     Active                          20.10.15# 查看管理者数量
docker info -f {{.Swarm.Managers}}[root@swarm-manager ~]# docker info -f {{.Swarm.Managers}}
2# 将manager降级为worker docker node demote  节点iddocker node demote itsuvlblh71h0kuv602pgd6xd[root@swarm-manager ~]# docker node demote itsuvlblh71h0kuv602pgd6xd
Manager itsuvlblh71h0kuv602pgd6xd demoted in the swarm.# 再次查看管理者数量
docker info -f {{.Swarm.Managers}}[root@swarm-manager ~]# docker info -f {{.Swarm.Managers}}
1# 卸载集群 （每个机器上执行哦）
docker swarm leave --force
[root@swarm-manager ~]# docker swarm leave --force
Node left the swarm.[root@swarm-manager ~]# docker node ls
Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.

5. 部署服务

5.1 nginx服务为例

# nginx 服务
docker service create --replicas 1 --name nginx-01 -p 80:80 nginx 
# 解释
- 该命令将创建服务。docker service create
- 该标志为服务命名。--name nginx-01
- 该标志指定 1 个正在运行的实例的所需状态。--replicas
- 该标识为服务映射宿主机80端口到 容器80端口 -p 80:80
- 镜像名称 nginx[root@swarm-manager ~]# docker service create --replicas 1 --name nginx-01 -p 80:80 nginx 
lah4f6xehd3ebymov6gb7sn77
overall progress: 1 out of 1 tasks 
1/1: running   [==================================================>] 
verify: Service converged # 查看服务列表
docker service ls[root@swarm-manager ~]# docker service ls
ID             NAME       MODE         REPLICAS   IMAGE          PORTS
lah4f6xehd3e   nginx-01   replicated   1/1        nginx:latest   *:80->80/tcp

5.2 检查服务

# 显示详细信息 
docker service inspect --pretty <service-id || service-name> [root@swarm-manager ~]# docker service inspect --pretty lah4f6xehd3eID:		lah4f6xehd3ebymov6gb7sn77
Name:		nginx-01
Service Mode:	ReplicatedReplicas:	1
Placement:
UpdateConfig:Parallelism:	1On failure:	pauseMonitoring Period: 5sMax failure ratio: 0Update order:      stop-first
RollbackConfig:Parallelism:	1On failure:	pauseMonitoring Period: 5sMax failure ratio: 0Rollback order:    stop-first
ContainerSpec:Image:		nginx:latest@sha256:0d17b565c37bcbd895e9d92315a05c1c3c9a29f762b011a10c54a66cd53c9b31Init:		false
Resources:
Endpoint Mode:	vip
Ports:PublishedPort = 80Protocol = tcpTargetPort = 80PublishMode = ingress 
# 如果要以 json 格式返回服务详细信息 docker service inspect <service-id || service-name># 查看哪些节点正在运行服务
docker service ps <service-id || service-name> [root@swarm-manager ~]# docker service ps lah4f6xehd3e
ID             NAME         IMAGE          NODE            DESIRED STATE   CURRENT STATE            ERROR     PORTS
kbf7vly6wpc2   nginx-01.1   nginx:latest   swarm-manager   Running         Running 11 minutes ago     # 在运行任务的节点上运行，查看有关任务的容器。docker ps
# 注意：如果在管理器节点以外的节点上运行， 您必须通过 SSH 连接到该节点

5.3 服务扩缩容

# docker service scale <service-id>=<number-of-tasks> # 查看当前机器nginx服务数量
docker service ps lah4f6xehd3e[root@swarm-manager ~]# docker service ps lah4f6xehd3e
ID             NAME         IMAGE          NODE            DESIRED STATE   CURRENT STATE            ERROR     PORTS
kbf7vly6wpc2   nginx-01.1   nginx:latest   swarm-manager   Running         Running 17 minutes ago             # 扩容nginx服务为3
docker service scale lah4f6xehd3e=3[root@swarm-manager ~]# docker service scale lah4f6xehd3e=3
lah4f6xehd3e scaled to 3
overall progress: 3 out of 3 tasks 
1/3: running   [==================================================>] 
2/3: running   [==================================================>] 
3/3: running   [==================================================>] 
verify: Service converged # 再次查看nginx服务 为三个
docker service ps lah4f6xehd3e[root@swarm-manager ~]# docker service ps lah4f6xehd3e
ID             NAME         IMAGE          NODE            DESIRED STATE   CURRENT STATE            ERROR     PORTS
kbf7vly6wpc2   nginx-01.1   nginx:latest   swarm-manager   Running         Running 19 minutes ago             
64oz5npq7rwi   nginx-01.2   nginx:latest   swarm-worker1   Running         Running 48 seconds ago             
j1g9zsn9y363   nginx-01.3   nginx:latest   swarm-worker2   Running         Running 37 seconds ago  # 缩容 比如 我只需要2个nginx实例
docker service scale lah4f6xehd3e=2[root@swarm-manager ~]# docker service scale lah4f6xehd3e=2
lah4f6xehd3e scaled to 2
overall progress: 2 out of 2 tasks 
1/2: running   [==================================================>] 
2/2: running   [==================================================>] 
verify: Service converged # 再次查看nginx服务 为两个
docker service ps lah4f6xehd3e[root@swarm-manager ~]# docker service ps lah4f6xehd3e
ID             NAME         IMAGE          NODE            DESIRED STATE   CURRENT STATE            ERROR     PORTS
kbf7vly6wpc2   nginx-01.1   nginx:latest   swarm-manager   Running         Running 22 minutes ago             
64oz5npq7rwi   nginx-01.2   nginx:latest   swarm-worker1   Running         Running 3 minutes ago              

5.3 删除服务

docker service rm <service-id || service-name>

5.4 服务滚动更新

# 部署一个基于 在 Redis 3.0.6 容器标签上。然后，升级服务为3.0.7 容器映像
docker service create \--replicas 3 \--name redis \--update-delay 10s \redis:3.0.6- 该标志配置对服务任务或任务集的更新之间的时间延迟。你可以将时间描述为秒数、分钟数或小时数的组合   --update-delay   [root@swarm-manager ~]# docker service create \
>   --replicas 3 \
>   --name redis \
>   --update-delay 10s \
>   redis:3.0.6
zipd741pdig2wfey520i31djg
overall progress: 3 out of 3 tasks 
1/3: running   [==================================================>] 
2/3: running   [==================================================>] 
3/3: running   [==================================================>] 
verify: Service converged # 检查服务 redis
docker service inspect --pretty redis[root@swarm-manager ~]# docker service inspect --pretty redis
ID:		zipd741pdig2wfey520i31djg
Name:		redis
Service Mode:	ReplicatedReplicas:	3
Placement:
UpdateConfig:Parallelism:	1Delay:		10sOn failure:	pauseMonitoring Period: 5sMax failure ratio: 0Update order:      stop-first
RollbackConfig:Parallelism:	1On failure:	pauseMonitoring Period: 5sMax failure ratio: 0Rollback order:    stop-first
ContainerSpec:Image:		redis:3.0.6@sha256:6a692a76c2081888b589e26e6ec835743119fe453d67ecf03df7de5b73d69842Init:		false# 更新容器镜像
docker service update --image redis:3.0.7 redis[root@swarm-manager ~]# docker service update --image redis:3.0.7 redis
redis
overall progress: 3 out of 3 tasks 
1/3: running   [==================================================>] 
2/3: running   [==================================================>] 
3/3: running   [==================================================>] 
verify: Service converged[root@swarm-manager ~]# docker service inspect --pretty redisID:		zipd741pdig2wfey520i31djg
Name:		redis
Service Mode:	ReplicatedReplicas:	3
UpdateStatus:State:		completedStarted:	2 minutes agoCompleted:	14 seconds agoMessage:	update completed
Placement:
UpdateConfig:Parallelism:	1Delay:		10sOn failure:	pauseMonitoring Period: 5sMax failure ratio: 0Update order:      stop-first
RollbackConfig:Parallelism:	1On failure:	pauseMonitoring Period: 5sMax failure ratio: 0Rollback order:    stop-first
ContainerSpec:Image:		redis:3.0.7@sha256:730b765df9fe96af414da64a2b67f3a5f70b8fd13a31e5096fee4807ed802e20Init:		false
Resources:
Endpoint Mode:	vip# 重新启动已暂停的更新
docker service update <service-id || service-name># 运行观看滚动更新 docker service ps <service-id || service-name>
docker service ps redis[root@swarm-manager ~]# docker service ps redis
ID             NAME          IMAGE         NODE            DESIRED STATE   CURRENT STATE            ERROR     PORTS
ps9mfrxsn4ck   redis.1       redis:3.0.7   swarm-worker2   Running         Running 5 minutes ago              
js5bykpb6o77    \_ redis.1   redis:3.0.6   swarm-worker2   Shutdown        Shutdown 6 minutes ago             
30oiggtwpbuw   redis.2       redis:3.0.7   swarm-worker1   Running         Running 7 minutes ago              
8ixep8jvzkt5    \_ redis.2   redis:3.0.6   swarm-worker1   Shutdown        Shutdown 8 minutes ago             
daus97jwupgg   redis.3       redis:3.0.7   swarm-manager   Running         Running 6 minutes ago              
jf6n83zmwn69    \_ redis.3   redis:3.0.6   swarm-manager   Shutdown        Shutdown 7 minutes ago        # 在 Swarm 更新所有任务之前，您可以看到一些任务正在运行，而另一些任务正在运行。上面的输出显示 滚动更新完成后的状态。redis:3.0.6 redis:3.0.7