K8S - Deployment 的版本回滚

当前状态

先看deployment

root@k8s-master:~# kubectl get deploy -o wide --show-labels
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS                 IMAGES                                                                       SELECTOR             LABELS
bq-api-service-deploy-sample   5/5     5            5           4h47m   bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.3   app=bq-api-service   app=bq-api-service,author=Jason
root@k8s-master:~# 

得知 这个deployment 的image 版本是1.1.3 ， 有5个运行副本

查看revisions 历史

root@k8s-master:~# kubectl rollout history deployment/bq-api-service-deploy-sample
deployment.apps/bq-api-service-deploy-sample 
REVISION  CHANGE-CAUSE
1         <none>
2         <none>
3         <none>

可以见到有3个revisions
其实他们对应的是image 版本1.1.1 ， 1.1.2 和 1.1.3

至于k8s 会为1个deployment 保留多少个revisions, 是由参数spec.revisionHistoryLimit 决定的

例如 下面配置意思就是期望运行5个pod 副本， 但是会保留pod 的10个历史版本

spec:replicas: 10            # desired replica count, Please note that the replica Pods in a Deployment are typically distributed across multiple nodes.revisionHistoryLimit: 10

查看ReplicaSet 信息

root@k8s-master:~# kubectl get rs -o wide -l app=bq-api-service
NAME                                      DESIRED   CURRENT   READY   AGE     CONTAINERS                 IMAGES                                                                       SELECTOR
bq-api-service-deploy-sample-7bff8fbc4b   5         5         5       81m     bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.3   app=bq-api-service,pod-template-hash=7bff8fbc4b
bq-api-service-deploy-sample-8d49d9845    0         0         0       88m     bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.2   app=bq-api-service,pod-template-hash=8d49d9845
bq-api-service-deploy-sample-9f8d9c988    0         0         0       5h23m   bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.1   app=bq-api-service,pod-template-hash=9f8d9c988
root@k8s-master:~# 

刚好也是3个rs, 其实他们刚好对应了3个revisions的image 版本
只不过1.1.3 版本的RS 有5个当前运行的POD 副本， 而1.1.1 和 1.1.2 的副本数量是0, 符合常规.

但是 ReplicaSet 和 Revision 不是永远可以一一对应的， 一是revisions数量有限制， 二是1个deployment 的replicaSet 有可能会被重用

查看Pod 信息

root@k8s-master:~# kubectl get po -o wide -l app=bq-api-service
NAME                                            READY   STATUS    RESTARTS   AGE   IP            NODE        NOMINATED NODE   READINESS GATES
bq-api-service-deploy-sample-7bff8fbc4b-b9c8v   1/1     Running   0          92m   10.244.3.40   k8s-node3   <none>           <none>
bq-api-service-deploy-sample-7bff8fbc4b-bf8bp   1/1     Running   0          92m   10.244.1.29   k8s-node1   <none>           <none>
bq-api-service-deploy-sample-7bff8fbc4b-hqfmw   1/1     Running   0          92m   10.244.1.30   k8s-node1   <none>           <none>
bq-api-service-deploy-sample-7bff8fbc4b-lgjj9   1/1     Running   0          92m   10.244.2.91   k8s-node0   <none>           <none>
bq-api-service-deploy-sample-7bff8fbc4b-wgrj5   1/1     Running   0          92m   10.244.2.89   k8s-node0   <none>           <none>

有5个pod 正在运行， 关键pod 名字是包含了 rs 的对象名字的
如
bq-api-service-deploy-sample-7bff8fbc4b-b9c8v
所以我们可以用pod 的名字来知道它是属于哪个 ReplicaSet的

用新的部署来实现回滚到1.1.2 版本

回滚1个service 的版本有多种方法

1. 是部署多一次旧的image 版本
2. 恢复已备份的service版本

在生产中， 我们可能会更常用第一种
这里我们也试试

方法很简单， 在上一篇文章介绍过来
用kubectl set image 命令来实现
kubectl set image deployment/<deployment_name> <container_name>=europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:<新版本号>

root@k8s-master:~# kubectl set image deployment/bq-api-service-deploy-sample bq-api-service-container=europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.2
deployment.apps/bq-api-service-deploy-sample image updated

更新回1.1.2 版本

root@k8s-master:~# kubectl get deploy -o wide --show-labels
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS                 IMAGES                                                                       SELECTOR             LABELS
bq-api-service-deploy-sample   5/5     5            5           5h53m   bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.2   app=bq-api-service   app=bq-api-service,author=Jason

检查rs

root@k8s-master:~# kubectl get rs -o wide -l app=bq-api-service
NAME                                      DESIRED   CURRENT   READY   AGE     CONTAINERS                 IMAGES                                                                       SELECTOR
bq-api-service-deploy-sample-7bff8fbc4b   0         0         0       111m    bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.3   app=bq-api-service,pod-template-hash=7bff8fbc4b
bq-api-service-deploy-sample-8d49d9845    5         5         5       119m    bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.2   app=bq-api-service,pod-template-hash=8d49d9845
bq-api-service-deploy-sample-9f8d9c988    0         0         0       5h53m   bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.1   app=bq-api-service,pod-template-hash=9f8d9c988

更新回之前的版本， 与一般的更新不同， 因为之前已经有1个RS 是for image 1.1.2的所以这次k8s 重用了之前的rs而并没有增加1个

查看deployment event

kubectl describe deployment bq-api-service-deploy-sample

Events:Type    Reason             Age                 From                   Message----    ------             ----                ----                   -------Normal  ScalingReplicaSet  12m (x2 over 131m)  deployment-controller  Scaled up replica set bq-api-service-deploy-sample-8d49d9845 to 3Normal  ScalingReplicaSet  12m                 deployment-controller  Scaled up replica set bq-api-service-deploy-sample-8d49d9845 to 2Normal  ScalingReplicaSet  12m                 deployment-controller  Scaled down replica set bq-api-service-deploy-sample-7bff8fbc4b to 4Normal  ScalingReplicaSet  12m                 deployment-controller  Scaled down replica set bq-api-service-deploy-sample-7bff8fbc4b to 3Normal  ScalingReplicaSet  12m                 deployment-controller  Scaled up replica set bq-api-service-deploy-sample-8d49d9845 to 4Normal  ScalingReplicaSet  12m                 deployment-controller  Scaled down replica set bq-api-service-deploy-sample-7bff8fbc4b to 2Normal  ScalingReplicaSet  12m                 deployment-controller  Scaled up replica set bq-api-service-deploy-sample-8d49d9845 to 5Normal  ScalingReplicaSet  12m                 deployment-controller  Scaled down replica set bq-api-service-deploy-sample-7bff8fbc4b to 1Normal  ScalingReplicaSet  12m                 deployment-controller  Scaled down replica set bq-api-service-deploy-sample-7bff8fbc4b to 0

可以看到回滚的实现就是无非把1.1.2的rs 里的pod副本逐渐增加到5, 而1.1.3 rs的pod副本从5 减少到0

查看revisions

root@k8s-master:~# kubectl rollout history deployment/bq-api-service-deploy-sample
deployment.apps/bq-api-service-deploy-sample 
REVISION  CHANGE-CAUSE
1         <none>
3         <none>
4         <none>

并我预想中的 1, 2,3 变成 1,2,3,4 而是把2删除了

用恢复备份的方法来回到1.1.1 版本

上面的方法其实有限制的， 前提就是 1.1.2的镜像还在server or GAR里存在

而用恢复备份的方法并没有限制

命令:
kubectl rollout undo deployment/deployment_name --to-revision=x x就是revisions 数字
如果不提供x 则默认回滚到上1个版本 (本文例子是1.1.3）

执行

root@k8s-master:~# kubectl rollout undo deployment/bq-api-service-deploy-sample --to-revision=1
deployment.apps/bq-api-service-deploy-sample rolled back

执行成功， 而且提示是rolled back

检查deployment

版本已经恢复到1.1.1

root@k8s-master:~# kubectl get deploy -o wide --show-labels
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS                 IMAGES                                                                       SELECTOR             LABELS
bq-api-service-deploy-sample   5/5     5            5           6h12m   bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.1   app=bq-api-service   app=bq-api-service,author=Jason
root@k8s-master:~# 

检查rs

同样这种方法也能重用之前的rs (1.1.1 版本）

root@k8s-master:~# kubectl get rs -o wide -l app=bq-api-service
NAME                                      DESIRED   CURRENT   READY   AGE     CONTAINERS                 IMAGES                                                                       SELECTOR
bq-api-service-deploy-sample-7bff8fbc4b   0         0         0       131m    bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.3   app=bq-api-service,pod-template-hash=7bff8fbc4b
bq-api-service-deploy-sample-8d49d9845    0         0         0       138m    bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.2   app=bq-api-service,pod-template-hash=8d49d9845
bq-api-service-deploy-sample-9f8d9c988    5         5         5       6h13m   bq-api-service-container   europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.1.1   app=bq-api-service,pod-template-hash=9f8d9c988
root@k8s-master:~# 

查看revisions

root@k8s-master:~# kubectl rollout history deployment/bq-api-service-deploy-sample
deployment.apps/bq-api-service-deploy-sample 
REVISION  CHANGE-CAUSE
3         <none>
4         <none>
5         <none>

变成了 3,4,5 其实就是1变成5

查看deployment event

kubectl describe deployment bq-api-service-deploy-sample

  Normal  ScalingReplicaSet  2m54s               deployment-controller  Scaled down replica set bq-api-service-deploy-sample-8d49d9845 to 4Normal  ScalingReplicaSet  2m54s               deployment-controller  Scaled up replica set bq-api-service-deploy-sample-9f8d9c988 to 2Normal  ScalingReplicaSet  2m53s               deployment-controller  Scaled down replica set bq-api-service-deploy-sample-8d49d9845 to 3Normal  ScalingReplicaSet  2m53s               deployment-controller  Scaled up replica set bq-api-service-deploy-sample-9f8d9c988 to 4Normal  ScalingReplicaSet  2m53s               deployment-controller  Scaled down replica set bq-api-service-deploy-sample-8d49d9845 to 2Normal  ScalingReplicaSet  2m53s               deployment-controller  Scaled up replica set bq-api-service-deploy-sample-9f8d9c988 to 5Normal  ScalingReplicaSet  2m53s               deployment-controller  Scaled down replica set bq-api-service-deploy-sample-8d49d9845 to 1Normal  ScalingReplicaSet  2m52s               deployment-controller  Scaled down replica set bq-api-service-deploy-sample-8d49d9845 to 0sh

一样， 1.1.2 的rs pod 副本数量被scale down ->0 , 1.1.1 的rs pod 副本数量 scale up-> 5