迁移docker部署的GitLab

1. 背景

最近接到一个任务，迁移docker到另外一台宿主机上。在迁移过程中发现有docker部署的GitLab。迁移过程中各种排错，现将遇到的问题记录如下

2. 参考

链接: Gitblab docker迁移数据出现权限问题解决
链接: docker 的gitlab数据迁移权限问题
链接: docker中的gitlab数据迁移
链接: docker 版的 gitlab 数据迁移(单容器版) – 备份恢复方式
链接: docker下Gitlab如何进行备份恢复与迁移？ – 备份恢复方式

3. 环境

• 操作系统 Centos7
• Docker version 26.0.0, build 2ae903e
• GItLab镜像 twang2218/gitlab-ce-zh

4. 过程

4.1 查看原docker启动命令

docker run -d --name gitlab  \
--restart=always \
-p 8022:22 \
-p 80:80 \
-p 8443:443 \
-v /home/gitlab/etc:/etc/gitlab \
-v /home/gitlab/log:/var/log/gitlab \
-v /home/gitlab/data:/var/opt/gitlab \
twang2218/gitlab-ce-zh

• 端口映射
  • 8022:22
  • 80:80
  • 8443:443
• 挂载目录
  • /home/gitlab/etc:/etc/gitlab
  • /home/gitlab/etc:/etc/gitlab
  • /home/gitlab/data:/var/opt/gitlab

4.2 打包挂载目录传至新宿主机并创建对应目录

• 源挂载目录迁移至新宿主机对应路径下

// 查看新宿主机对应路径
tree -L 1 /home/gitlab
/home/gitlab
├── data
├── docker-compose-gitlab.yml
├── etc
└── log

4.3 保存镜像并传至新宿主机下

• 保存源宿主机下镜像twang2218/gitlab-ce-zh

docker save twang2218/gitlab-ce-zh > gitlab-ce-zh.tar

• 在新宿主机下还原镜像

docker load < gitlab-ce-zh.tar

• 新宿主机查看镜像

docker image ls twang2218/gitlab-ce-zh
// 显示结果
REPOSITORY               TAG       IMAGE ID       CREATED       SIZE
twang2218/gitlab-ce-zh   latest    18da462b5ff5   5 years ago   1.61GB

4.4 新宿主机启动GitLab容器

• 使用同样命令容器

docker run -d --name gitlab  \
--restart=always \
-p 8022:22 \
-p 80:80 \
-p 8443:443 \
-v /home/gitlab/etc:/etc/gitlab \
-v /home/gitlab/log:/var/log/gitlab \
-v /home/gitlab/data:/var/opt/gitlab \
twang2218/gitlab-ce-zh

5 故障

5.1 容器不断重启

• 现象：容器不断重启，GitLab服务无法正常运行。
• 定位：查看容器日志

Preparing services...
Starting services...
/opt/gitlab/embedded/bin/runsvdir-start: line 24: ulimit: pending signals: cannot modify limit: Operation not permitted
/opt/gitlab/embedded/bin/runsvdir-start: line 37: /proc/sys/fs/file-max: Read-only file system
Configuring GitLab package...
Configuring GitLab...================================================================================
Error executing action `run` on resource 'ruby_block[directory resource: /var/opt/gitlab/git-data/repositories]'
================================================================================

• 参考：Gitblab docker迁移数据出现权限问题解决
• 解决：修改对应宿主机路径权限

chmod 2770 /home/gitlab/data/git-data/repositories

5.2 权限拒绝

• 现象：GitLab服务无法正常运行。
• 定位：查看容器日志

================================================================================Error executing action `run` on resource 'execute[/opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-keys check-permissions]'================================================================================Mixlib::ShellOut::ShellCommandFailed------------------------------------Expected process to exit with [0], but received '1'---- Begin output of /opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-keys check-permissions ----STDOUT: error: could not open /var/opt/gitlab/.ssh/authorized_keys: Permission denied @ rb_sysopen - /var/opt/gitlab/.ssh/authorized_keys-rw-------. 1 root root 8474 Apr 11 22:27 /var/opt/gitlab/.ssh/authorized_keysSTDERR: ---- End output of /opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-keys check-permissions ----Ran /opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-keys check-permissions returned 1

• 参考：Gitblab docker迁移数据出现权限问题解决
• 解决：不断执行 docker exec -it gitlab update-permissions

For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.mdIf this container fails to start due to permission problems try to fix it by executing:docker exec -it gitlab update-permissionsdocker restart gitlab

• 注意：执行过程中可能爆缺失文件错误，例如： /var/opt/gitlab/gitlab-rails/shared/registry需要手动创建后，继续执行docker exec -it gitlab update-permissions，直至全部执行成功且无报错。

mkdir /home/gitlab/data/gitlab-rails/shared/registry

5.3 容器内错误日志

• 现象：使用交互模式启动容器排错。
  • 先停止并删除之前容器，然后使用以下命令（-i）启动容器查看错误日志

docker run -i --name gitlab  \
--restart=always \
-p 8022:22 \
-p 80:80 \
-p 8443:443 \
-v /home/gitlab/etc:/etc/gitlab \
-v /home/gitlab/log:/var/log/gitlab \
-v /home/gitlab/data:/var/opt/gitlab \
twang2218/gitlab-ce-zh

• 定位：查看日志
  2024-04-12_06:53:51.49881 level=error ts=2024-04-12T06:53:51.498534519Z caller=main.go:226 err=“open /var/opt/gitlab/alertmanager/data/nflog: permission denied”

• 原因：

1. 进入容器内查看 /var/opt/gitlab/alertmanager/，该目录的用户和组是git

root@14dd4de37cbf:/var/opt/gitlab/alertmanager# ll
total 8
drwxr-x---.  3 gitlab-prometheus root   42 Apr 12 13:36 ./
drwxr-xr-x. 20 root              root 4096 Apr 12 13:36 ../
-rw-r--r--.  1 gitlab-prometheus root  283 Apr 12 10:16 alertmanager.yml
drwx------.  2 git               git    35 Apr 12 10:16 data/

2. 实际上应该属于gitlab-prometheus

git:x:998:998::/var/opt/gitlab:/bin/sh
gitlab-www:x:999:999::/var/opt/gitlab/nginx:/bin/false
gitlab-redis:x:997:997::/var/opt/gitlab/redis:/bin/false
gitlab-psql:x:996:996::/var/opt/gitlab/postgresql:/bin/sh
mattermost:x:994:994::/var/opt/gitlab/mattermost:/bin/sh
registry:x:993:993::/var/opt/gitlab/registry:/bin/sh
gitlab-prometheus:x:992:992::/var/opt/gitlab/prometheus:/bin/sh
gitlab-consul:x:991:991::/var/opt/gitlab/consul:/bin/sh

• 解决：
  在容器内修改目录用户和组，

root@14dd4de37cbf:/var/opt/gitlab/alertmanager# chown -R gitlab-prometheus.gitlab-prometheus data/

6 重启容器服务正常

• 关闭并删除之前容器
• 使用正常启动命令重新启动容器

docker run -d --name gitlab  \
--restart=always \
-p 8022:22 \
-p 80:80 \
-p 8443:443 \
-v /home/gitlab/etc:/etc/gitlab \
-v /home/gitlab/log:/var/log/gitlab \
-v /home/gitlab/data:/var/opt/gitlab \
twang2218/gitlab-ce-zh

• GitLab服务访问正常
  

7 总结

GitLab docker迁移工作并不复杂，主要麻烦在GitLab本身的权限上。迁移可通过容器启动日志和容器内交互方式排除故障。