hadoop容器化部署

1、原容器

 java:openjdk-8u111-jre

jre路径：

/usr/lib/jvm/java-8-openjdk-amd64
/usr/lib/jvm/java-1.8.0-openjdk-amd64

2、安装ssh

docker run -it --name hadoop-test java:openjdk-8u111-jre bash
apt-get update
apt-get install openssh
service ssh start

3、生成密钥

ssh-keygen -t rsa
cd /root/.ssh
cat id_rsa.pub >> authorized_keys

4、修改ssh配置

 将配置文件/etc/ssh/ssh_config 中的 StrictHostKeyChecking ask改为 StrictHostKeyChecking no，GSSAPIAuthentication yes 改为 GSSAPIAuthentication no。

5、从官网上下载hadoop安装包，并解压到容器下/home路径下 

6、run.sh文件编写, 放在/home路径下

#!/bin/bash
service ssh restart
cd /home/hadoop-2.10.1/sbinif [[ "$TYPE" == "master" ]];then./start-all.sh
fiwhile true;
dosleep 1m
done

7、提交镜像

docker commit hadoop-test hadoop:2.10.1

8、编写Dockerfile

FROM hadoop:2.10.1WORKDIR /homeEXPOSE 9000ENTRYPOINT ["bash", "run.sh"]

9、构建容器：

docker build . --rm --tag=hadoop:2.10.1-u1

10、配置容器

参考：https://blog.csdn.net/m0_67390969/article/details/126553657

11、启动容器

#新建容器网络
docker network create -d bridge hadoop#启动slave容器
docker run -d --name hadoop-slave1 --net=hadoop -h slave1 -e TYPE="slave" -v /opt/docker/hadoop/conf:/home/hadoop-2.10.1/etc/hadoop/ -v  /opt/docker/hadoop/logs:/home/hadoop-2.10.1/logs -v /opt/docker/hadoop/tmp:/hadoop/tmp -v /opt/docker/hadoop/data:/hadoop/data -v /opt/docker/hadoop/name:/hadoop/name -v /opt/docker/hadoop/ssh/authorized_keys:/root/.ssh/authorized_keys hadoop:2.10.1-u1 docker run -d --name hadoop-slave2 --net=hadoop -h slave2 -e TYPE="slave" -v /opt/docker/hadoop/conf:/home/hadoop-2.10.1/etc/hadoop/ -v  /opt/docker/hadoop/logs:/home/hadoop-2.10.1/logs -v /opt/docker/hadoop/tmp:/hadoop/tmp -v /opt/docker/hadoop/data:/hadoop/data -v /opt/docker/hadoop/name:/hadoop/name -v /opt/docker/hadoop/ssh/authorized_keys:/root/.ssh/authorized_keys hadoop:2.10.1-u1  #启动master容器
docker run -d --name hadoop-master --net=hadoop -h master -e TYPE="master" -v /opt/docker/hadoop/conf:/home/hadoop-2.10.1/etc/hadoop/ -v  /opt/docker/hadoop/logs:/home/hadoop-2.10.1/logs -v /opt/docker/hadoop/tmp:/hadoop/tmp -v /opt/docker/hadoop/data:/hadoop/data -v /opt/docker/hadoop/name:/hadoop/name -v /opt/docker/hadoop/ssh/authorized_keys:/root/.ssh/authorized_keys hadoop:2.10.1-u1 

参考资料：

https://blog.csdn.net/m0_67390969/article/details/126553657

https://devpress.csdn.net/cloudnative/62f442f67e668234661883e1.html
https://www.shuzhiduo.com/A/D854NPDVzE/
https://blog.csdn.net/zhangvalue/article/details/103748438
https://blog.csdn.net/wejack/article/details/126368643