docker部署kafka zookeeper模式集群
单机模式链接：https://blog.csdn.net/wsdhla/article/details/133032238
kraft集群模式链接：部署Kafka_kafka 部署-CSDN博客
zookeeper选举机制举例：
目前有5台服务器，每台服务器均没有数据，它们的编号分别是1,2,3,4,5,按编号依次启动，它们的选择举过程如下：
- 服务器1启动，给自己投票，然后发投票信息，由于其它机器还没有启动所以它收不到反馈信息，服务器1的状态一直属于Looking(选举状态)。
- 服务器2启动，给自己投票，同时与之前启动的服务器1交换结果，由于服务器2的编号大所以服务器2胜出，但此时投票数没有大于半数，所以两个服务器的状态依然是LOOKING。
- 服务器3启动，给自己投票，同时与之前启动的服务器1,2交换信息，由于服务器3的编号最大所以服务器3胜出，此时投票数正好大于半数，所以服务器3成为领导者，服务器1,2成为小弟。
- 服务器4启动，给自己投票，同时与之前启动的服务器1,2,3交换信息，尽管服务器4的编号大，但之前服务器3已经胜出，所以服务器4只能成为小弟。
- 服务器5启动，后面的逻辑同服务器4成为小弟。
Docker 自定义静态网络
Docker默认提供了3种网络模式，生成容器时不指定网络模式下默认使用bridge桥接模式。每次容器启动时ip会发生变更，无法达到我们集群的效果。
创建自定义网络模式，指定网段。
docker network create --subnet=172.18.0.0/16 net-kafka若报错：Error response from daemon: Pool overlaps with other one on this address space
说明指定的网络地址段已存在
1、Docker部署zookeeper集群
zk使用官方镜像：
1.1、主机创建目录
[zk01]
D:/mysoft/dev/docker-zk-cluster/zk01/conf
D:/mysoft/dev/docker-zk-cluster/zk01/data
D:/mysoft/dev/docker-zk-cluster/zk01/datalog[zk02]
D:/mysoft/dev/docker-zk-cluster/zk02/conf
D:/mysoft/dev/docker-zk-cluster/zk02/data
D:/mysoft/dev/docker-zk-cluster/zk02/datalog[zk03]
D:/mysoft/dev/docker-zk-cluster/zk03/conf
D:/mysoft/dev/docker-zk-cluster/zk03/data
D:/mysoft/dev/docker-zk-cluster/zk03/datalog1.2、修改配置文件
分别拷贝到zk01、zk02、zk03
[zoo.cfg]dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;21811.3、指定ip创建启动容器
docker run -d --name zk01 -p 2181:2181 --network net-kafka --ip 172.18.0.11 -v D:/mysoft/dev/docker-zk-cluster/zk01/conf:/conf -v D:/mysoft/dev/docker-zk-cluster/zk01/data:/data -v D:/mysoft/dev/docker-zk-cluster/zk01/datalog:/datalog zookeeperdocker run -d --name zk02 -p 2182:2181 --network net-kafka --ip 172.18.0.12 -v D:/mysoft/dev/docker-zk-cluster/zk02/conf:/conf -v D:/mysoft/dev/docker-zk-cluster/zk02/data:/data -v D:/mysoft/dev/docker-zk-cluster/zk02/datalog:/datalog zookeeperdocker run -d --name zk03 -p 2183:2181 --network net-kafka --ip 172.18.0.13 -v D:/mysoft/dev/docker-zk-cluster/zk03/conf:/conf -v D:/mysoft/dev/docker-zk-cluster/zk03/data:/data -v D:/mysoft/dev/docker-zk-cluster/zk03/datalog:/datalog zookeeperzk01可以正常启动，zk02和zk03启动失败
警告：这个正常，是因为节点启动时选举机制的原因
zk02：
WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 2 at election address /172.18.0.12:3888
zk03：
WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
异常：
zk02：
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Exception while listening to address /172.18.0.11:3888
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Leaving listener thread for address 172.18.0.11:3888 after 3 errors. Use zookeeper.electionPortBindRetry property to increase retry count.
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - As I'm leaving the listener thread, I won't be able to participate in leader election any longer: 172.18.0.11:3888
zk03：
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Exception while listening to address /172.18.0.11:3888
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Leaving listener thread for address 172.18.0.11:3888 after 3 errors. Use zookeeper.electionPortBindRetry property to increase retry count.
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - As I'm leaving the listener thread, I won't be able to participate in leader election any longer: 172.18.0.11:3888
原因是没有配置quorumListenOnAllIPs，默认为false，需要设置为true
[zoo.cfg]dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181quorumListenOnAllIPs：
当设置为true时，ZooKeeper服务器将在所有可用IP地址上侦听来自其对等方的连接，而不仅是在配置文件的服务器列表中配置的地址。它会影响处理ZAB协议和快速领导者选举协议的连接。默认值为false。
有一些WARN:
### 1、只启动zk01：
# zk01：
01:21:17.746 [QuorumConnectionThread-[myid=1]-3] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 2 at election address /172.18.0.12:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)01:21:20.904 [QuorumConnectionThread-[myid=1]-1] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)### 2、 只启动zk01、zk02：
# zk01：
01:21:46.504 [QuorumConnectionThread-[myid=1]-3] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)# zk02：
01:21:43.350 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.01:21:48.824 [QuorumConnectionThread-[myid=1]-2] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)### 2、 启动zk01、zk02、zk03：
# zk01正常：
01:31:34.579 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:31:34.579 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:31:34.580 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 2 already, it is in the lastCommittedView
01:31:34.584 [QuorumConnectionThread-[myid=1]-21] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 2
01:31:34.584 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 3 already, it is in the lastCommittedView
01:31:34.585 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] INFO org.apache.zookeeper.server.quorum.FastLeaderElection - Notification time out: 60000 ms
01:31:34.585 [QuorumConnectionThread-[myid=1]-22] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 3
01:31:34.585 [QuorumConnectionThread-[myid=1]-21] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 2 using election address: /172.18.0.12:3888
01:31:34.585 [QuorumConnectionThread-[myid=1]-21] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:2)
01:31:34.586 [QuorumConnectionThread-[myid=1]-21] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2)
01:31:34.585 [QuorumConnectionThread-[myid=1]-22] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 3 using election address: /172.18.0.13:3888
01:31:34.586 [QuorumConnectionThread-[myid=1]-22] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:3)
01:31:34.586 [QuorumConnectionThread-[myid=1]-22] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:3)
01:31:52.173 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - node0 scavenging sessions
01:31:52.174 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - org.eclipse.jetty.server.session.SessionHandler873610597==dftMaxIdleSec=-1 scavenging sessions
01:31:52.174 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - org.eclipse.jetty.server.session.SessionHandler873610597==dftMaxIdleSec=-1 scavenging session ids []
01:31:52.174 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - org.eclipse.jetty.server.session.DefaultSessionCache@57d7f8ca[evict=-1,removeUnloadable=false,saveOnCreate=false,saveOnInactiveEvict=false] checking expiration on []# zk02：
01:35:15.331 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:35:15.332 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:35:15.333 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 2 already, it is in the lastCommittedView
01:35:15.334 [QuorumConnectionThread-[myid=1]-27] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 2
01:35:15.334 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 3 already, it is in the lastCommittedView
01:35:15.335 [QuorumConnectionThread-[myid=1]-27] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 2 using election address: /172.18.0.12:3888
01:35:15.335 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] INFO org.apache.zookeeper.server.quorum.FastLeaderElection - Notification time out: 60000 ms
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.12:50246
01:35:15.335 [QuorumConnectionThread-[myid=1]-27] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:2)
01:35:15.335 [QuorumConnectionThread-[myid=1]-28] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 3
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.12:50246
01:35:15.335 [QuorumConnectionThread-[myid=1]-27] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2)
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:35:15.335 [QuorumConnectionThread-[myid=1]-28] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 3 using election address: /172.18.0.13:3888
01:35:15.335 [QuorumConnectionThread-[myid=1]-28] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:3)
01:35:15.336 [QuorumConnectionThread-[myid=1]-28] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:3)
01:35:34.600 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.11:49750
01:35:34.601 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.11:49750
01:35:34.601 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:35:34.601 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:35:59.199 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.13:56956
01:35:59.200 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.13:56956
01:35:59.200 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:35:59.200 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.# zk03：
01:36:59.201 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:36:59.201 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:36:59.202 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 2 already, it is in the lastCommittedView
01:36:59.202 [QuorumConnectionThread-[myid=1]-15] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 2
01:36:59.203 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 3 already, it is in the lastCommittedView
01:36:59.203 [QuorumConnectionThread-[myid=1]-15] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 2 using election address: /172.18.0.12:3888
01:36:59.203 [QuorumConnectionThread-[myid=1]-15] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:2)
01:36:59.203 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] INFO org.apache.zookeeper.server.quorum.FastLeaderElection - Notification time out: 60000 ms
01:36:59.203 [QuorumConnectionThread-[myid=1]-16] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 3
01:36:59.203 [QuorumConnectionThread-[myid=1]-15] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2)
01:36:59.204 [QuorumConnectionThread-[myid=1]-16] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 3 using election address: /172.18.0.13:3888
01:36:59.204 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.13:35192
01:36:59.204 [QuorumConnectionThread-[myid=1]-16] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:3)
01:36:59.204 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.13:35192
01:36:59.204 [QuorumConnectionThread-[myid=1]-16] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:3)
01:36:59.204 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:36:59.205 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.12:35956
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.12:35956
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.11:46420
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.11:46420
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
找到原因了，因为data/myid没有修改，目前3个节点的配置全部为1，分别改为1、2、3就正常了。
启动zkui
docker run -d --name zkui -p 9090:9090 --network net-kafka -e ZKLIST=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 maauso/zkui访问localhost:9090，登录之后报错：KeeperErrorCode = ConnectionLoss for /
删除data\version-2\snapshot.0依然不行
stat is not executed because it is not in the whitelist.
envi is not executed because it is not in the whitelist.
4lw.commands.whitelist=*
16:49:58.466 [NIOWorkerThread-10] DEBUG org.apache.zookeeper.server.NIOServerCnxn - Closed socket connection for client /172.18.0.2:42978 (no session established for client)
16:49:58.499 [NIOServerCxnFactory.AcceptThread:/0.0.0.0:2181] DEBUG org.apache.zookeeper.server.NIOServerCnxnFactory - Accepted socket connection from /172.18.0.2:42980
16:49:58.500 [NIOWorkerThread-11] WARN org.apache.zookeeper.server.NIOServerCnxn - Close of session 0x0
java.io.IOException: ZooKeeperServer not runningat org.apache.zookeeper.server.NIOServerCnxn.readLength(NIOServerCnxn.java:569)at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:349)at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:508)at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:153)at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)at java.base/java.lang.Thread.run(Unknown Source)好像改为zk的data/myid配置，上面的问题就没有了。
2、Docker部署kafka集群
kafka没有官方镜像，使用wurstmeister/kafka镜像
2.1、主机创建目录
[kafka01]
D:/mysoft/dev/docker-kafka-cluster/kafka01/config
D:/mysoft/dev/docker-kafka-cluster/kafka01/data
D:/mysoft/dev/docker-kafka-cluster/kafka01/logs
D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts[kafka02]
D:/mysoft/dev/docker-kafka-cluster/kafka02/config
D:/mysoft/dev/docker-kafka-cluster/kafka02/data
D:/mysoft/dev/docker-kafka-cluster/kafka02/logs
D:/mysoft/dev/docker-kafka-cluster/kafka02/hosts[kafka03]
D:/mysoft/dev/docker-kafka-cluster/kafka03/config
D:/mysoft/dev/docker-kafka-cluster/kafka03/data
D:/mysoft/dev/docker-kafka-cluster/kafka03/logs
D:/mysoft/dev/docker-kafka-cluster/kafka03/hosts2.2、修改配置文件
分别拷贝到kafka01、kafka02、kafka03
[server.properties]broker.id=1
advertised.listeners=PLAINTEXT://172.18.0.21:9092
# port=9092
# log.dirs=/kafka/kafka-logs-1ddafafe54d0
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:21812.3、指定ip创建启动容器
# docker run -d --name kafka01 --restart=always --net=host -v D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts:/etc/hosts -v D:/mysoft/dev/docker-kafka-cluster/kafka01/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka01/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka01/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_HOST_NAME=192.168.180.42 -e JMX_PORT=9999 -e HOST_IP=127.0.0.1 -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_ZOOKEEPER_CONNECT=127.0.0.1:2181 -e KAFKA_BROKER_ID=1 -e KAFKA_HEAP_OPTS="-Xmx6G -Xms6G" wurstmeister/kafka
# docker run -d --name kafka01 --restart=always --network net-kafka --ip 172.18.0.21 -v D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka01/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka01/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka01/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_HOST_NAME=172.18.0.21 -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.21:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=1 -e KAFKA_LOG_DIRS=/opt/kafka/logs wurstmeister/kafkadocker run -d --name kafka01 -p 9192:9092 --restart=always --network net-kafka --ip 172.18.0.21 -v D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka01/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka01/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka01/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.21:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=1 wurstmeister/kafkadocker run -d --name kafka02 -p 9292:9092 --restart=always --network net-kafka --ip 172.18.0.22 -v D:/mysoft/dev/docker-kafka-cluster/kafka02/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka02/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka02/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka02/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.22:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=2 wurstmeister/kafkadocker run -d --name kafka03 -p 9392:9092 --restart=always --network net-kafka --ip 172.18.0.23 -v D:/mysoft/dev/docker-kafka-cluster/kafka03/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka03/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka03/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka03/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.23:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=3 wurstmeister/kafkakafka容器异常，一直自动反复重启，UP状态维持1分钟左右
[Configuring] 'advertised.listeners' in '/opt/kafka/config/server.properties'
[Configuring] 'port' in '/opt/kafka/config/server.properties'
Excluding KAFKA_HOME from broker config
[Configuring] 'log.dirs' in '/opt/kafka/config/server.properties'
[Configuring] 'listeners' in '/opt/kafka/config/server.properties'
Excluding KAFKA_VERSION from broker config
[Configuring] 'zookeeper.connect' in '/opt/kafka/config/server.properties'
[Configuring] 'broker.id' in '/opt/kafka/config/server.properties'
log4j:ERROR Could not read configuration file from URL [file:/opt/kafka/bin/../config/log4j.properties].
java.io.FileNotFoundException: /opt/kafka/bin/../config/log4j.properties (No such file or directory)at java.base/java.io.FileInputStream.open0(Native Method)at java.base/java.io.FileInputStream.open(Unknown Source)at java.base/java.io.FileInputStream.<init>(Unknown Source)at java.base/java.io.FileInputStream.<init>(Unknown Source)at java.base/sun.net.www.protocol.file.FileURLConnection.connect(Unknown Source)at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(Unknown Source)at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:557)at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)at com.typesafe.scalalogging.Logger$.apply(Logger.scala:48)at kafka.utils.Log4jControllerRegistration$.<clinit>(Logging.scala:25)at kafka.Kafka$.<clinit>(Kafka.scala:30)at kafka.Kafka.main(Kafka.scala)
log4j:ERROR Ignoring configuration file [file:/opt/kafka/bin/../config/log4j.properties].
log4j:WARN No appenders could be found for logger (kafka.utils.Log4jControllerRegistration$).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.指定-e KAFKA_LOG_DIRS也依然不可以，手动拷贝一份log4j.properties到映射的目录D:/mysoft/dev/docker-kafka-cluster/kafka01/logs下问题就解决了。
但是出现另外一个异常：
[2023-12-23 15:20:50,136] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,204] INFO Socket connection established, initiating session, client: /172.18.0.21:39814, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,273] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,749] INFO Opening socket connection to server zk03.net-kafka/172.18.0.13:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,811] INFO Socket connection established, initiating session, client: /172.18.0.21:49980, server: zk03.net-kafka/172.18.0.13:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,914] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,268] INFO Opening socket connection to server zk02.net-kafka/172.18.0.12:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,356] INFO Socket connection established, initiating session, client: /172.18.0.21:40344, server: zk02.net-kafka/172.18.0.12:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,446] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,617] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2023-12-23 15:20:52,817] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,855] INFO Socket connection established, initiating session, client: /172.18.0.21:39828, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:53,036] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2023-12-23 15:20:53,037] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:53,076] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2023-12-23 15:20:53,222] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTINGat kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:271)at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:125)at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1948)at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:431)at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:456)at kafka.server.KafkaServer.startup(KafkaServer.scala:191)at kafka.Kafka$.main(Kafka.scala:109)at kafka.Kafka.main(Kafka.scala)
[2023-12-23 15:20:53,290] INFO shutting down (kafka.server.KafkaServer)
[2023-12-23 15:20:53,375] INFO App info kafka.server for 1 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2023-12-23 15:20:53,440] INFO shut down completed (kafka.server.KafkaServer)
[2023-12-23 15:20:53,515] ERROR Exiting Kafka. (kafka.Kafka$)
[2023-12-23 15:20:53,590] INFO shutting down (kafka.server.KafkaServer)回头看zk的控制台，发现报错java.io.IOException: ZooKeeperServer not running，跟用zkui客户端连zk集群一样的错，通过修改zk的data/myid配置，之后，3个kafka服务都能正常启动了，而且也不会反复自动重启了。
# kafka01
[2023-12-25 02:08:01,176] INFO [KafkaServer id=1] started (kafka.server.KafkaServer)# kafka02
[2023-12-25 02:09:56,391] INFO [KafkaServer id=2] started (kafka.server.KafkaServer)# kafka03
[2023-12-25 02:10:14,479] INFO [KafkaServer id=3] started (kafka.server.KafkaServer)虽然kafka服务在启动的初始阶段，仍然有一下异常，但是目前没有发现有什么影响：
[2023-12-25 00:52:15,882] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:15,932] INFO Socket connection established, initiating session, client: /172.18.0.21:49568, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:15,984] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,040] INFO Opening socket connection to server zk03.net-kafka/172.18.0.13:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,094] INFO Socket connection established, initiating session, client: /172.18.0.21:59734, server: zk03.net-kafka/172.18.0.13:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,148] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,979] INFO Opening socket connection to server zk02.net-kafka/172.18.0.12:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:18,033] INFO Socket connection established, initiating session, client: /172.18.0.21:50098, server: zk02.net-kafka/172.18.0.12:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:18,088] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,308] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,357] INFO Socket connection established, initiating session, client: /172.18.0.21:49580, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,403] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,974] INFO Opening socket connection to server zk03.net-kafka/172.18.0.13:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,027] INFO Socket connection established, initiating session, client: /172.18.0.21:59746, server: zk03.net-kafka/172.18.0.13:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,075] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,549] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient) [2023-12-25 00:52:20,812] INFO Opening socket connection to server zk02.net-kafka/172.18.0.12:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,860] INFO Socket connection established, initiating session, client: /172.18.0.21:50110, server: zk02.net-kafka/172.18.0.12:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:21,016] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper) [2023-12-25 00:52:21,017] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:21,062] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient) [2023-12-25 00:52:21,164] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTINGat kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:271)at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:125)at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1948)at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:431)at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:456)at kafka.server.KafkaServer.startup(KafkaServer.scala:191)at kafka.Kafka$.main(Kafka.scala:109)at kafka.Kafka.main(Kafka.scala) [2023-12-25 00:52:21,213] INFO shutting down (kafka.server.KafkaServer) [2023-12-25 00:52:21,274] INFO App info kafka.server for 1 unregistered (org.apache.kafka.common.utils.AppInfoParser) [2023-12-25 00:52:21,319] INFO shut down completed (kafka.server.KafkaServer) [2023-12-25 00:52:21,371] ERROR Exiting Kafka. (kafka.Kafka$) [2023-12-25 00:52:21,425] INFO shutting down (kafka.server.KafkaServer)
启动sheepkiller/kafka-manager
# docker run -d --name kafka-manager -p 9000:9000 --network net-kafka --ip 172.18.0.31 -e ZK_HOSTS=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 sheepkiller/kafka-managerdocker run -d --name kafka-manager -p 9000:9000 --network net-kafka -e ZK_HOSTS=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 sheepkiller/kafka-manager
或者使用provectuslabs/kafka-ui:latest
# docker run -d --name kafka-ui -p 8080:8080 --network net-kafka --ip 172.18.0.32 -e DYNAMIC_CONFIG_ENABLED=true provectuslabs/kafka-uidocker run -d --name kafka-ui -p 8080:8080 --network net-kafka -e DYNAMIC_CONFIG_ENABLED=true provectuslabs/kafka-ui
3、完整配置
3.1 zk完整配置
zk01/conf/zoo.cfg
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181
4lw.commands.whitelist=*zk01/data/myid
1zk02/conf/zoo.cfg
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181
4lw.commands.whitelist=*zk02/data/myid
2zk03/conf/zoo.cfg
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181
4lw.commands.whitelist=*zk03/data/myid
33.2 kafka完整配置
kafka01/config/server.properties
broker.id=1
advertised.listeners=PLAINTEXT://172.18.0.21:9092
port=9092
log.dirs=/kafka/kafka-logs-7725cecb8a4f
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181kafka02/config/server.properties
broker.id=2
advertised.listeners=PLAINTEXT://172.18.0.22:9092
port=9092
log.dirs=/kafka/kafka-logs-2b3d2a64f445
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181kafka03/config/server.properties
broker.id=3
advertised.listeners=PLAINTEXT://172.18.0.23:9092
port=9092
log.dirs=/kafka/kafka-logs-d9101a867f99
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:21814、问题
SpringBoot配置：
spring:kafka:bootstrap-servers: localhost:9192,localhost:9292,localhost:9392报错，与单机版错误一致
2023-12-26 15:27:29.296 INFO 11904 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka version: 2.8.1
2023-12-26 15:27:29.298 INFO 11904 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka commitId: 839b886f9b732b15
2023-12-26 15:27:29.298 INFO 11904 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka startTimeMs: 1703575649293
2023-12-26 15:27:29.302 INFO 11904 --- [ main] o.a.k.clients.consumer.KafkaConsumer : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Subscribed to topic(s): test-topic032023-12-26 15:27:30.191 INFO 11904 --- [ntainer#0-0-C-1] org.apache.kafka.clients.Metadata : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Cluster ID: o6TS0_kER-KgmyB7NWjjEw
2023-12-26 15:27:30.194 INFO 11904 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Discovered group coordinator 172.18.0.23:9092 (id: 2147483644 rack: null)
2023-12-26 15:27:30.200 INFO 11904 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] (Re-)joining group
2023-12-26 15:27:38.804 INFO 11904 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Group coordinator 172.18.0.23:9092 (id: 2147483644 rack: null) is unavailable or invalid due to cause: null.isDisconnected: true. Rediscovery will be attempted.相关文章：
docker部署kafka zookeeper模式集群
单机模式链接：https://blog.csdn.net/wsdhla/article/details/133032238 kraft集群模式链接：部署Kafka_kafka 部署-CSDN博客 zookeeper选举机制举例： 目前有5台服务器，每台服务器均没有数据，它们的编号分别是1,2,3,4,5…...
Apache Flink连载（二十一）：Flink On Yarn运行原理-Yarn Application模式
🏡 个人主页：IT贫道_大数据OLAP体系技术栈,Apache Doris,Clickhouse 技术-CSDN博客 🚩 私聊博主：加入大数据技术讨论群聊，获取更多大数据资料。 🔔 博主个人B栈地址：豹哥教你大数据的个人空间-豹哥教你大数据个人主页-哔哩哔哩视频 目录 1. 任务提交命令...
《深入理解C++11：C++11新特性解析与应用》笔记五
第五章 提高类型安全 5.1 强类型枚举 5.1.1 枚举：分门别类与数值的名字 具名枚举类型一般声明类似：enum Gender { Male, Female }。 匿名枚举类型可以使用三种方式实现： 第一种方式时宏，比如 #define Male 0 #define Femal…...
Y9000P + ubuntu22.04 配置Anaconda+pycharm +pytorch
Anaconda3 的安装及使用方法安装 Anaconda3 Anaconda3 是 Anaconda 的具体版本 Anaconda3 中的 Python 解释器默认使用的是 Python3.x 版本，而不是 Python2.x 版本 Python2.x 版本中，字符串是以 ASCII 编码处理的，而在 Python3.x 版本中&am…...
使用Ubuntu编译FFmpeg生成Android动态库/静态库
环境 我这里使用windows里的wsl2的ubuntu，使用物理机或者vmware，vbox之类的安装的ubuntu理论上也可以. gcc编译使用的环境如下: Ndk使用17 FFmpeg使用4.0.2. clang编译使用的环境如下: Ndk使用21.4 FFmpeg使用6.1 FFmpeg下载地址:https://ffmpeg.org/…...
【AIGC-图片生成视频系列-2】八仙过海，各显神通：AI生成视频相关汇总剖析
最近「图片生成视频系列」层出不穷，我拜读并结合实践（对，就是手撕代码，有开源就撕），并对以下几篇文章的相似点以及关键点稍微做个总结： 一. 生成视频中图像的一致性 在图像生成视频的这个过程…...
SpringBoot集成RabbitMq消息队列【附源码】
1. 项目背景 要啥项目背景，就是干！！！ SpringBoot版本：2.7.12 2. Rabbit MQ安装 这里讲解使用docker安装RabbitMQ，如果在windows下面安装RabbitMQ，参考下文 【笑小枫的按步照搬系列】Window…...
MySQL数据库的安装与环境配置
下载 下载MySQL8 安装 解压 配置MySQL环境变量 系统环境变量path D:\ProgramFiles\mysql-8.0.20-winx64\bin 1.点击属性 2.点击高级系统设置 3.点击环境变量 4.在系统变量中找到path 注意这里不是用户变量 5.新建后输入解压的地址 MySQL初始化和启动 以管理员身份运行cmd…...
【广州华锐互动】VR科技科普展厅平台：快速、便捷地创建出属于自己的虚拟展馆
随着科技的不断进步，虚拟现实(VR)技术已经在许多领域取得了显著的成果。尤其是在展馆设计领域，VR科技科普展厅平台已经实现了许多令人瞩目的新突破。 VR科技科普展厅平台是广州华锐互动专门为企业和机构提供虚拟展馆设计和制作的在线平台。通过这个平台&…...
XML Extension Supplement
LEGAL ISSUES, COMPANY POLICIES AND STANDARDS Web Services A Web service is a software system designed to support interoperable machine-to-machine interaction over a network. URI和URL URI，全称是统一资源标识符（Uniform Resource Ident…...
手拉手Springboot获取yml配置文件信息
环境介绍 技术栈 springboot3 软件 版本 mysql 8 IDEA IntelliJ IDEA 2022.2.1 JDK 17 Spring Boot 3.1.7 配置文件说明：启动配置文件优先级：properties高于yml 配置文件application.yml yml是 JSON 的超集，简洁而强大…...
行人重识别(ReID)基础知识入门
这里写目录标题 1、ReID技术概述1.1 基本原理1.2 实现流程1.3 重识别存在的技术挑战 2、训练数据格式介绍 1、ReID技术概述 1.1 基本原理 ReID，全称Re-identification，目的是利用各种智能算法在图像数据库中找到与要搜索的目标相似的对象。ReID是图像检…...
【音视频 ffmpeg 学习】 跑示例程序 持续更新中
环境准备 在上一篇文章 把mux.c 拷贝到main.c 中 使用 attribute(unused) 消除警告 __attribute__(unused)/** Copyright (c) 2003 Fabrice Bellard** Permission is hereby granted, free of charge, to any person obtaining a copy* of this software and associated docu…...
前端axios与python库requests的区别
当涉及到发送HTTP请求时，Axios和Python中的requests库都是常用的工具。下面是它们的详细说明： Axios： Axios是一个基于Promise的HTTP客户端，主要用于浏览器和Node.js环境中发送HTTP请求。以下是Axios的一些特点和用法࿱…...
达梦数据库文档
1：达梦数据库(DM8)简介 达梦数据库管理系统是武汉达梦公司推出的具有完全自主知识产权的高性能数据库管理系统，简称DM。达梦数据库管理系统目前最新的版本是8.0版本，简称DM8。 DM8是达梦公司在总结DM系列产品研发与应用经验的基础上…...
CorelDRAW2024新功能有哪些？CorelDRAW2024最新版本更新怎么样？
CorelDRAW2024新功能有哪些？CorelDRAW2024最新版本更新怎么样？让我们带您详细了解！ CorelDRAW Graphics Suite 是矢量制图行业的标杆软件，2024年全新版本为您带来多项新功能和优化改进。本次更新强调易用性，包括更强大…...
基于Mapify的在线艺术地图设计
地图是传递空间信息的有效载体，更加美观、生动的地图产品也是我们追求目标。 那么，我们如何才能制出如下图所示这样一幅艺术性较高的地图呢？今天我们来一探究竟吧！ 按照惯例，现将网址给出： https://www.m…...
mxxWechatBot微信机器人V2版本文档说明
大家伙，我是雄雄，欢迎关注微信公众号：雄雄的小课堂。 先看这里 一、前言二、mxxWechatBot流程图三、怎么使用？ 一、前言 经过不断地探索与研究，mxxWechatBot正式上线，届时全面开放使用。 mxxWechatBot&am…...
红队打靶练习:MISDIRECTION: 1
信息收集 1、arp ┌──(root㉿ru)-[~/kali] └─# arp-scan -l Interface: eth0, type: EN10MB, MAC: 00:0c:29:69:c7:bf, IPv4: 192.168.12.128 Starting arp-scan 1.10.0 with 256 hosts (https://github.com/royhills/arp-scan) 192.168.12.1 00:50:56:c0:00:08 …...
Jmeter吞吐量控制器总结
吞吐量控制器(Throughput Controller) 场景： 在同一个线程组里, 有10个并发, 7个做A业务, 3个做B业务,要模拟这种场景,可以通过吞吐量模拟器来实现。 添加吞吐量控制器 用法1: Percent Executions 在一个线程组内分别建立两个吞吐量控制器, 分别放业务A和业务B …...
【XML】TinyXML 详解（二）：接口详解
【C】郭老二博文之：C目录 1、XML测试文件（laoer.xml） <?xml version"1.0" standalone"no" ?> <!-- Hello World !--> <root><child name"childName" id"1"><c_child…...
【机器学习】人工智能概述
人工智能（Artificial Intelligence，简称AI）是一门研究如何使机器能够像人一样思考、学习和执行任务的学科。它是计算机科学的一个重要分支，涉及机器学习、自然语言处理、计算机视觉等多个领域。 人工智能的概念最早可以追溯到20世…...
flink 实时写入 hudi 参数推荐
数据湖任务并行度计算...
传统项目基于tomcat cookie单体会话升级分布式会话解决方案
传统捞项目基于servlet容器 cookie单体会话改造分布式会话方案 ##引入redis,spring-session依赖 <!--redis依赖 --><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-redis</artifactId>&…...
Unity 关于json数据的解析方式（LitJson.dll插件）
关于json数据的解析方式（LitJson.dll插件） void ParseItemJson(){TextAsset itemText Resources.Load<TextAsset>("Items");//读取Resources中Items文件，需要将Items文件放到Resources文件夹中string itemJson itemText.te…...
智能监控平台/视频共享融合系统EasyCVR海康设备国标GB28181接入流程
TSINGSEE青犀视频监控汇聚平台EasyCVR可拓展性强、视频能力灵活、部署轻快，可支持的主流标准协议有国标GB28181、RTSP/Onvif、RTMP等，以及支持厂家私有协议与SDK接入，包括海康Ehome、海大宇等设备的SDK等。平台既具备传统安防视频监控的能力&…...
expdp到ASM 文件系统 并拷贝
1.创建asm导出数据目录 sql>select name,total_mb,free_mb from v$asm_diskgroup; 确认集群asm磁盘组环境 asmcmd>cd DGDSDB asmcmd>mkdir dpbak asmcmd>ls -l sql>conn / as sysdba create directory expdp_asm_dir as DGDSDB/dpbak; create directory expdp_l…...
【2023】通过docker安装hadoop以及常见报错
💻目录 1、准备2、安装镜像2.1、创建centos-ssh的镜像2.2、创建hadoop的镜像 3、配置ssh网络3.1、搭建同一网段的网络3.2、配置host实现互相之间可以免密登陆3.3、查看是否成功 4、安装配置Hadoop4.1、添加存储文件夹4.2、添加指定配置4.3、同步数据 5、测试启动5.1…...
Baumer工业相机堡盟工业相机如何通过NEOAPI SDK获取相机当前实时帧率（C++）
Baumer工业相机堡盟工业相机如何通过NEOAPI SDK获取相机当前实时帧率（C） Baumer工业相机Baumer工业相机的帧率的技术背景Baumer工业相机的帧率获取方式CameraExplorer如何查看相机帧率信息在NEOAPI SDK里通过函数获取相机帧率（C） …...
SpringBoot项目部署及多环境
1、多环境 2、项目部署上线 原始前端 / 后端项目宝塔Linux容器容器平台 3、前后端联调 4、项目扩展和规划 多环境 程序员鱼皮-参考文章 本地开发：localhost（127.0.0.1） 多环境：指同一套项目代码在把不同的阶段需要根据实际…...
视频解析网站如何做搜索/网站怎样才能在百度被搜索到
alibaba/p3c github地址 : https://github.com/alibaba/p3c可以直接clone到本地(含阿里巴巴Java开发手册(终极版).pdf)主要说说eclipse安装1.在线安装方法Help >> Install New Software 然后输入下面的地址 https://p3c.alibaba.com/plugi...等下面的列表加载出来,直接下…...
seo外贸网站建设/东营网站seo
最近，遇到一个问题，安装完成VMware 11之后， VMware Network Adapter VMnet1 没有生成，只有VMware Network Adapter VMnet8，上网找了一下，解决办法如下：编辑--虚拟机网络编辑器--还原默认设置&am…...
小型购物网站开发/苏州百度推广公司
ESC 退出 0 进度条开关 1 屏幕原始大小 2 屏幕1/2大小 3 屏幕1/3大小 4 屏幕1/4大小 S 下一帧 [ -2秒 ] 2秒 ; -1秒1秒 < -0.05秒 > 下一个帧 -> -5秒ffmpeg-20160811-bin.7z转载于:https://www.cnblo…...
专业的单位网站开发/广告公司营销策划方案
1. UDP概述 UDP:User Datagram Protocol 用户数据报协议 2. UDP 有如下的特点： 无连接、不可靠 无连接：意思就是在通讯之前不需要建立连接，直接传输数据。 不可靠：是将数据报的分组从一台主机发送到另一台主机，但并不保证数据报能够到达另一端，任何必须的可靠性都由应用…...
公司为什么建立网站/整合营销策略有哪些
关于讲理老公：你不讲理。老婆：和你我从来就没讲过理，家就不是讲理的地方。再说你是男的，还比我大8个月呢，你就得让着我。关于钱老公：以后我挣的钱，按比例给你吧，我挣的多时留得也多一…...
三级 做爰免费网站/优化科技
写在前面的话 通过上一节，知道了 K8S 有 Master / Node 组成，但是具体怎么个组成法，就是这一节具体谈的内容。概念性的东西我们会尽量以实验的形式将其复现。 部署 K8S 集群 互联网常用的 K8S 集群部署方式： 1. kubeadm࿰…...