docker部署kafka zookeeper模式集群
单机模式链接：https://blog.csdn.net/wsdhla/article/details/133032238
kraft集群模式链接：部署Kafka_kafka 部署-CSDN博客
zookeeper选举机制举例：
目前有5台服务器，每台服务器均没有数据，它们的编号分别是1,2,3,4,5,按编号依次启动，它们的选择举过程如下：
- 服务器1启动，给自己投票，然后发投票信息，由于其它机器还没有启动所以它收不到反馈信息，服务器1的状态一直属于Looking(选举状态)。
- 服务器2启动，给自己投票，同时与之前启动的服务器1交换结果，由于服务器2的编号大所以服务器2胜出，但此时投票数没有大于半数，所以两个服务器的状态依然是LOOKING。
- 服务器3启动，给自己投票，同时与之前启动的服务器1,2交换信息，由于服务器3的编号最大所以服务器3胜出，此时投票数正好大于半数，所以服务器3成为领导者，服务器1,2成为小弟。
- 服务器4启动，给自己投票，同时与之前启动的服务器1,2,3交换信息，尽管服务器4的编号大，但之前服务器3已经胜出，所以服务器4只能成为小弟。
- 服务器5启动，后面的逻辑同服务器4成为小弟。
Docker 自定义静态网络
Docker默认提供了3种网络模式，生成容器时不指定网络模式下默认使用bridge桥接模式。每次容器启动时ip会发生变更，无法达到我们集群的效果。
创建自定义网络模式，指定网段。
docker network create --subnet=172.18.0.0/16 net-kafka若报错：Error response from daemon: Pool overlaps with other one on this address space
说明指定的网络地址段已存在
1、Docker部署zookeeper集群
zk使用官方镜像：
1.1、主机创建目录
[zk01]
D:/mysoft/dev/docker-zk-cluster/zk01/conf
D:/mysoft/dev/docker-zk-cluster/zk01/data
D:/mysoft/dev/docker-zk-cluster/zk01/datalog[zk02]
D:/mysoft/dev/docker-zk-cluster/zk02/conf
D:/mysoft/dev/docker-zk-cluster/zk02/data
D:/mysoft/dev/docker-zk-cluster/zk02/datalog[zk03]
D:/mysoft/dev/docker-zk-cluster/zk03/conf
D:/mysoft/dev/docker-zk-cluster/zk03/data
D:/mysoft/dev/docker-zk-cluster/zk03/datalog1.2、修改配置文件
分别拷贝到zk01、zk02、zk03
[zoo.cfg]dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;21811.3、指定ip创建启动容器
docker run -d --name zk01 -p 2181:2181 --network net-kafka --ip 172.18.0.11 -v D:/mysoft/dev/docker-zk-cluster/zk01/conf:/conf -v D:/mysoft/dev/docker-zk-cluster/zk01/data:/data -v D:/mysoft/dev/docker-zk-cluster/zk01/datalog:/datalog zookeeperdocker run -d --name zk02 -p 2182:2181 --network net-kafka --ip 172.18.0.12 -v D:/mysoft/dev/docker-zk-cluster/zk02/conf:/conf -v D:/mysoft/dev/docker-zk-cluster/zk02/data:/data -v D:/mysoft/dev/docker-zk-cluster/zk02/datalog:/datalog zookeeperdocker run -d --name zk03 -p 2183:2181 --network net-kafka --ip 172.18.0.13 -v D:/mysoft/dev/docker-zk-cluster/zk03/conf:/conf -v D:/mysoft/dev/docker-zk-cluster/zk03/data:/data -v D:/mysoft/dev/docker-zk-cluster/zk03/datalog:/datalog zookeeperzk01可以正常启动，zk02和zk03启动失败
警告：这个正常，是因为节点启动时选举机制的原因
zk02：
WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 2 at election address /172.18.0.12:3888
zk03：
WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
异常：
zk02：
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Exception while listening to address /172.18.0.11:3888
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Leaving listener thread for address 172.18.0.11:3888 after 3 errors. Use zookeeper.electionPortBindRetry property to increase retry count.
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - As I'm leaving the listener thread, I won't be able to participate in leader election any longer: 172.18.0.11:3888
zk03：
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Exception while listening to address /172.18.0.11:3888
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Leaving listener thread for address 172.18.0.11:3888 after 3 errors. Use zookeeper.electionPortBindRetry property to increase retry count.
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - As I'm leaving the listener thread, I won't be able to participate in leader election any longer: 172.18.0.11:3888
原因是没有配置quorumListenOnAllIPs，默认为false，需要设置为true
[zoo.cfg]dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181quorumListenOnAllIPs：
当设置为true时，ZooKeeper服务器将在所有可用IP地址上侦听来自其对等方的连接，而不仅是在配置文件的服务器列表中配置的地址。它会影响处理ZAB协议和快速领导者选举协议的连接。默认值为false。
有一些WARN:
### 1、只启动zk01：
# zk01：
01:21:17.746 [QuorumConnectionThread-[myid=1]-3] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 2 at election address /172.18.0.12:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)01:21:20.904 [QuorumConnectionThread-[myid=1]-1] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)### 2、 只启动zk01、zk02：
# zk01：
01:21:46.504 [QuorumConnectionThread-[myid=1]-3] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)# zk02：
01:21:43.350 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.01:21:48.824 [QuorumConnectionThread-[myid=1]-2] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)### 2、 启动zk01、zk02、zk03：
# zk01正常：
01:31:34.579 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:31:34.579 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:31:34.580 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 2 already, it is in the lastCommittedView
01:31:34.584 [QuorumConnectionThread-[myid=1]-21] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 2
01:31:34.584 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 3 already, it is in the lastCommittedView
01:31:34.585 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] INFO org.apache.zookeeper.server.quorum.FastLeaderElection - Notification time out: 60000 ms
01:31:34.585 [QuorumConnectionThread-[myid=1]-22] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 3
01:31:34.585 [QuorumConnectionThread-[myid=1]-21] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 2 using election address: /172.18.0.12:3888
01:31:34.585 [QuorumConnectionThread-[myid=1]-21] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:2)
01:31:34.586 [QuorumConnectionThread-[myid=1]-21] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2)
01:31:34.585 [QuorumConnectionThread-[myid=1]-22] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 3 using election address: /172.18.0.13:3888
01:31:34.586 [QuorumConnectionThread-[myid=1]-22] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:3)
01:31:34.586 [QuorumConnectionThread-[myid=1]-22] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:3)
01:31:52.173 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - node0 scavenging sessions
01:31:52.174 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - org.eclipse.jetty.server.session.SessionHandler873610597==dftMaxIdleSec=-1 scavenging sessions
01:31:52.174 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - org.eclipse.jetty.server.session.SessionHandler873610597==dftMaxIdleSec=-1 scavenging session ids []
01:31:52.174 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - org.eclipse.jetty.server.session.DefaultSessionCache@57d7f8ca[evict=-1,removeUnloadable=false,saveOnCreate=false,saveOnInactiveEvict=false] checking expiration on []# zk02：
01:35:15.331 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:35:15.332 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:35:15.333 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 2 already, it is in the lastCommittedView
01:35:15.334 [QuorumConnectionThread-[myid=1]-27] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 2
01:35:15.334 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 3 already, it is in the lastCommittedView
01:35:15.335 [QuorumConnectionThread-[myid=1]-27] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 2 using election address: /172.18.0.12:3888
01:35:15.335 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] INFO org.apache.zookeeper.server.quorum.FastLeaderElection - Notification time out: 60000 ms
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.12:50246
01:35:15.335 [QuorumConnectionThread-[myid=1]-27] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:2)
01:35:15.335 [QuorumConnectionThread-[myid=1]-28] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 3
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.12:50246
01:35:15.335 [QuorumConnectionThread-[myid=1]-27] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2)
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:35:15.335 [QuorumConnectionThread-[myid=1]-28] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 3 using election address: /172.18.0.13:3888
01:35:15.335 [QuorumConnectionThread-[myid=1]-28] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:3)
01:35:15.336 [QuorumConnectionThread-[myid=1]-28] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:3)
01:35:34.600 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.11:49750
01:35:34.601 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.11:49750
01:35:34.601 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:35:34.601 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:35:59.199 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.13:56956
01:35:59.200 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.13:56956
01:35:59.200 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:35:59.200 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.# zk03：
01:36:59.201 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:36:59.201 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:36:59.202 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 2 already, it is in the lastCommittedView
01:36:59.202 [QuorumConnectionThread-[myid=1]-15] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 2
01:36:59.203 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 3 already, it is in the lastCommittedView
01:36:59.203 [QuorumConnectionThread-[myid=1]-15] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 2 using election address: /172.18.0.12:3888
01:36:59.203 [QuorumConnectionThread-[myid=1]-15] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:2)
01:36:59.203 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] INFO org.apache.zookeeper.server.quorum.FastLeaderElection - Notification time out: 60000 ms
01:36:59.203 [QuorumConnectionThread-[myid=1]-16] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 3
01:36:59.203 [QuorumConnectionThread-[myid=1]-15] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2)
01:36:59.204 [QuorumConnectionThread-[myid=1]-16] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 3 using election address: /172.18.0.13:3888
01:36:59.204 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.13:35192
01:36:59.204 [QuorumConnectionThread-[myid=1]-16] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:3)
01:36:59.204 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.13:35192
01:36:59.204 [QuorumConnectionThread-[myid=1]-16] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:3)
01:36:59.204 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:36:59.205 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.12:35956
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.12:35956
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.11:46420
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.11:46420
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
找到原因了，因为data/myid没有修改，目前3个节点的配置全部为1，分别改为1、2、3就正常了。
启动zkui
docker run -d --name zkui -p 9090:9090 --network net-kafka -e ZKLIST=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 maauso/zkui访问localhost:9090，登录之后报错：KeeperErrorCode = ConnectionLoss for /
删除data\version-2\snapshot.0依然不行
stat is not executed because it is not in the whitelist.
envi is not executed because it is not in the whitelist.
4lw.commands.whitelist=*
16:49:58.466 [NIOWorkerThread-10] DEBUG org.apache.zookeeper.server.NIOServerCnxn - Closed socket connection for client /172.18.0.2:42978 (no session established for client)
16:49:58.499 [NIOServerCxnFactory.AcceptThread:/0.0.0.0:2181] DEBUG org.apache.zookeeper.server.NIOServerCnxnFactory - Accepted socket connection from /172.18.0.2:42980
16:49:58.500 [NIOWorkerThread-11] WARN org.apache.zookeeper.server.NIOServerCnxn - Close of session 0x0
java.io.IOException: ZooKeeperServer not runningat org.apache.zookeeper.server.NIOServerCnxn.readLength(NIOServerCnxn.java:569)at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:349)at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:508)at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:153)at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)at java.base/java.lang.Thread.run(Unknown Source)好像改为zk的data/myid配置，上面的问题就没有了。
2、Docker部署kafka集群
kafka没有官方镜像，使用wurstmeister/kafka镜像
2.1、主机创建目录
[kafka01]
D:/mysoft/dev/docker-kafka-cluster/kafka01/config
D:/mysoft/dev/docker-kafka-cluster/kafka01/data
D:/mysoft/dev/docker-kafka-cluster/kafka01/logs
D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts[kafka02]
D:/mysoft/dev/docker-kafka-cluster/kafka02/config
D:/mysoft/dev/docker-kafka-cluster/kafka02/data
D:/mysoft/dev/docker-kafka-cluster/kafka02/logs
D:/mysoft/dev/docker-kafka-cluster/kafka02/hosts[kafka03]
D:/mysoft/dev/docker-kafka-cluster/kafka03/config
D:/mysoft/dev/docker-kafka-cluster/kafka03/data
D:/mysoft/dev/docker-kafka-cluster/kafka03/logs
D:/mysoft/dev/docker-kafka-cluster/kafka03/hosts2.2、修改配置文件
分别拷贝到kafka01、kafka02、kafka03
[server.properties]broker.id=1
advertised.listeners=PLAINTEXT://172.18.0.21:9092
# port=9092
# log.dirs=/kafka/kafka-logs-1ddafafe54d0
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:21812.3、指定ip创建启动容器
# docker run -d --name kafka01 --restart=always --net=host -v D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts:/etc/hosts -v D:/mysoft/dev/docker-kafka-cluster/kafka01/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka01/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka01/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_HOST_NAME=192.168.180.42 -e JMX_PORT=9999 -e HOST_IP=127.0.0.1 -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_ZOOKEEPER_CONNECT=127.0.0.1:2181 -e KAFKA_BROKER_ID=1 -e KAFKA_HEAP_OPTS="-Xmx6G -Xms6G" wurstmeister/kafka
# docker run -d --name kafka01 --restart=always --network net-kafka --ip 172.18.0.21 -v D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka01/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka01/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka01/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_HOST_NAME=172.18.0.21 -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.21:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=1 -e KAFKA_LOG_DIRS=/opt/kafka/logs wurstmeister/kafkadocker run -d --name kafka01 -p 9192:9092 --restart=always --network net-kafka --ip 172.18.0.21 -v D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka01/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka01/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka01/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.21:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=1 wurstmeister/kafkadocker run -d --name kafka02 -p 9292:9092 --restart=always --network net-kafka --ip 172.18.0.22 -v D:/mysoft/dev/docker-kafka-cluster/kafka02/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka02/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka02/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka02/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.22:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=2 wurstmeister/kafkadocker run -d --name kafka03 -p 9392:9092 --restart=always --network net-kafka --ip 172.18.0.23 -v D:/mysoft/dev/docker-kafka-cluster/kafka03/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka03/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka03/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka03/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.23:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=3 wurstmeister/kafkakafka容器异常，一直自动反复重启，UP状态维持1分钟左右
[Configuring] 'advertised.listeners' in '/opt/kafka/config/server.properties'
[Configuring] 'port' in '/opt/kafka/config/server.properties'
Excluding KAFKA_HOME from broker config
[Configuring] 'log.dirs' in '/opt/kafka/config/server.properties'
[Configuring] 'listeners' in '/opt/kafka/config/server.properties'
Excluding KAFKA_VERSION from broker config
[Configuring] 'zookeeper.connect' in '/opt/kafka/config/server.properties'
[Configuring] 'broker.id' in '/opt/kafka/config/server.properties'
log4j:ERROR Could not read configuration file from URL [file:/opt/kafka/bin/../config/log4j.properties].
java.io.FileNotFoundException: /opt/kafka/bin/../config/log4j.properties (No such file or directory)at java.base/java.io.FileInputStream.open0(Native Method)at java.base/java.io.FileInputStream.open(Unknown Source)at java.base/java.io.FileInputStream.<init>(Unknown Source)at java.base/java.io.FileInputStream.<init>(Unknown Source)at java.base/sun.net.www.protocol.file.FileURLConnection.connect(Unknown Source)at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(Unknown Source)at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:557)at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)at com.typesafe.scalalogging.Logger$.apply(Logger.scala:48)at kafka.utils.Log4jControllerRegistration$.<clinit>(Logging.scala:25)at kafka.Kafka$.<clinit>(Kafka.scala:30)at kafka.Kafka.main(Kafka.scala)
log4j:ERROR Ignoring configuration file [file:/opt/kafka/bin/../config/log4j.properties].
log4j:WARN No appenders could be found for logger (kafka.utils.Log4jControllerRegistration$).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.指定-e KAFKA_LOG_DIRS也依然不可以，手动拷贝一份log4j.properties到映射的目录D:/mysoft/dev/docker-kafka-cluster/kafka01/logs下问题就解决了。
但是出现另外一个异常：
[2023-12-23 15:20:50,136] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,204] INFO Socket connection established, initiating session, client: /172.18.0.21:39814, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,273] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,749] INFO Opening socket connection to server zk03.net-kafka/172.18.0.13:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,811] INFO Socket connection established, initiating session, client: /172.18.0.21:49980, server: zk03.net-kafka/172.18.0.13:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,914] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,268] INFO Opening socket connection to server zk02.net-kafka/172.18.0.12:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,356] INFO Socket connection established, initiating session, client: /172.18.0.21:40344, server: zk02.net-kafka/172.18.0.12:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,446] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,617] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2023-12-23 15:20:52,817] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,855] INFO Socket connection established, initiating session, client: /172.18.0.21:39828, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:53,036] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2023-12-23 15:20:53,037] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:53,076] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2023-12-23 15:20:53,222] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTINGat kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:271)at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:125)at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1948)at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:431)at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:456)at kafka.server.KafkaServer.startup(KafkaServer.scala:191)at kafka.Kafka$.main(Kafka.scala:109)at kafka.Kafka.main(Kafka.scala)
[2023-12-23 15:20:53,290] INFO shutting down (kafka.server.KafkaServer)
[2023-12-23 15:20:53,375] INFO App info kafka.server for 1 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2023-12-23 15:20:53,440] INFO shut down completed (kafka.server.KafkaServer)
[2023-12-23 15:20:53,515] ERROR Exiting Kafka. (kafka.Kafka$)
[2023-12-23 15:20:53,590] INFO shutting down (kafka.server.KafkaServer)回头看zk的控制台，发现报错java.io.IOException: ZooKeeperServer not running，跟用zkui客户端连zk集群一样的错，通过修改zk的data/myid配置，之后，3个kafka服务都能正常启动了，而且也不会反复自动重启了。
# kafka01
[2023-12-25 02:08:01,176] INFO [KafkaServer id=1] started (kafka.server.KafkaServer)# kafka02
[2023-12-25 02:09:56,391] INFO [KafkaServer id=2] started (kafka.server.KafkaServer)# kafka03
[2023-12-25 02:10:14,479] INFO [KafkaServer id=3] started (kafka.server.KafkaServer)虽然kafka服务在启动的初始阶段，仍然有一下异常，但是目前没有发现有什么影响：
[2023-12-25 00:52:15,882] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:15,932] INFO Socket connection established, initiating session, client: /172.18.0.21:49568, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:15,984] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,040] INFO Opening socket connection to server zk03.net-kafka/172.18.0.13:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,094] INFO Socket connection established, initiating session, client: /172.18.0.21:59734, server: zk03.net-kafka/172.18.0.13:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,148] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,979] INFO Opening socket connection to server zk02.net-kafka/172.18.0.12:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:18,033] INFO Socket connection established, initiating session, client: /172.18.0.21:50098, server: zk02.net-kafka/172.18.0.12:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:18,088] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,308] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,357] INFO Socket connection established, initiating session, client: /172.18.0.21:49580, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,403] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,974] INFO Opening socket connection to server zk03.net-kafka/172.18.0.13:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,027] INFO Socket connection established, initiating session, client: /172.18.0.21:59746, server: zk03.net-kafka/172.18.0.13:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,075] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,549] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient) [2023-12-25 00:52:20,812] INFO Opening socket connection to server zk02.net-kafka/172.18.0.12:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,860] INFO Socket connection established, initiating session, client: /172.18.0.21:50110, server: zk02.net-kafka/172.18.0.12:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:21,016] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper) [2023-12-25 00:52:21,017] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:21,062] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient) [2023-12-25 00:52:21,164] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTINGat kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:271)at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:125)at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1948)at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:431)at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:456)at kafka.server.KafkaServer.startup(KafkaServer.scala:191)at kafka.Kafka$.main(Kafka.scala:109)at kafka.Kafka.main(Kafka.scala) [2023-12-25 00:52:21,213] INFO shutting down (kafka.server.KafkaServer) [2023-12-25 00:52:21,274] INFO App info kafka.server for 1 unregistered (org.apache.kafka.common.utils.AppInfoParser) [2023-12-25 00:52:21,319] INFO shut down completed (kafka.server.KafkaServer) [2023-12-25 00:52:21,371] ERROR Exiting Kafka. (kafka.Kafka$) [2023-12-25 00:52:21,425] INFO shutting down (kafka.server.KafkaServer)
启动sheepkiller/kafka-manager
# docker run -d --name kafka-manager -p 9000:9000 --network net-kafka --ip 172.18.0.31 -e ZK_HOSTS=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 sheepkiller/kafka-managerdocker run -d --name kafka-manager -p 9000:9000 --network net-kafka -e ZK_HOSTS=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 sheepkiller/kafka-manager
或者使用provectuslabs/kafka-ui:latest
# docker run -d --name kafka-ui -p 8080:8080 --network net-kafka --ip 172.18.0.32 -e DYNAMIC_CONFIG_ENABLED=true provectuslabs/kafka-uidocker run -d --name kafka-ui -p 8080:8080 --network net-kafka -e DYNAMIC_CONFIG_ENABLED=true provectuslabs/kafka-ui
3、完整配置
3.1 zk完整配置
zk01/conf/zoo.cfg
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181
4lw.commands.whitelist=*zk01/data/myid
1zk02/conf/zoo.cfg
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181
4lw.commands.whitelist=*zk02/data/myid
2zk03/conf/zoo.cfg
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181
4lw.commands.whitelist=*zk03/data/myid
33.2 kafka完整配置
kafka01/config/server.properties
broker.id=1
advertised.listeners=PLAINTEXT://172.18.0.21:9092
port=9092
log.dirs=/kafka/kafka-logs-7725cecb8a4f
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181kafka02/config/server.properties
broker.id=2
advertised.listeners=PLAINTEXT://172.18.0.22:9092
port=9092
log.dirs=/kafka/kafka-logs-2b3d2a64f445
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181kafka03/config/server.properties
broker.id=3
advertised.listeners=PLAINTEXT://172.18.0.23:9092
port=9092
log.dirs=/kafka/kafka-logs-d9101a867f99
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:21814、问题
SpringBoot配置：
spring:kafka:bootstrap-servers: localhost:9192,localhost:9292,localhost:9392报错，与单机版错误一致
2023-12-26 15:27:29.296 INFO 11904 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka version: 2.8.1
2023-12-26 15:27:29.298 INFO 11904 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka commitId: 839b886f9b732b15
2023-12-26 15:27:29.298 INFO 11904 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka startTimeMs: 1703575649293
2023-12-26 15:27:29.302 INFO 11904 --- [ main] o.a.k.clients.consumer.KafkaConsumer : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Subscribed to topic(s): test-topic032023-12-26 15:27:30.191 INFO 11904 --- [ntainer#0-0-C-1] org.apache.kafka.clients.Metadata : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Cluster ID: o6TS0_kER-KgmyB7NWjjEw
2023-12-26 15:27:30.194 INFO 11904 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Discovered group coordinator 172.18.0.23:9092 (id: 2147483644 rack: null)
2023-12-26 15:27:30.200 INFO 11904 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] (Re-)joining group
2023-12-26 15:27:38.804 INFO 11904 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Group coordinator 172.18.0.23:9092 (id: 2147483644 rack: null) is unavailable or invalid due to cause: null.isDisconnected: true. Rediscovery will be attempted.相关文章：
docker部署kafka zookeeper模式集群
单机模式链接：https://blog.csdn.net/wsdhla/article/details/133032238 kraft集群模式链接：部署Kafka_kafka 部署-CSDN博客 zookeeper选举机制举例： 目前有5台服务器，每台服务器均没有数据，它们的编号分别是1,2,3,4,5…...
Apache Flink连载（二十一）：Flink On Yarn运行原理-Yarn Application模式
🏡 个人主页：IT贫道_大数据OLAP体系技术栈,Apache Doris,Clickhouse 技术-CSDN博客 🚩 私聊博主：加入大数据技术讨论群聊，获取更多大数据资料。 🔔 博主个人B栈地址：豹哥教你大数据的个人空间-豹哥教你大数据个人主页-哔哩哔哩视频 目录 1. 任务提交命令...
《深入理解C++11：C++11新特性解析与应用》笔记五
第五章 提高类型安全 5.1 强类型枚举 5.1.1 枚举：分门别类与数值的名字 具名枚举类型一般声明类似：enum Gender { Male, Female }。 匿名枚举类型可以使用三种方式实现： 第一种方式时宏，比如 #define Male 0 #define Femal…...
Y9000P + ubuntu22.04 配置Anaconda+pycharm +pytorch
Anaconda3 的安装及使用方法安装 Anaconda3 Anaconda3 是 Anaconda 的具体版本 Anaconda3 中的 Python 解释器默认使用的是 Python3.x 版本，而不是 Python2.x 版本 Python2.x 版本中，字符串是以 ASCII 编码处理的，而在 Python3.x 版本中&am…...
使用Ubuntu编译FFmpeg生成Android动态库/静态库
环境 我这里使用windows里的wsl2的ubuntu，使用物理机或者vmware，vbox之类的安装的ubuntu理论上也可以. gcc编译使用的环境如下: Ndk使用17 FFmpeg使用4.0.2. clang编译使用的环境如下: Ndk使用21.4 FFmpeg使用6.1 FFmpeg下载地址:https://ffmpeg.org/…...
【AIGC-图片生成视频系列-2】八仙过海，各显神通：AI生成视频相关汇总剖析
最近「图片生成视频系列」层出不穷，我拜读并结合实践（对，就是手撕代码，有开源就撕），并对以下几篇文章的相似点以及关键点稍微做个总结： 一. 生成视频中图像的一致性 在图像生成视频的这个过程…...
SpringBoot集成RabbitMq消息队列【附源码】
1. 项目背景 要啥项目背景，就是干！！！ SpringBoot版本：2.7.12 2. Rabbit MQ安装 这里讲解使用docker安装RabbitMQ，如果在windows下面安装RabbitMQ，参考下文 【笑小枫的按步照搬系列】Window…...
MySQL数据库的安装与环境配置
下载 下载MySQL8 安装 解压 配置MySQL环境变量 系统环境变量path D:\ProgramFiles\mysql-8.0.20-winx64\bin 1.点击属性 2.点击高级系统设置 3.点击环境变量 4.在系统变量中找到path 注意这里不是用户变量 5.新建后输入解压的地址 MySQL初始化和启动 以管理员身份运行cmd…...
【广州华锐互动】VR科技科普展厅平台：快速、便捷地创建出属于自己的虚拟展馆
随着科技的不断进步，虚拟现实(VR)技术已经在许多领域取得了显著的成果。尤其是在展馆设计领域，VR科技科普展厅平台已经实现了许多令人瞩目的新突破。 VR科技科普展厅平台是广州华锐互动专门为企业和机构提供虚拟展馆设计和制作的在线平台。通过这个平台&…...
XML Extension Supplement
LEGAL ISSUES, COMPANY POLICIES AND STANDARDS Web Services A Web service is a software system designed to support interoperable machine-to-machine interaction over a network. URI和URL URI，全称是统一资源标识符（Uniform Resource Ident…...
手拉手Springboot获取yml配置文件信息
环境介绍 技术栈 springboot3 软件 版本 mysql 8 IDEA IntelliJ IDEA 2022.2.1 JDK 17 Spring Boot 3.1.7 配置文件说明：启动配置文件优先级：properties高于yml 配置文件application.yml yml是 JSON 的超集，简洁而强大…...
行人重识别(ReID)基础知识入门
这里写目录标题 1、ReID技术概述1.1 基本原理1.2 实现流程1.3 重识别存在的技术挑战 2、训练数据格式介绍 1、ReID技术概述 1.1 基本原理 ReID，全称Re-identification，目的是利用各种智能算法在图像数据库中找到与要搜索的目标相似的对象。ReID是图像检…...
【音视频 ffmpeg 学习】 跑示例程序 持续更新中
环境准备 在上一篇文章 把mux.c 拷贝到main.c 中 使用 attribute(unused) 消除警告 __attribute__(unused)/** Copyright (c) 2003 Fabrice Bellard** Permission is hereby granted, free of charge, to any person obtaining a copy* of this software and associated docu…...
前端axios与python库requests的区别
当涉及到发送HTTP请求时，Axios和Python中的requests库都是常用的工具。下面是它们的详细说明： Axios： Axios是一个基于Promise的HTTP客户端，主要用于浏览器和Node.js环境中发送HTTP请求。以下是Axios的一些特点和用法࿱…...
达梦数据库文档
1：达梦数据库(DM8)简介 达梦数据库管理系统是武汉达梦公司推出的具有完全自主知识产权的高性能数据库管理系统，简称DM。达梦数据库管理系统目前最新的版本是8.0版本，简称DM8。 DM8是达梦公司在总结DM系列产品研发与应用经验的基础上…...
CorelDRAW2024新功能有哪些？CorelDRAW2024最新版本更新怎么样？
CorelDRAW2024新功能有哪些？CorelDRAW2024最新版本更新怎么样？让我们带您详细了解！ CorelDRAW Graphics Suite 是矢量制图行业的标杆软件，2024年全新版本为您带来多项新功能和优化改进。本次更新强调易用性，包括更强大…...
基于Mapify的在线艺术地图设计
地图是传递空间信息的有效载体，更加美观、生动的地图产品也是我们追求目标。 那么，我们如何才能制出如下图所示这样一幅艺术性较高的地图呢？今天我们来一探究竟吧！ 按照惯例，现将网址给出： https://www.m…...
mxxWechatBot微信机器人V2版本文档说明
大家伙，我是雄雄，欢迎关注微信公众号：雄雄的小课堂。 先看这里 一、前言二、mxxWechatBot流程图三、怎么使用？ 一、前言 经过不断地探索与研究，mxxWechatBot正式上线，届时全面开放使用。 mxxWechatBot&am…...
红队打靶练习:MISDIRECTION: 1
信息收集 1、arp ┌──(root㉿ru)-[~/kali] └─# arp-scan -l Interface: eth0, type: EN10MB, MAC: 00:0c:29:69:c7:bf, IPv4: 192.168.12.128 Starting arp-scan 1.10.0 with 256 hosts (https://github.com/royhills/arp-scan) 192.168.12.1 00:50:56:c0:00:08 …...
Jmeter吞吐量控制器总结
吞吐量控制器(Throughput Controller) 场景： 在同一个线程组里, 有10个并发, 7个做A业务, 3个做B业务,要模拟这种场景,可以通过吞吐量模拟器来实现。 添加吞吐量控制器 用法1: Percent Executions 在一个线程组内分别建立两个吞吐量控制器, 分别放业务A和业务B …...
智慧医疗能源事业线深度画像分析（上）
引言 医疗行业作为现代社会的关键基础设施，其能源消耗与环境影响正日益受到关注。随着全球"双碳"目标的推进和可持续发展理念的深入，智慧医疗能源事业线应运而生，致力于通过创新技术与管理方案，重构医疗领域的能源使用模式。这一事业线融合了能源管理、可持续发…...
React Native 导航系统实战（React Navigation）
导航系统实战（React Navigation） React Navigation 是 React Native 应用中最常用的导航库之一，它提供了多种导航模式，如堆栈导航（Stack Navigator）、标签导航（Tab Navigator）和抽屉…...
三维GIS开发cesium智慧地铁教程（5）Cesium相机控制
一、环境搭建 <script src"../cesium1.99/Build/Cesium/Cesium.js"></script> <link rel"stylesheet" href"../cesium1.99/Build/Cesium/Widgets/widgets.css"> 关键配置点： 路径验证：确保相对路径.…...
2021-03-15 iview一些问题
1.iview 在使用tree组件时，发现没有set类的方法，只有get，那么要改变tree值，只能遍历treeData，递归修改treeData的checked，发现无法更改，原因在于check模式下，子元素的勾选状态跟父节…...
全面解析各类VPN技术：GRE、IPsec、L2TP、SSL与MPLS VPN对比
目录 引言 VPN技术概述 GRE VPN 3.1 GRE封装结构 3.2 GRE的应用场景 GRE over IPsec 4.1 GRE over IPsec封装结构 4.2 为什么使用GRE over IPsec？ IPsec VPN 5.1 IPsec传输模式（Transport Mode） 5.2 IPsec隧道模式（Tunne…...
【Oracle】分区表
个人主页：Guiat 归属专栏：Oracle 文章目录 1. 分区表基础概述1.1 分区表的概念与优势1.2 分区类型概览1.3 分区表的工作原理 2. 范围分区 (RANGE Partitioning)2.1 基础范围分区2.1.1 按日期范围分区2.1.2 按数值范围分区 2.2 间隔分区 (INTERVAL Partit…...
USB Over IP专用硬件的5个特点
USB over IP技术通过将USB协议数据封装在标准TCP/IP网络数据包中，从根本上改变了USB连接。这允许客户端通过局域网或广域网远程访问和控制物理连接到服务器的USB设备（如专用硬件设备），从而消除了直接物理连接的需要。USB over IP的…...
使用Matplotlib创建炫酷的3D散点图：数据可视化的新维度
文章目录 基础实现代码代码解析进阶技巧1. 自定义点的大小和颜色2. 添加图例和样式美化3. 真实数据应用示例实用技巧与注意事项完整示例（带样式）应用场景在数据科学和可视化领域，三维图形能为我们提供更丰富的数据洞察。本文将手把手教你如何使用Python的Matplotlib库创建引…...
Python基于历史模拟方法实现投资组合风险管理的VaR与ES模型项目实战
说明：这是一个机器学习实战项目（附带数据代码文档），如需数据代码文档可以直接到文章最后关注获取。 1.项目背景 在金融市场日益复杂和波动加剧的背景下，风险管理成为金融机构和个人投资者关注的核心议题之一。VaR&…...
人机融合智能 | “人智交互”跨学科新领域
本文系统地提出基于“以人为中心AI(HCAI)”理念的人-人工智能交互(人智交互)这一跨学科新领域及框架,定义人智交互领域的理念、基本理论和关键问题、方法、开发流程和参与团队等,阐述提出人智交互新领域的意义。然后,提出人智交互研究的三种新范式取向以及它们的意义。最后,总结…...