docker部署kafka zookeeper模式集群
单机模式链接：https://blog.csdn.net/wsdhla/article/details/133032238
kraft集群模式链接：部署Kafka_kafka 部署-CSDN博客
zookeeper选举机制举例：
目前有5台服务器，每台服务器均没有数据，它们的编号分别是1,2,3,4,5,按编号依次启动，它们的选择举过程如下：
- 服务器1启动，给自己投票，然后发投票信息，由于其它机器还没有启动所以它收不到反馈信息，服务器1的状态一直属于Looking(选举状态)。
- 服务器2启动，给自己投票，同时与之前启动的服务器1交换结果，由于服务器2的编号大所以服务器2胜出，但此时投票数没有大于半数，所以两个服务器的状态依然是LOOKING。
- 服务器3启动，给自己投票，同时与之前启动的服务器1,2交换信息，由于服务器3的编号最大所以服务器3胜出，此时投票数正好大于半数，所以服务器3成为领导者，服务器1,2成为小弟。
- 服务器4启动，给自己投票，同时与之前启动的服务器1,2,3交换信息，尽管服务器4的编号大，但之前服务器3已经胜出，所以服务器4只能成为小弟。
- 服务器5启动，后面的逻辑同服务器4成为小弟。
Docker 自定义静态网络
Docker默认提供了3种网络模式，生成容器时不指定网络模式下默认使用bridge桥接模式。每次容器启动时ip会发生变更，无法达到我们集群的效果。
创建自定义网络模式，指定网段。
docker network create --subnet=172.18.0.0/16 net-kafka若报错：Error response from daemon: Pool overlaps with other one on this address space
说明指定的网络地址段已存在
1、Docker部署zookeeper集群
zk使用官方镜像：
1.1、主机创建目录
[zk01]
D:/mysoft/dev/docker-zk-cluster/zk01/conf
D:/mysoft/dev/docker-zk-cluster/zk01/data
D:/mysoft/dev/docker-zk-cluster/zk01/datalog[zk02]
D:/mysoft/dev/docker-zk-cluster/zk02/conf
D:/mysoft/dev/docker-zk-cluster/zk02/data
D:/mysoft/dev/docker-zk-cluster/zk02/datalog[zk03]
D:/mysoft/dev/docker-zk-cluster/zk03/conf
D:/mysoft/dev/docker-zk-cluster/zk03/data
D:/mysoft/dev/docker-zk-cluster/zk03/datalog1.2、修改配置文件
分别拷贝到zk01、zk02、zk03
[zoo.cfg]dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;21811.3、指定ip创建启动容器
docker run -d --name zk01 -p 2181:2181 --network net-kafka --ip 172.18.0.11 -v D:/mysoft/dev/docker-zk-cluster/zk01/conf:/conf -v D:/mysoft/dev/docker-zk-cluster/zk01/data:/data -v D:/mysoft/dev/docker-zk-cluster/zk01/datalog:/datalog zookeeperdocker run -d --name zk02 -p 2182:2181 --network net-kafka --ip 172.18.0.12 -v D:/mysoft/dev/docker-zk-cluster/zk02/conf:/conf -v D:/mysoft/dev/docker-zk-cluster/zk02/data:/data -v D:/mysoft/dev/docker-zk-cluster/zk02/datalog:/datalog zookeeperdocker run -d --name zk03 -p 2183:2181 --network net-kafka --ip 172.18.0.13 -v D:/mysoft/dev/docker-zk-cluster/zk03/conf:/conf -v D:/mysoft/dev/docker-zk-cluster/zk03/data:/data -v D:/mysoft/dev/docker-zk-cluster/zk03/datalog:/datalog zookeeperzk01可以正常启动，zk02和zk03启动失败
警告：这个正常，是因为节点启动时选举机制的原因
zk02：
WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 2 at election address /172.18.0.12:3888
zk03：
WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
异常：
zk02：
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Exception while listening to address /172.18.0.11:3888
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Leaving listener thread for address 172.18.0.11:3888 after 3 errors. Use zookeeper.electionPortBindRetry property to increase retry count.
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - As I'm leaving the listener thread, I won't be able to participate in leader election any longer: 172.18.0.11:3888
zk03：
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Exception while listening to address /172.18.0.11:3888
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - Leaving listener thread for address 172.18.0.11:3888 after 3 errors. Use zookeeper.electionPortBindRetry property to increase retry count.
ERROR org.apache.zookeeper.server.quorum.QuorumCnxManager - As I'm leaving the listener thread, I won't be able to participate in leader election any longer: 172.18.0.11:3888
原因是没有配置quorumListenOnAllIPs，默认为false，需要设置为true
[zoo.cfg]dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181quorumListenOnAllIPs：
当设置为true时，ZooKeeper服务器将在所有可用IP地址上侦听来自其对等方的连接，而不仅是在配置文件的服务器列表中配置的地址。它会影响处理ZAB协议和快速领导者选举协议的连接。默认值为false。
有一些WARN:
### 1、只启动zk01：
# zk01：
01:21:17.746 [QuorumConnectionThread-[myid=1]-3] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 2 at election address /172.18.0.12:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)01:21:20.904 [QuorumConnectionThread-[myid=1]-1] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)### 2、 只启动zk01、zk02：
# zk01：
01:21:46.504 [QuorumConnectionThread-[myid=1]-3] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)# zk02：
01:21:43.350 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.01:21:48.824 [QuorumConnectionThread-[myid=1]-2] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - Cannot open channel to 3 at election address /172.18.0.13:3888
java.net.NoRouteToHostException: No route to host (Host unreachable)### 2、 启动zk01、zk02、zk03：
# zk01正常：
01:31:34.579 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:31:34.579 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:31:34.580 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 2 already, it is in the lastCommittedView
01:31:34.584 [QuorumConnectionThread-[myid=1]-21] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 2
01:31:34.584 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 3 already, it is in the lastCommittedView
01:31:34.585 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] INFO org.apache.zookeeper.server.quorum.FastLeaderElection - Notification time out: 60000 ms
01:31:34.585 [QuorumConnectionThread-[myid=1]-22] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 3
01:31:34.585 [QuorumConnectionThread-[myid=1]-21] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 2 using election address: /172.18.0.12:3888
01:31:34.585 [QuorumConnectionThread-[myid=1]-21] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:2)
01:31:34.586 [QuorumConnectionThread-[myid=1]-21] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2)
01:31:34.585 [QuorumConnectionThread-[myid=1]-22] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 3 using election address: /172.18.0.13:3888
01:31:34.586 [QuorumConnectionThread-[myid=1]-22] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:3)
01:31:34.586 [QuorumConnectionThread-[myid=1]-22] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:3)
01:31:52.173 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - node0 scavenging sessions
01:31:52.174 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - org.eclipse.jetty.server.session.SessionHandler873610597==dftMaxIdleSec=-1 scavenging sessions
01:31:52.174 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - org.eclipse.jetty.server.session.SessionHandler873610597==dftMaxIdleSec=-1 scavenging session ids []
01:31:52.174 [Session-HouseKeeper-383bfa16-1] DEBUG org.eclipse.jetty.server.session - org.eclipse.jetty.server.session.DefaultSessionCache@57d7f8ca[evict=-1,removeUnloadable=false,saveOnCreate=false,saveOnInactiveEvict=false] checking expiration on []# zk02：
01:35:15.331 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:35:15.332 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:35:15.333 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 2 already, it is in the lastCommittedView
01:35:15.334 [QuorumConnectionThread-[myid=1]-27] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 2
01:35:15.334 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 3 already, it is in the lastCommittedView
01:35:15.335 [QuorumConnectionThread-[myid=1]-27] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 2 using election address: /172.18.0.12:3888
01:35:15.335 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] INFO org.apache.zookeeper.server.quorum.FastLeaderElection - Notification time out: 60000 ms
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.12:50246
01:35:15.335 [QuorumConnectionThread-[myid=1]-27] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:2)
01:35:15.335 [QuorumConnectionThread-[myid=1]-28] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 3
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.12:50246
01:35:15.335 [QuorumConnectionThread-[myid=1]-27] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2)
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:35:15.335 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:35:15.335 [QuorumConnectionThread-[myid=1]-28] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 3 using election address: /172.18.0.13:3888
01:35:15.335 [QuorumConnectionThread-[myid=1]-28] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:3)
01:35:15.336 [QuorumConnectionThread-[myid=1]-28] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:3)
01:35:34.600 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.11:49750
01:35:34.601 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.11:49750
01:35:34.601 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:35:34.601 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:35:59.199 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.13:56956
01:35:59.200 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.13:56956
01:35:59.200 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:35:59.200 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.# zk03：
01:36:59.201 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:36:59.201 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Queue size: 1
01:36:59.202 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 2 already, it is in the lastCommittedView
01:36:59.202 [QuorumConnectionThread-[myid=1]-15] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 2
01:36:59.203 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Server 1 knows 3 already, it is in the lastCommittedView
01:36:59.203 [QuorumConnectionThread-[myid=1]-15] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 2 using election address: /172.18.0.12:3888
01:36:59.203 [QuorumConnectionThread-[myid=1]-15] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:2)
01:36:59.203 [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled)] INFO org.apache.zookeeper.server.quorum.FastLeaderElection - Notification time out: 60000 ms
01:36:59.203 [QuorumConnectionThread-[myid=1]-16] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Opening channel to server 3
01:36:59.203 [QuorumConnectionThread-[myid=1]-15] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2)
01:36:59.204 [QuorumConnectionThread-[myid=1]-16] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Connected to server 3 using election address: /172.18.0.13:3888
01:36:59.204 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.13:35192
01:36:59.204 [QuorumConnectionThread-[myid=1]-16] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - startConnection (myId:1 --> sid:3)
01:36:59.204 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.13:35192
01:36:59.204 [QuorumConnectionThread-[myid=1]-16] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Have smaller server identifier, so dropping the connection: (myId:1 --> sid:3)
01:36:59.204 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:36:59.205 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.12:35956
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.12:35956
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:37:15.342 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] INFO org.apache.zookeeper.server.quorum.QuorumCnxManager - Received connection request from /172.18.0.11:46420
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Sync handling of connection request received from: /172.18.0.11:46420
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] DEBUG org.apache.zookeeper.server.quorum.QuorumCnxManager - Initial message parsed by 1: InitialMessage{sid=1, electionAddr=[/172.18.0.11:3888]}
01:37:34.609 [ListenerHandler-0.0.0.0/0.0.0.0:3888] WARN org.apache.zookeeper.server.quorum.QuorumCnxManager - We got a connection request from a server with our own ID. This should be either a configuration error, or a bug.
找到原因了，因为data/myid没有修改，目前3个节点的配置全部为1，分别改为1、2、3就正常了。
启动zkui
docker run -d --name zkui -p 9090:9090 --network net-kafka -e ZKLIST=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 maauso/zkui访问localhost:9090，登录之后报错：KeeperErrorCode = ConnectionLoss for /
删除data\version-2\snapshot.0依然不行
stat is not executed because it is not in the whitelist.
envi is not executed because it is not in the whitelist.
4lw.commands.whitelist=*
16:49:58.466 [NIOWorkerThread-10] DEBUG org.apache.zookeeper.server.NIOServerCnxn - Closed socket connection for client /172.18.0.2:42978 (no session established for client)
16:49:58.499 [NIOServerCxnFactory.AcceptThread:/0.0.0.0:2181] DEBUG org.apache.zookeeper.server.NIOServerCnxnFactory - Accepted socket connection from /172.18.0.2:42980
16:49:58.500 [NIOWorkerThread-11] WARN org.apache.zookeeper.server.NIOServerCnxn - Close of session 0x0
java.io.IOException: ZooKeeperServer not runningat org.apache.zookeeper.server.NIOServerCnxn.readLength(NIOServerCnxn.java:569)at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:349)at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:508)at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:153)at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)at java.base/java.lang.Thread.run(Unknown Source)好像改为zk的data/myid配置，上面的问题就没有了。
2、Docker部署kafka集群
kafka没有官方镜像，使用wurstmeister/kafka镜像
2.1、主机创建目录
[kafka01]
D:/mysoft/dev/docker-kafka-cluster/kafka01/config
D:/mysoft/dev/docker-kafka-cluster/kafka01/data
D:/mysoft/dev/docker-kafka-cluster/kafka01/logs
D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts[kafka02]
D:/mysoft/dev/docker-kafka-cluster/kafka02/config
D:/mysoft/dev/docker-kafka-cluster/kafka02/data
D:/mysoft/dev/docker-kafka-cluster/kafka02/logs
D:/mysoft/dev/docker-kafka-cluster/kafka02/hosts[kafka03]
D:/mysoft/dev/docker-kafka-cluster/kafka03/config
D:/mysoft/dev/docker-kafka-cluster/kafka03/data
D:/mysoft/dev/docker-kafka-cluster/kafka03/logs
D:/mysoft/dev/docker-kafka-cluster/kafka03/hosts2.2、修改配置文件
分别拷贝到kafka01、kafka02、kafka03
[server.properties]broker.id=1
advertised.listeners=PLAINTEXT://172.18.0.21:9092
# port=9092
# log.dirs=/kafka/kafka-logs-1ddafafe54d0
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:21812.3、指定ip创建启动容器
# docker run -d --name kafka01 --restart=always --net=host -v D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts:/etc/hosts -v D:/mysoft/dev/docker-kafka-cluster/kafka01/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka01/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka01/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_HOST_NAME=192.168.180.42 -e JMX_PORT=9999 -e HOST_IP=127.0.0.1 -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_ZOOKEEPER_CONNECT=127.0.0.1:2181 -e KAFKA_BROKER_ID=1 -e KAFKA_HEAP_OPTS="-Xmx6G -Xms6G" wurstmeister/kafka
# docker run -d --name kafka01 --restart=always --network net-kafka --ip 172.18.0.21 -v D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka01/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka01/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka01/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_HOST_NAME=172.18.0.21 -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.21:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=1 -e KAFKA_LOG_DIRS=/opt/kafka/logs wurstmeister/kafkadocker run -d --name kafka01 -p 9192:9092 --restart=always --network net-kafka --ip 172.18.0.21 -v D:/mysoft/dev/docker-kafka-cluster/kafka01/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka01/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka01/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka01/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.21:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=1 wurstmeister/kafkadocker run -d --name kafka02 -p 9292:9092 --restart=always --network net-kafka --ip 172.18.0.22 -v D:/mysoft/dev/docker-kafka-cluster/kafka02/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka02/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka02/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka02/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.22:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=2 wurstmeister/kafkadocker run -d --name kafka03 -p 9392:9092 --restart=always --network net-kafka --ip 172.18.0.23 -v D:/mysoft/dev/docker-kafka-cluster/kafka03/hosts:/etc -v D:/mysoft/dev/docker-kafka-cluster/kafka03/data:/opt/kafka/data -v D:/mysoft/dev/docker-kafka-cluster/kafka03/config:/opt/kafka/config -v D:/mysoft/dev/docker-kafka-cluster/kafka03/logs:/opt/kafka/logs -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.18.0.23:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 -e KAFKA_BROKER_ID=3 wurstmeister/kafkakafka容器异常，一直自动反复重启，UP状态维持1分钟左右
[Configuring] 'advertised.listeners' in '/opt/kafka/config/server.properties'
[Configuring] 'port' in '/opt/kafka/config/server.properties'
Excluding KAFKA_HOME from broker config
[Configuring] 'log.dirs' in '/opt/kafka/config/server.properties'
[Configuring] 'listeners' in '/opt/kafka/config/server.properties'
Excluding KAFKA_VERSION from broker config
[Configuring] 'zookeeper.connect' in '/opt/kafka/config/server.properties'
[Configuring] 'broker.id' in '/opt/kafka/config/server.properties'
log4j:ERROR Could not read configuration file from URL [file:/opt/kafka/bin/../config/log4j.properties].
java.io.FileNotFoundException: /opt/kafka/bin/../config/log4j.properties (No such file or directory)at java.base/java.io.FileInputStream.open0(Native Method)at java.base/java.io.FileInputStream.open(Unknown Source)at java.base/java.io.FileInputStream.<init>(Unknown Source)at java.base/java.io.FileInputStream.<init>(Unknown Source)at java.base/sun.net.www.protocol.file.FileURLConnection.connect(Unknown Source)at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(Unknown Source)at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:557)at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)at com.typesafe.scalalogging.Logger$.apply(Logger.scala:48)at kafka.utils.Log4jControllerRegistration$.<clinit>(Logging.scala:25)at kafka.Kafka$.<clinit>(Kafka.scala:30)at kafka.Kafka.main(Kafka.scala)
log4j:ERROR Ignoring configuration file [file:/opt/kafka/bin/../config/log4j.properties].
log4j:WARN No appenders could be found for logger (kafka.utils.Log4jControllerRegistration$).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.指定-e KAFKA_LOG_DIRS也依然不可以，手动拷贝一份log4j.properties到映射的目录D:/mysoft/dev/docker-kafka-cluster/kafka01/logs下问题就解决了。
但是出现另外一个异常：
[2023-12-23 15:20:50,136] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,204] INFO Socket connection established, initiating session, client: /172.18.0.21:39814, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,273] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,749] INFO Opening socket connection to server zk03.net-kafka/172.18.0.13:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,811] INFO Socket connection established, initiating session, client: /172.18.0.21:49980, server: zk03.net-kafka/172.18.0.13:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:50,914] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,268] INFO Opening socket connection to server zk02.net-kafka/172.18.0.12:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,356] INFO Socket connection established, initiating session, client: /172.18.0.21:40344, server: zk02.net-kafka/172.18.0.12:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,446] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,617] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2023-12-23 15:20:52,817] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:52,855] INFO Socket connection established, initiating session, client: /172.18.0.21:39828, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:53,036] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2023-12-23 15:20:53,037] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2023-12-23 15:20:53,076] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2023-12-23 15:20:53,222] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTINGat kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:271)at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:125)at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1948)at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:431)at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:456)at kafka.server.KafkaServer.startup(KafkaServer.scala:191)at kafka.Kafka$.main(Kafka.scala:109)at kafka.Kafka.main(Kafka.scala)
[2023-12-23 15:20:53,290] INFO shutting down (kafka.server.KafkaServer)
[2023-12-23 15:20:53,375] INFO App info kafka.server for 1 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2023-12-23 15:20:53,440] INFO shut down completed (kafka.server.KafkaServer)
[2023-12-23 15:20:53,515] ERROR Exiting Kafka. (kafka.Kafka$)
[2023-12-23 15:20:53,590] INFO shutting down (kafka.server.KafkaServer)回头看zk的控制台，发现报错java.io.IOException: ZooKeeperServer not running，跟用zkui客户端连zk集群一样的错，通过修改zk的data/myid配置，之后，3个kafka服务都能正常启动了，而且也不会反复自动重启了。
# kafka01
[2023-12-25 02:08:01,176] INFO [KafkaServer id=1] started (kafka.server.KafkaServer)# kafka02
[2023-12-25 02:09:56,391] INFO [KafkaServer id=2] started (kafka.server.KafkaServer)# kafka03
[2023-12-25 02:10:14,479] INFO [KafkaServer id=3] started (kafka.server.KafkaServer)虽然kafka服务在启动的初始阶段，仍然有一下异常，但是目前没有发现有什么影响：
[2023-12-25 00:52:15,882] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:15,932] INFO Socket connection established, initiating session, client: /172.18.0.21:49568, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:15,984] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,040] INFO Opening socket connection to server zk03.net-kafka/172.18.0.13:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,094] INFO Socket connection established, initiating session, client: /172.18.0.21:59734, server: zk03.net-kafka/172.18.0.13:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,148] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:17,979] INFO Opening socket connection to server zk02.net-kafka/172.18.0.12:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:18,033] INFO Socket connection established, initiating session, client: /172.18.0.21:50098, server: zk02.net-kafka/172.18.0.12:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:18,088] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,308] INFO Opening socket connection to server zk01.net-kafka/172.18.0.11:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,357] INFO Socket connection established, initiating session, client: /172.18.0.21:49580, server: zk01.net-kafka/172.18.0.11:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,403] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:19,974] INFO Opening socket connection to server zk03.net-kafka/172.18.0.13:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,027] INFO Socket connection established, initiating session, client: /172.18.0.21:59746, server: zk03.net-kafka/172.18.0.13:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,075] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,549] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient) [2023-12-25 00:52:20,812] INFO Opening socket connection to server zk02.net-kafka/172.18.0.12:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:20,860] INFO Socket connection established, initiating session, client: /172.18.0.21:50110, server: zk02.net-kafka/172.18.0.12:2181 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:21,016] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper) [2023-12-25 00:52:21,017] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn) [2023-12-25 00:52:21,062] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient) [2023-12-25 00:52:21,164] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTINGat kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:271)at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:125)at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1948)at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:431)at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:456)at kafka.server.KafkaServer.startup(KafkaServer.scala:191)at kafka.Kafka$.main(Kafka.scala:109)at kafka.Kafka.main(Kafka.scala) [2023-12-25 00:52:21,213] INFO shutting down (kafka.server.KafkaServer) [2023-12-25 00:52:21,274] INFO App info kafka.server for 1 unregistered (org.apache.kafka.common.utils.AppInfoParser) [2023-12-25 00:52:21,319] INFO shut down completed (kafka.server.KafkaServer) [2023-12-25 00:52:21,371] ERROR Exiting Kafka. (kafka.Kafka$) [2023-12-25 00:52:21,425] INFO shutting down (kafka.server.KafkaServer)
启动sheepkiller/kafka-manager
# docker run -d --name kafka-manager -p 9000:9000 --network net-kafka --ip 172.18.0.31 -e ZK_HOSTS=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 sheepkiller/kafka-managerdocker run -d --name kafka-manager -p 9000:9000 --network net-kafka -e ZK_HOSTS=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181 sheepkiller/kafka-manager
或者使用provectuslabs/kafka-ui:latest
# docker run -d --name kafka-ui -p 8080:8080 --network net-kafka --ip 172.18.0.32 -e DYNAMIC_CONFIG_ENABLED=true provectuslabs/kafka-uidocker run -d --name kafka-ui -p 8080:8080 --network net-kafka -e DYNAMIC_CONFIG_ENABLED=true provectuslabs/kafka-ui
3、完整配置
3.1 zk完整配置
zk01/conf/zoo.cfg
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181
4lw.commands.whitelist=*zk01/data/myid
1zk02/conf/zoo.cfg
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181
4lw.commands.whitelist=*zk02/data/myid
2zk03/conf/zoo.cfg
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true
server.1=172.18.0.11:2888:3888;2181
server.2=172.18.0.12:2888:3888;2181
server.3=172.18.0.13:2888:3888;2181
4lw.commands.whitelist=*zk03/data/myid
33.2 kafka完整配置
kafka01/config/server.properties
broker.id=1
advertised.listeners=PLAINTEXT://172.18.0.21:9092
port=9092
log.dirs=/kafka/kafka-logs-7725cecb8a4f
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181kafka02/config/server.properties
broker.id=2
advertised.listeners=PLAINTEXT://172.18.0.22:9092
port=9092
log.dirs=/kafka/kafka-logs-2b3d2a64f445
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:2181kafka03/config/server.properties
broker.id=3
advertised.listeners=PLAINTEXT://172.18.0.23:9092
port=9092
log.dirs=/kafka/kafka-logs-d9101a867f99
listeners=PLAINTEXT://0.0.0.0:9092
zookeeper.connect=172.18.0.11:2181,172.18.0.12:2181,172.18.0.13:21814、问题
SpringBoot配置：
spring:kafka:bootstrap-servers: localhost:9192,localhost:9292,localhost:9392报错，与单机版错误一致
2023-12-26 15:27:29.296 INFO 11904 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka version: 2.8.1
2023-12-26 15:27:29.298 INFO 11904 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka commitId: 839b886f9b732b15
2023-12-26 15:27:29.298 INFO 11904 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka startTimeMs: 1703575649293
2023-12-26 15:27:29.302 INFO 11904 --- [ main] o.a.k.clients.consumer.KafkaConsumer : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Subscribed to topic(s): test-topic032023-12-26 15:27:30.191 INFO 11904 --- [ntainer#0-0-C-1] org.apache.kafka.clients.Metadata : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Cluster ID: o6TS0_kER-KgmyB7NWjjEw
2023-12-26 15:27:30.194 INFO 11904 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Discovered group coordinator 172.18.0.23:9092 (id: 2147483644 rack: null)
2023-12-26 15:27:30.200 INFO 11904 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] (Re-)joining group
2023-12-26 15:27:38.804 INFO 11904 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-test-consumer-group01-1, groupId=test-consumer-group01] Group coordinator 172.18.0.23:9092 (id: 2147483644 rack: null) is unavailable or invalid due to cause: null.isDisconnected: true. Rediscovery will be attempted.相关文章：
docker部署kafka zookeeper模式集群
单机模式链接：https://blog.csdn.net/wsdhla/article/details/133032238 kraft集群模式链接：部署Kafka_kafka 部署-CSDN博客 zookeeper选举机制举例： 目前有5台服务器，每台服务器均没有数据，它们的编号分别是1,2,3,4,5…...
Apache Flink连载（二十一）：Flink On Yarn运行原理-Yarn Application模式
🏡 个人主页：IT贫道_大数据OLAP体系技术栈,Apache Doris,Clickhouse 技术-CSDN博客 🚩 私聊博主：加入大数据技术讨论群聊，获取更多大数据资料。 🔔 博主个人B栈地址：豹哥教你大数据的个人空间-豹哥教你大数据个人主页-哔哩哔哩视频 目录 1. 任务提交命令...
《深入理解C++11：C++11新特性解析与应用》笔记五
第五章 提高类型安全 5.1 强类型枚举 5.1.1 枚举：分门别类与数值的名字 具名枚举类型一般声明类似：enum Gender { Male, Female }。 匿名枚举类型可以使用三种方式实现： 第一种方式时宏，比如 #define Male 0 #define Femal…...
Y9000P + ubuntu22.04 配置Anaconda+pycharm +pytorch
Anaconda3 的安装及使用方法安装 Anaconda3 Anaconda3 是 Anaconda 的具体版本 Anaconda3 中的 Python 解释器默认使用的是 Python3.x 版本，而不是 Python2.x 版本 Python2.x 版本中，字符串是以 ASCII 编码处理的，而在 Python3.x 版本中&am…...
使用Ubuntu编译FFmpeg生成Android动态库/静态库
环境 我这里使用windows里的wsl2的ubuntu，使用物理机或者vmware，vbox之类的安装的ubuntu理论上也可以. gcc编译使用的环境如下: Ndk使用17 FFmpeg使用4.0.2. clang编译使用的环境如下: Ndk使用21.4 FFmpeg使用6.1 FFmpeg下载地址:https://ffmpeg.org/…...
【AIGC-图片生成视频系列-2】八仙过海，各显神通：AI生成视频相关汇总剖析
最近「图片生成视频系列」层出不穷，我拜读并结合实践（对，就是手撕代码，有开源就撕），并对以下几篇文章的相似点以及关键点稍微做个总结： 一. 生成视频中图像的一致性 在图像生成视频的这个过程…...
SpringBoot集成RabbitMq消息队列【附源码】
1. 项目背景 要啥项目背景，就是干！！！ SpringBoot版本：2.7.12 2. Rabbit MQ安装 这里讲解使用docker安装RabbitMQ，如果在windows下面安装RabbitMQ，参考下文 【笑小枫的按步照搬系列】Window…...
MySQL数据库的安装与环境配置
下载 下载MySQL8 安装 解压 配置MySQL环境变量 系统环境变量path D:\ProgramFiles\mysql-8.0.20-winx64\bin 1.点击属性 2.点击高级系统设置 3.点击环境变量 4.在系统变量中找到path 注意这里不是用户变量 5.新建后输入解压的地址 MySQL初始化和启动 以管理员身份运行cmd…...
【广州华锐互动】VR科技科普展厅平台：快速、便捷地创建出属于自己的虚拟展馆
随着科技的不断进步，虚拟现实(VR)技术已经在许多领域取得了显著的成果。尤其是在展馆设计领域，VR科技科普展厅平台已经实现了许多令人瞩目的新突破。 VR科技科普展厅平台是广州华锐互动专门为企业和机构提供虚拟展馆设计和制作的在线平台。通过这个平台&…...
XML Extension Supplement
LEGAL ISSUES, COMPANY POLICIES AND STANDARDS Web Services A Web service is a software system designed to support interoperable machine-to-machine interaction over a network. URI和URL URI，全称是统一资源标识符（Uniform Resource Ident…...
手拉手Springboot获取yml配置文件信息
环境介绍 技术栈 springboot3 软件 版本 mysql 8 IDEA IntelliJ IDEA 2022.2.1 JDK 17 Spring Boot 3.1.7 配置文件说明：启动配置文件优先级：properties高于yml 配置文件application.yml yml是 JSON 的超集，简洁而强大…...
行人重识别(ReID)基础知识入门
这里写目录标题 1、ReID技术概述1.1 基本原理1.2 实现流程1.3 重识别存在的技术挑战 2、训练数据格式介绍 1、ReID技术概述 1.1 基本原理 ReID，全称Re-identification，目的是利用各种智能算法在图像数据库中找到与要搜索的目标相似的对象。ReID是图像检…...
【音视频 ffmpeg 学习】 跑示例程序 持续更新中
环境准备 在上一篇文章 把mux.c 拷贝到main.c 中 使用 attribute(unused) 消除警告 __attribute__(unused)/** Copyright (c) 2003 Fabrice Bellard** Permission is hereby granted, free of charge, to any person obtaining a copy* of this software and associated docu…...
前端axios与python库requests的区别
当涉及到发送HTTP请求时，Axios和Python中的requests库都是常用的工具。下面是它们的详细说明： Axios： Axios是一个基于Promise的HTTP客户端，主要用于浏览器和Node.js环境中发送HTTP请求。以下是Axios的一些特点和用法࿱…...
达梦数据库文档
1：达梦数据库(DM8)简介 达梦数据库管理系统是武汉达梦公司推出的具有完全自主知识产权的高性能数据库管理系统，简称DM。达梦数据库管理系统目前最新的版本是8.0版本，简称DM8。 DM8是达梦公司在总结DM系列产品研发与应用经验的基础上…...
CorelDRAW2024新功能有哪些？CorelDRAW2024最新版本更新怎么样？
CorelDRAW2024新功能有哪些？CorelDRAW2024最新版本更新怎么样？让我们带您详细了解！ CorelDRAW Graphics Suite 是矢量制图行业的标杆软件，2024年全新版本为您带来多项新功能和优化改进。本次更新强调易用性，包括更强大…...
基于Mapify的在线艺术地图设计
地图是传递空间信息的有效载体，更加美观、生动的地图产品也是我们追求目标。 那么，我们如何才能制出如下图所示这样一幅艺术性较高的地图呢？今天我们来一探究竟吧！ 按照惯例，现将网址给出： https://www.m…...
mxxWechatBot微信机器人V2版本文档说明
大家伙，我是雄雄，欢迎关注微信公众号：雄雄的小课堂。 先看这里 一、前言二、mxxWechatBot流程图三、怎么使用？ 一、前言 经过不断地探索与研究，mxxWechatBot正式上线，届时全面开放使用。 mxxWechatBot&am…...
红队打靶练习:MISDIRECTION: 1
信息收集 1、arp ┌──(root㉿ru)-[~/kali] └─# arp-scan -l Interface: eth0, type: EN10MB, MAC: 00:0c:29:69:c7:bf, IPv4: 192.168.12.128 Starting arp-scan 1.10.0 with 256 hosts (https://github.com/royhills/arp-scan) 192.168.12.1 00:50:56:c0:00:08 …...
Jmeter吞吐量控制器总结
吞吐量控制器(Throughput Controller) 场景： 在同一个线程组里, 有10个并发, 7个做A业务, 3个做B业务,要模拟这种场景,可以通过吞吐量模拟器来实现。 添加吞吐量控制器 用法1: Percent Executions 在一个线程组内分别建立两个吞吐量控制器, 分别放业务A和业务B …...
LBE-LEX系列工业语音播放器|预警播报器|喇叭蜂鸣器的上位机配置操作说明
LBE-LEX系列工业语音播放器|预警播报器|喇叭蜂鸣器专为工业环境精心打造，完美适配AGV和无人叉车。同时，集成以太网与语音合成技术，为各类高级系统（如MES、调度系统、库位管理、立库等）提供高效便捷的语音交互体验。 L…...
手游刚开服就被攻击怎么办？如何防御DDoS？
开服初期是手游最脆弱的阶段，极易成为DDoS攻击的目标。一旦遭遇攻击，可能导致服务器瘫痪、玩家流失，甚至造成巨大经济损失。本文为开发者提供一套简洁有效的应急与防御方案，帮助快速应对并构建长期防护体系。 一、遭遇攻击的紧急应…...
大数据零基础学习day1之环境准备和大数据初步理解
学习大数据会使用到多台Linux服务器。 一、环境准备 1、VMware 基于VMware构建Linux虚拟机 是大数据从业者或者IT从业者的必备技能之一也是成本低廉的方案 所以VMware虚拟机方案是必须要学习的。 （1）设置网关 打开VMware虚拟机，点击编辑…...
Java 加密常用的各种算法及其选择
在数字化时代，数据安全至关重要，Java 作为广泛应用的编程语言，提供了丰富的加密算法来保障数据的保密性、完整性和真实性。了解这些常用加密算法及其适用场景，有助于开发者在不同的业务需求中做出正确的选择。 一、对称加密算法…...
css的定位（position）详解：相对定位 绝对定位 固定定位
在 CSS 中，元素的定位通过 position 属性控制，共有 5 种定位模式：static（静态定位）、relative（相对定位）、absolute（绝对定位）、fixed（固定定位）和…...
【开发技术】.Net使用FFmpeg视频特定帧上绘制内容
目录 一、目的 二、解决方案 2.1 什么是FFmpeg 2.2 FFmpeg主要功能 2.3 使用Xabe.FFmpeg调用FFmpeg功能 2.4 使用 FFmpeg 的 drawbox 滤镜来绘制 ROI 三、总结 一、目的 当前市场上有很多目标检测智能识别的相关算法，当前调用一个医疗行业的AI识别算法后返回…...
蓝桥杯3498 01串的熵
问题描述 对于一个长度为 23333333的 01 串, 如果其信息熵为 11625907.5798， 且 0 出现次数比 1 少, 那么这个 01 串中 0 出现了多少次? #include<iostream> #include<cmath> using namespace std;int n 23333333;int main() {//枚举 0 出现的次数//因…...
Go 语言并发编程基础：无缓冲与有缓冲通道
在上一章节中，我们了解了 Channel 的基本用法。本章将重点分析 Go 中通道的两种类型 —— 无缓冲通道与有缓冲通道，它们在并发编程中各具特点和应用场景。 一、通道的基本分类 类型定义形式特点无缓冲通道make(chan T)发送和接收都必须准备好࿰…...
C#中的CLR属性、依赖属性与附加属性
CLR属性的主要特征 封装性： 隐藏字段的实现细节 提供对字段的受控访问 访问控制： 可单独设置get/set访问器的可见性 可创建只读或只写属性 计算属性： 可以在getter中执行计算逻辑 不需要直接对应一个字段 验证逻辑： 可以…...
虚拟电厂发展三大趋势：市场化、技术主导、车网互联
市场化：从政策驱动到多元盈利 政策全面赋能 2025年4月，国家发改委、能源局发布《关于加快推进虚拟电厂发展的指导意见》，首次明确虚拟电厂为“独立市场主体”，提出硬性目标：2027年全国调节能力≥2000万千瓦࿰…...