CentOS7 安装 Kafka 集群

Wed, 30 Oct 2019 01:22:25 +0800

环境

主机名	IP	操作系统	kafka 版本
kafka224	192.168.1.224	CentOS7.5	2.12
kafka225	192.168.1.225	CentOS7.5	2.12
kafka226	192.168.1.226	CentOS7.5	2.12

下载 kafka_2.12-2.1.0.tgz
zookeeper 连接: 192.168.1.221:2181,192.168.1.222:2181,192.168.1.223:2181

各节点初始配置

关闭 selinux、防火墙
部署 java 运行环境
创建数据目录
1

mkdir -p /var/lib/kafka
创建日志目录
1

mkdir -p /var/log/kafka

部署 kafka

登陆 kafka224，下载 kafka，解压至 /opt/ 下

修改日志目录

1
2

cd /opt/kafka/
rm -rf logs && ln -s /var/log/kafka logs

修改 /opt/kafka/config/server.properties

broker.id=224
listeners=PLAINTEXT://192.168.1.224:9092
log.dirs=/var/lib/kafka
zookeeper.connect=192.168.1.221:2181,192.168.1.222:2181,192.168.1.223:2181

启动脚本(/opt/kafka/bin/kafka-server-start.sh)太繁琐，我精简了下

#!/bin/bash
base_dir=$(dirname $0)
if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then
    export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
fi
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    # jvm 堆内存根据实际情况修改
    export KAFKA_HEAP_OPTS="-Xmx2G -Xms2G"
    # 开启 jmx
    export JMX_PORT=1099
fi
EXTRA_ARGS=${EXTRA_ARGS-'-name kafkaServer -loggc'}
EXTRA_ARGS="-daemon "$EXTRA_ARGS
exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$base_dir/../config/server.properties"

打包 kafka 目录，复制到 kafka225 和 kafka226 上，并修改 server.properties

# kafka225
broker.id=225
listeners=PLAINTEXT://192.168.1.225:9092
# kafka226
broker.id=226
listeners=PLAINTEXT://192.168.1.226:9092

启动集群(两种启动方式)

直接启动二进制
- 在每个节点上启动 kafka 服务
  1
  
  /opt/kafka/bin/kafka-server-start.sh

systemd 启动

创建 /usr/lib/systemd/system/kafka.service

[Unit]
Description=Kafka
Requires=network.service
After=network.service
[Service]
Environment=JAVA_HOME=/opt/jre
ExecStart=/opt/kafka/bin/kafka-server-start.sh
ExecStop=/opt/kafka/bin/kafka-server-stop.sh
Type=forking
KillMode=none
[Install]
WantedBy=multi-user.target

启动 kafka 服务

1
2

systemctl daemon-reload
systemctl start kafka

MQ on Colben Notes

CentOS7 安装 Kafka 集群

环境

各节点初始配置

部署 kafka

启动集群(两种启动方式)