# kafka_nginx
**Repository Path**: wangzhe_spring/kafka_nginx
## Basic Information
- **Project Name**: kafka_nginx
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No
## Statistics
- **Stars**: 0
- **Forks**: 0
- **Created**: 2020-12-15
- **Last Updated**: 2020-12-19
## Categories & Tags
**Categories**: Uncategorized
**Tags**: None
## README
## 使用Kafka做日志收集
#### 1. 安装Kafka
1.1 安装zookeeper
```shell
# 解压zookeeper
tar -zxvf zookeeper-3.4.14.tar.gz -C /root
# 修改zookeeper配置
cd /root/zookeeper-3.4.14/conf
cp zoo_sample.cfg zoo.cfg
vim zoo.cfg
# 修改以下配置
clientPort=2181
dataDir=/root/zookeeper-3.4.14/data
dataLogDir=/root/zookeeper-3.4.14/data/logs
# 启动zookeeper
cd /root/zookeeper-3.4.14/bin
sh zkServer.sh start
# 查看zookeeper启动情况
sh zkServer.sh status
```
1.2 安装Kafka
```shell
# 解压Kafka
tar -zxvf kafka_2.12-1.0.2.tgz -C /root
# 配置环境变量
vim /etc/profile
# 添加以下配置
export KAFKA_HOME=/root/kafka_2.12-1.0.2
export PATH=$PATH:$KAFKA_HOME/bin
# 修改Kafka配置
vim /root/kafka_2.12-1.0.2/config/server.properties
# 添加以下配置
listeners=PLAINTEXT://192.168.42.170:9092
advertised.listeners=PLAINTEXT://192.168.42.170:9092
log.dirs=/root/kafka_2.12-1.0.2/kafka-logs
zookeeper.connect=localhost:2181/myKafka
# 启动Kafka
cd /root/kafka_2.12-1.0.2
kafka-server-start.sh -daemon config/server.properties
```
#### 2. 安装Nginx
```shell
# 安装编译环境
yum install git gcc gcc-c++ make automake vim wget libevent -y
# 下载nginx
wget http://nginx.org/download/nginx-1.15.6.tar.gz
# 解压nginx
tar -zxvf nginx-1.15.6.tar.gz -C /root
```
#### 3. Nginx配置kafka模块ngx_kafka_module
```shell
# 下载安装librdkafka
cd /root
git clone https://github.com/edenhill/librdkafka
cd librdkafka
./configure
make
sudo make install
```
```shell
# 下载kafka模块ngx_kafka_module
cd /root/nginx-1.15.6
git clone https://github.com/brg-liuwei/ngx_kafka_module
# nginx添加ngx_kafka_module模块
./configure --add-module=/root/nginx-1.15.6/ngx_kafka_module
# 编译
make && make install
# 查看模块是否安装上
/usr/local/nginx/sbin/nginx -V
```
```shell
# 修改nginx配置
vim /usr/local/nginx/conf/nginx.conf
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# 配置kafka节点
kafka;
kafka_broker_list 192.168.42.170:9092;
server {
listen 80;
server_name localhost;
# 配置全局跨域
add_header Access-Control-Allow-Origin *;
add_header Access-Control-Allow-Methods 'GET,POST';
add_header Access-Control-Allow-Headers 'DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization';
location / {
alias /usr/local/nginx/my_html/;
allow all;
}
location /my_log/ {
# 将日志发送到my_log主题
kafka_topic my_log;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
```
```shell
# 启动nginx
/usr/local/nginx/sbin/nginx
```
#### 4. 开发HTML页面
login.html
```html
Title
```
job.html
```html
Title
```