# springboot-kafka-starter

**Repository Path**: richardfyoung/springboot-kafka-starter

## Basic Information

- **Project Name**: springboot-kafka-starter
- **Description**: 1. 通过logback 日志增强收集到kafka里
2. 里面有consumer、producer测试kafka
- **Primary Language**: Java
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2024-12-28
- **Last Updated**: 2024-12-30

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

## 1、 config consumer producer 只是方便你测试kafka的联通性
## 2、 真正有用的就是项目的 logback-spring.xml
# 常见的实现

## 1. 社区实现   （本项目采用的实现方式）
<dependency>
    <groupId>com.github.danielwegener</groupId>
    <artifactId>logback-kafka-appender</artifactId>
    <version>0.2.0</version>
</dependency>


## 2. Logstash Logback Encoder
<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>7.4</version>
</dependency>
## 3. Spring Cloud Stream
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-stream-kafka</artifactId>
    <version>4.0.4</version>
</dependency>

## 4. Log4j Kafka Appender (如果使用 Log4j2)
<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-log4j-appender</artifactId>
    <version>3.4.0</version>
</dependency>

## 5. 自定义的社区实现
<dependency>
    <groupId>com.gitee.dengmin</groupId>
    <artifactId>logback-kafka-appender</artifactId>
    <version>1.0.0</version>
</dependency>

## 6. KafkaAppender-Logback

<dependency>
    <groupId>com.github.mgljava</groupId>
    <artifactId>kafka-appender</artifactId>
    <version>1.0.0</version>
</dependency>


 **_推荐使用 Logstash Logback Encoder，因为它：
1. 维护活跃
功能完善
社区支持好
与 ELK 栈集成良好
支持 JSON 格式化
_** 



### 1. docker部署kafka: 
https://juejin.cn/post/7350957295192539145

### 2. 拉取镜像

`docker pull bitnami/kafka:3.5`


### 3.启动

```
docker run -itd --name kafka -p 9092:9092 \
    -e KAFKA_CFG_NODE_ID=1 \
    -e KAFKA_CFG_PROCESS_ROLES=controller,broker \
    -e KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093 \
    -e KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT \
    -e KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=1@localhost:9093 \
    -e KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER \
    -v ~/Working/my-data/kafka:/bitnami/kafka \
    bitnami/kafka:3.5
```






### 4. 创建topic列表

```
docker exec -it kafka bash /opt/bitnami/kafka/bin/kafka-topics.sh \
    --bootstrap-server localhost:9092 --create --topic zfq-test-topic
```


### 5. 查看topic列表

```
docker exec -it kafka bash /opt/bitnami/kafka/bin/kafka-topics.sh \
    --bootstrap-server localhost:9092 --list
```


### 6. 控制台查看不带key
`docker exec -it kafka kafka-console-consumer.sh --topic zfq-test-topic --from-beginning --bootstrap-server localhost:9092`

### 7. 控制台查看带key


```
docker exec -it kafka kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 \
  --topic zfq-test-topic \
  --property print.key=true \
  --from-beginning | grep "^my-key"
```



### 8.检查jar中是否存在类
 
```
 mvn dependency:copy -Dartifact=net.logstash.logback:logstash-logback-encoder:7.3:jar -DoutputDirectory=./temp
    cd temp
    jar tf logstash-logback-encoder-7.3.jar | grep KafkaAppender
```




效果 
![输入图片说明](image.png)