# flink-iceberg-demo

**Repository Path**: index103000/flink-iceberg-demo

## Basic Information

- **Project Name**: flink-iceberg-demo
- **Description**: flink iceberg integration tests, jobs running on yarn.
- **Primary Language**: Unknown
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 2
- **Created**: 2022-03-25
- **Last Updated**: 2022-03-25

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Flink iceberg integration tests 
Testing flink jobs on yarn, jobs integrated with iceberg, hive, hbase, kafka, elasticsearch, etc.

## Used components

* Apache Hadoop 3.2
* Kafka 2.0.0
* Hbase 2.0.0
* Hive 3.1.2
* ELK 7.9.1
* Iceberg 0.10.0
* Flink 1.11.3

## How to Run

* Deploy a testing big data platform using docker-compose. [Download here](https://github.com/spancer/bigdata-docker-compose)

* Build each jobs and run it on yarn with yarn session or pre-job. 

## TODO
1. testing ozone and alluxio.
2. deep integration with iceberg and flink sql.
3. running jobs on k8s.