# 大数据离线电商数仓项目

**Repository Path**: sky_blue_gl/Data-Warehouse

## Basic Information

- **Project Name**: 大数据离线电商数仓项目
- **Description**: 😃本仓库为尚硅谷电商离线数仓项目4.0,主要技术栈为Hadoop，Flume，Hbase，Kafka，zookeeper，Spark，Hive等相关技术栈，个人在学习过程中完成了笔记的编写，绘制了数仓分层的分层结构图，以及表关系的整理，可辅助学习者对项目进行更进一步的理解。
- **Primary Language**: Unknown
- **License**: AFL-3.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 21
- **Forks**: 19
- **Created**: 2022-10-03
- **Last Updated**: 2025-07-14

## Categories & Tags

**Categories**: Uncategorized

**Tags**: hadoop, hive

## README

## 💎💎💎

<p align="center"><a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-hadoop-green" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-hive-yellowgreen" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-zookeeper-yellowgreen" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-hbase-yellowgreen" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-flume-yellowgreen" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-kafka-yellowgreen" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-spark-yellowgreen" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-sqoop-yellowgreen" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-flink-yellowgreen" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-java-yellowgreen" alt="github"></a></p>

<p align="center"><a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-scala-yellowgreen" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei"><img src="https://img.shields.io/badge/-AI-yellowgreen" alt="github"></a>&emsp;<a href="https://gitee.com/fanggaolei" align="center"><img src="https://img.shields.io/badge/-ssm-yellowgreen" alt="github"></a></p>

>### **原创项目架构图：** [https://kdocs.cn/l/ccj89kGKAYFY](https://kdocs.cn/l/ccj89kGKAYFY)

## ✈️1.0 架构说明

![image-20220831234500440](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20220831234500440.png)

![image-20220901111526317](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20220901111526317.png)

​		**技术选型主要考虑因素：数据量大小、业务需求、行业内经验、技术成熟度、开发维护成本、总成本预算。**

>数据采集传输：Flume,.Kafka,Sqoop;Logstash,DataX
>
>数据存储：MySQL,HDFS,HBase,Redis,MongoDB
>
>数据计算：Hive,Tez,Spark,Flink,Storm
>
>数据查询：Presto,Kylin,Impala,Duid,ClickHouse,Doris
>
>数据可视化：Echats,Superset,QuickBI,DataV
>
>任务调度：Azkaban,Oozie,DolphinScheduler,Airflow
>
>集群监控：Zabbi，Prometheus
>
>元数据管理：Atlas
>
>权限管理：Ranger,Sentry

## 2.0 🚀数据说明

![image-20221108154359536](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154359536.png)

![image-20221108154430923](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154430923.png)

![image-20221108154451158](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154451158.png)

![image-20221108154507757](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154507757.png)

![image-20221108154523990](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154523990.png)

## 3.0🙂数据生成

![image-20221108154703847](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154703847.png)

![image-20221108154648900](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154648900.png)

## 4.0🐘建模理论

![image-20221108154722469](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154722469.png)

## 5.0😎数仓建模

![image-20221108154754349](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154754349.png)

![image-20221108154813954](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154813954.png)

![image-20221108154836124](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154836124.png)

![image-20221108154852970](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108154852970.png)

## 6.0🍔数据流分析

![image-20221108155009660](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108155009660.png)

![image-20221108155034840](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108155034840.png)

![image-20221108155055147](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108155055147.png)

![image-20221108155108915](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108155108915.png)

![image-20221108155123652](https://pic-1313413291.cos.ap-nanjing.myqcloud.com/image-20221108155123652.png)