# bigdata-sql-parser **Repository Path**: duhanmin_admin/bigdata-sql-parser ## Basic Information - **Project Name**: bigdata-sql-parser - **Description**: 数据血缘,支持spark sql,presto sql,tidb sql, flink sql, datax血缘,Spark/flink jar 运行命令的血缘解析;支持with语法 - **Primary Language**: Java - **License**: Apache-2.0 - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 38 - **Forks**: 17 - **Created**: 2021-08-18 - **Last Updated**: 2025-04-03 ## Categories & Tags **Categories**: database-dev **Tags**: 血缘, SQL, Spark, hive, flink ## README # 介绍 数据血缘,支持spark sql,presto sql,tidb sql, flink sql, Spark/flink jar 运行命令的血缘解析 * 特别提示:presto和spark均支持with语法 * 目前解析到表级别,可以扩展到字段级别 * 支持datax血缘(mysql/pg/s3) # 编译 mvn antlr4:antlr4 mvn clean package # 使用 * spark SQL ```kotlin val sql = "CREATE DATABASE IF NOT EXISTS bigdata" val statementData = SparkSQLHelper.getStatementData(sql) println(statementData.statement) ``` * presto SQL ```kotlin val sql = "INSERT INTO adc.fsfd with recursive t as (select a,b,v from a.x) select a,b,v from t" val statementData = PrestoSQLHelper.getStatementData(sql) println(statementData.statement) ``` * Tidb SQL/MySQL ```kotlin val sql = "insert into bigdata.user select * from users a left outer join address b on a.address_id = b.id" val statementData = PrestoSQLHelper.getStatementData(sql) println(statementData.statement) ``` * Flink SQL ```kotlin val sql = "CREATE SOURCE TABLE student_scores (\n" + " student_number varchar comment '学号', \n" + " student_name varchar comment '姓名', \n" + " subject varchar comment '学科',\n" + " score INT comment '成绩' \n" + ")\n" + "WITH (\n" + " type = \"dis\",\n" + " kafka.region = \"cn-north-1\"\n" + ") TIMESTAMP BY proctime proctime1" val statementData = StreamSQLHelper.getStatementData(sql).get(0) println(statementData.statement) ```