public class LineageUtils implements NodeProcessor {// 存放输入表TreeSet<String> inputTableList = new TreeSet<String>();// 存放目标表TreeSet<String> outputTableList = new TreeSet<String>();//存放with子句中的别名, 最终的输入表是 inputTableList减去withTableListTreeSet<String> withTableList = new TreeSet<String>();public TreeSet getInputTableList() {return inputTableList;}public TreeSet getOutputTableList() {return outputTableList;}public TreeSet getWithTableList() {return withTableList;}public Object process(Node nd, Stack stack, NodeProcessorCtx procCtx, Object... nodeOutputs) throws SemanticException {ASTNode pt = (ASTNode) nd;switch (pt.getToken().getType()) {//create语句case HiveParser.TOK_CREATETABLE: {String createName = BaseSemanticAnalyzer.getUnescapedName((ASTNode) pt.getChild(0));outputTableList.add(createName);break;}//insert语句case HiveParser.TOK_TAB: {// System.out.println(pt.getChildCount() + "tab");String insertName = BaseSemanticAnalyzer.getUnescapedName((ASTNode) pt.getChild(0));outputTableList.add(insertName);//  System.out.println("insertName  " + insertName);break;}//from语句case HiveParser.TOK_TABREF: {ASTNode tabTree = (ASTNode) pt.getChild(0);String fromName = (tabTree.getChildCount() == 1) ? BaseSemanticAnalyzer.getUnescapedName((ASTNode) tabTree.getChild(0)) : BaseSemanticAnalyzer.getUnescapedName((ASTNode) tabTree.getChild(0)) + "." + tabTree.getChild(1);inputTableList.add(fromName);break;}// with.....语句case HiveParser.TOK_CTE: {for (int i = 0; i < pt.getChildCount(); i++) {ASTNode temp = (ASTNode) pt.getChild(i);String cteName = BaseSemanticAnalyzer.getUnescapedName((ASTNode) temp.getChild(1));withTableList.add(cteName);}break;}}return null;}public void getLineageInfo(String query) throws ParseException, SemanticException {ParseDriver pd = new ParseDriver();ASTNode tree = pd.parse(query);while ((tree.getToken() == null) && (tree.getChildCount() > 0)) {tree = (ASTNode) tree.getChild(0);}inputTableList.clear();outputTableList.clear();withTableList.clear();Map<Rule, NodeProcessor> rules = new LinkedHashMap<Rule, NodeProcessor>();Dispatcher disp = new DefaultRuleDispatcher(this, rules, null);GraphWalker ogw = new DefaultGraphWalker(disp);ArrayList topNodes = new ArrayList();topNodes.add(tree);ogw.startWalking(topNodes, null);}//进行测试，sql语句是瞎写的，但是语法是对的public static void main(String[] args) throws IOException, ParseException, SemanticException {//String query = "insert into qc.tables_lins_cnt partition(dt='2016-09-15') select a.x from (select x from cc group by x) a left  join yy b on a.id = b.id left join (select * from zz where id=1) c on c.id=b.id";// String query ="from (select id,name from xx where id=1) a insert overwrite table  dsl.dwm_all_als_active_d partition (dt='main') select id group by id insert overwrite table  dsl.dwm_all_als_active_d2 partition (dt='main') select name group by name";String query = "with q1 as ( select key from src where key = '5'), q2 as ( select key from with1 a inner join with2 b on a.id = b.id) insert overwrite table temp.dt_mobile_play_d_tmp2 partition(dt='2018-07-17') select * from q1 cross join q2";LineageUtils lep = new LineageUtils();lep.getLineageInfo(query);System.out.println("Input tables = " + lep.getInputTableList());System.out.println("Output tables = " + lep.getOutputTableList());System.out.println("with tables = " + lep.getWithTableList());}
}