Currently, DataSQRL supports straight up FlinkSQL INSERT statements but doesn't capture them as ExportNodes in the DAG, which means these statements (and the entire processing tree leading up to them) is not captured in the graph.
This only affect edge use cases, but it would be nice to capture INSERTs as ExportNodes in the DAG and therefore ensure we capture the lineage. Otherwise, people could consider this a bug because the plan doesn't look right.
It also means that if the tables we are inserting into are mutation tables, we are not capturing those either and won't plan those. That means the combination of INSERT with mutation tables does not currently work.
Currently, DataSQRL supports straight up FlinkSQL
INSERTstatements but doesn't capture them as ExportNodes in the DAG, which means these statements (and the entire processing tree leading up to them) is not captured in the graph.This only affect edge use cases, but it would be nice to capture INSERTs as ExportNodes in the DAG and therefore ensure we capture the lineage. Otherwise, people could consider this a bug because the plan doesn't look right.
It also means that if the tables we are inserting into are mutation tables, we are not capturing those either and won't plan those. That means the combination of INSERT with mutation tables does not currently work.