Code

Snowvation snowpark streaming #2745

Code

Draft

sfc-gh-mvashishtha wants to merge 5 commits intomainsnowflakedb/snowpark-python:mainfrom mvashishtha/snowvation-snowpark-streaming-2024snowflakedb/snowpark-python:mvashishtha/snowvation-snowpark-streaming-2024Copy head branch name to clipboard

+544-61

Commits

Commits on Dec 11, 2024

• Make streams ingest via a UDTF. The UDTF runs forever and can't connect to the kafka cluster.

  Show description for df04f58
  sfc-gh-mvashishtha

  committed
  df04f58

Commits on Dec 12, 2024

• Join UDTF result to a static table and write the result to a dynamic table

  Show description for 6cb6e16
  sfc-gh-mvashishtha

  committed
  6cb6e16

• Instead of writing result to dynamic table, call a UDF to write stream

  Show description for 6418c58
  sfc-gh-mvashishtha

  committed
  6418c58

• Read stream from table

  Show description for e60adfe
  sfc-gh-mvashishtha

  committed
  e60adfe

Commits on Dec 13, 2024

• Support writing to dynamic table if stream source is table. Otherwise, use UDTF to stream to table.

  Show description for f926024
  sfc-gh-mvashishtha

  committed
  f926024