Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-692968: Async queries support #787

Merged
merged 62 commits into from
Mar 5, 2025
Merged
Changes from 18 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
763153f
Add async query support
sfc-gh-ext-simba-nl Nov 28, 2024
974613b
Add async test file
sfc-gh-ext-simba-nl Nov 28, 2024
a2f2701
Fix linux build by including unistd.h
sfc-gh-ext-simba-nl Nov 28, 2024
3829b60
Remove unnecessary test
sfc-gh-ext-simba-nl Nov 28, 2024
d947159
Add query status to C API, refactor some query parameters, move getti…
sfc-gh-ext-simba-nl Dec 3, 2024
8ab3758
Fix typo
sfc-gh-ext-simba-nl Dec 3, 2024
43c5653
Fix typo
sfc-gh-ext-simba-nl Dec 3, 2024
76ac1f1
Add more test cases, fix async fetching bugs
sfc-gh-ext-simba-nl Dec 4, 2024
c4fa76d
Fix bug with normal queries that go async after a while
sfc-gh-ext-simba-nl Dec 4, 2024
387a59f
fix typo
sfc-gh-ext-simba-nl Dec 4, 2024
cb067ec
Remove status check from fake table
sfc-gh-ext-simba-nl Dec 4, 2024
6cd5c78
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Dec 10, 2024
582913e
Fix linux warnings
sfc-gh-ext-simba-nl Dec 10, 2024
98cb876
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-jszczerbinski Dec 12, 2024
d3b94a0
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Dec 12, 2024
f84fc25
Improve error handling and logging
sfc-gh-ext-simba-nl Dec 12, 2024
97531c8
Merge branch 'SNOW-692968-async-queries-support' of https://github.co…
sfc-gh-ext-simba-nl Dec 12, 2024
f32ada0
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Dec 19, 2024
942ae0a
Fix memory issues in test cases
sfc-gh-ext-simba-nl Dec 19, 2024
ed33af6
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Dec 20, 2024
75e6565
organize enums, add test
sfc-gh-ext-simba-nl Dec 21, 2024
9e457ae
Lower the rowcount for the test
sfc-gh-ext-simba-nl Dec 21, 2024
2aaea2a
Have get_query_metadata return a struct instead of a string
sfc-gh-ext-simba-nl Jan 20, 2025
9003f03
merge mastre
sfc-gh-ext-simba-nl Jan 20, 2025
eb4ae08
Fix build issue with merge master
sfc-gh-ext-simba-nl Jan 20, 2025
a793549
Fix merge issue
sfc-gh-ext-simba-nl Jan 20, 2025
45fa5a3
Fix formatting
sfc-gh-ext-simba-nl Jan 20, 2025
734a064
remove sf_sleep_ms from platform header file
sfc-gh-ext-simba-nl Jan 22, 2025
67e6db3
Move sf_sleep_ms from platform to util
sfc-gh-ext-simba-nl Jan 22, 2025
7671e14
Fix linux compilation and remove extra newline in platform.h
sfc-gh-ext-simba-nl Jan 22, 2025
4efbd8b
Fix build errors
sfc-gh-ext-simba-nl Jan 22, 2025
04100c3
Add util.h to test file
sfc-gh-ext-simba-nl Jan 22, 2025
c89f086
Merge master
sfc-gh-ext-simba-nl Jan 29, 2025
8b39c03
Add missing files from merge
sfc-gh-ext-simba-nl Jan 29, 2025
c9d9bbb
Fix formatting, add test case for retries
sfc-gh-ext-simba-nl Jan 30, 2025
17d6236
Minor logic fix, uncomment test cases
sfc-gh-ext-simba-nl Jan 30, 2025
f298aa2
change get results logic to query /queries/{sfqid}/result instead of …
sfc-gh-ext-simba-nl Jan 31, 2025
73a1a98
merge master
sfc-gh-ext-simba-nl Jan 31, 2025
79a8df0
Fix async timeout test
sfc-gh-ext-simba-nl Feb 1, 2025
2f99bdd
Remove unused variables
sfc-gh-ext-simba-nl Feb 1, 2025
31ac782
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Feb 3, 2025
ae46e5a
Forgot to close connection on test case
sfc-gh-ext-simba-nl Feb 4, 2025
fa9b3e0
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Feb 4, 2025
32f127a
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Feb 4, 2025
6891894
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Feb 5, 2025
5788a0f
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Feb 6, 2025
bde8ccc
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Feb 10, 2025
2030329
Make less unnecessary calls to get query metadata, rename to make thi…
sfc-gh-ext-simba-nl Feb 11, 2025
d2371b7
Uncomment test cases
sfc-gh-ext-simba-nl Feb 11, 2025
5b1198f
Fix formatting
sfc-gh-ext-simba-nl Feb 11, 2025
a6c707c
Minor fix
sfc-gh-ext-simba-nl Feb 12, 2025
222b94b
Fix code quality warnings
sfc-gh-ext-simba-nl Feb 13, 2025
79e5e20
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-dprzybysz Feb 19, 2025
11b2c40
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Feb 19, 2025
2313d73
Merge branch 'SNOW-692968-async-queries-support' of https://github.co…
sfc-gh-ext-simba-nl Feb 19, 2025
6b37e6a
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Feb 20, 2025
e86aa64
Fix build error
sfc-gh-ext-simba-nl Feb 21, 2025
e711c27
Fix filename
sfc-gh-ext-simba-nl Feb 21, 2025
2c27a38
Fix windows VS17 build error
sfc-gh-ext-simba-nl Mar 3, 2025
463e016
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-jszczerbinski Mar 4, 2025
284de79
Revert fix in SnowflakeCommon, remove unused stats from SF_QUERY_META…
sfc-gh-ext-simba-nl Mar 4, 2025
6e448c8
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-jszczerbinski Mar 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions include/snowflake/client.h
Original file line number Diff line number Diff line change
@@ -294,6 +294,25 @@ typedef enum SF_STMT_ATTRIBUTE {
SF_STMT_USER_REALLOC_FUNC
} SF_STMT_ATTRIBUTE;

/**
* The query status
*/
typedef enum SF_QUERY_STATUS {
SF_QUERY_STATUS_RUNNING,
SF_QUERY_STATUS_ABORTING,
SF_QUERY_STATUS_SUCCESS,
SF_QUERY_STATUS_FAILED_WITH_ERROR,
SF_QUERY_STATUS_ABORTED,
SF_QUERY_STATUS_QUEUED,
SF_QUERY_STATUS_FAILED_WITH_INCIDENT,
SF_QUERY_STATUS_DISCONNECTED,
SF_QUERY_STATUS_RESUMING_WAREHOUSE,
SF_QUERY_STATUS_QUEUED_REPAIRING_WAREHOUSE,
SF_QUERY_STATUS_RESTARTED,
SF_QUERY_STATUS_BLOCKED,
SF_QUERY_STATUS_NO_DATA
} SF_QUERY_STATUS;

/**
* Snowflake Error
*/
@@ -476,6 +495,8 @@ typedef struct SF_STMT {
SF_STATS *stats;
void *stmt_attrs;
sf_bool is_dml;
sf_bool is_async;
sf_bool is_async_initialized;

/**
* User realloc function used in snowflake_fetch
@@ -613,6 +634,25 @@ SF_STATUS STDCALL snowflake_get_attribute(
*/
SF_STMT *STDCALL snowflake_stmt(SF_CONNECT *sf);

/**
* Creates sf SNOWFLAKE_STMT context for async queries.
*
* @param sf The SF_CONNECT context.
* @param query_id the query id of the async query.
*
* @return sfstmt SNOWFLAKE_STMT context for async queries.
*/
SF_STMT* STDCALL snowflake_create_async_query_result(SF_CONNECT *sf, const char *query_id);

/**
* Get the status of a query
*
* @param sfstmt The SF_STMT context.
*
* @return The query status.
*/
SF_QUERY_STATUS STDCALL snowflake_get_query_status(SF_STMT *sfstmt);

/**
* Frees the memory used by a SF_QUERY_RESULT_CAPTURE struct.
* Note that this only frees the struct itself, and *not* the underlying
@@ -775,6 +815,14 @@ snowflake_stmt_get_attr(SF_STMT *sfstmt, SF_STMT_ATTRIBUTE type, void **value);
*/
SF_STATUS STDCALL snowflake_execute(SF_STMT *sfstmt);

/**
* Executes a statement asynchronously.
* @param sfstmt SNOWFLAKE_STMT context.
*
* @return 0 if success, otherwise an errno is returned.
*/
SF_STATUS STDCALL snowflake_async_execute(SF_STMT *sfstmt);

/**
* Executes a statement with capture.
* @param sfstmt SNOWFLAKE_STMT context.
3 changes: 3 additions & 0 deletions include/snowflake/platform.h
Original file line number Diff line number Diff line change
@@ -142,6 +142,9 @@ void STDCALL sf_memory_error_handler();
// this should be called by application before any calls of sfclient
void STDCALL sf_exception_on_memory_failure();

void STDCALL sf_sleep_ms(int sleep_ms);


#ifdef __cplusplus
}
#endif
Loading