Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-692968: Async queries support #787

Open
wants to merge 22 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
763153f
Add async query support
sfc-gh-ext-simba-nl Nov 28, 2024
974613b
Add async test file
sfc-gh-ext-simba-nl Nov 28, 2024
a2f2701
Fix linux build by including unistd.h
sfc-gh-ext-simba-nl Nov 28, 2024
3829b60
Remove unnecessary test
sfc-gh-ext-simba-nl Nov 28, 2024
d947159
Add query status to C API, refactor some query parameters, move getti…
sfc-gh-ext-simba-nl Dec 3, 2024
8ab3758
Fix typo
sfc-gh-ext-simba-nl Dec 3, 2024
43c5653
Fix typo
sfc-gh-ext-simba-nl Dec 3, 2024
76ac1f1
Add more test cases, fix async fetching bugs
sfc-gh-ext-simba-nl Dec 4, 2024
c4fa76d
Fix bug with normal queries that go async after a while
sfc-gh-ext-simba-nl Dec 4, 2024
387a59f
fix typo
sfc-gh-ext-simba-nl Dec 4, 2024
cb067ec
Remove status check from fake table
sfc-gh-ext-simba-nl Dec 4, 2024
6cd5c78
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Dec 10, 2024
582913e
Fix linux warnings
sfc-gh-ext-simba-nl Dec 10, 2024
98cb876
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-jszczerbinski Dec 12, 2024
d3b94a0
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Dec 12, 2024
f84fc25
Improve error handling and logging
sfc-gh-ext-simba-nl Dec 12, 2024
97531c8
Merge branch 'SNOW-692968-async-queries-support' of https://github.co…
sfc-gh-ext-simba-nl Dec 12, 2024
f32ada0
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Dec 19, 2024
942ae0a
Fix memory issues in test cases
sfc-gh-ext-simba-nl Dec 19, 2024
ed33af6
Merge branch 'master' into SNOW-692968-async-queries-support
sfc-gh-ext-simba-nl Dec 20, 2024
75e6565
organize enums, add test
sfc-gh-ext-simba-nl Dec 21, 2024
9e457ae
Lower the rowcount for the test
sfc-gh-ext-simba-nl Dec 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions include/snowflake/client.h
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,25 @@ typedef enum SF_STMT_ATTRIBUTE {
SF_STMT_USER_REALLOC_FUNC
} SF_STMT_ATTRIBUTE;

/**
* The query status
*/
typedef enum SF_QUERY_STATUS {
SF_QUERY_STATUS_RUNNING,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's sort statuses alphabetically

SF_QUERY_STATUS_ABORTING,
SF_QUERY_STATUS_SUCCESS,
SF_QUERY_STATUS_FAILED_WITH_ERROR,
SF_QUERY_STATUS_ABORTED,
SF_QUERY_STATUS_QUEUED,
SF_QUERY_STATUS_FAILED_WITH_INCIDENT,
SF_QUERY_STATUS_DISCONNECTED,
SF_QUERY_STATUS_RESUMING_WAREHOUSE,
SF_QUERY_STATUS_QUEUED_REPAIRING_WAREHOUSE,
SF_QUERY_STATUS_RESTARTED,
SF_QUERY_STATUS_BLOCKED,
SF_QUERY_STATUS_NO_DATA
} SF_QUERY_STATUS;

/**
* Snowflake Error
*/
Expand Down Expand Up @@ -476,6 +495,8 @@ typedef struct SF_STMT {
SF_STATS *stats;
void *stmt_attrs;
sf_bool is_dml;
sf_bool is_async;
sf_bool is_async_initialized;

/**
* User realloc function used in snowflake_fetch
Expand Down Expand Up @@ -613,6 +634,25 @@ SF_STATUS STDCALL snowflake_get_attribute(
*/
SF_STMT *STDCALL snowflake_stmt(SF_CONNECT *sf);

/**
* Creates sf SNOWFLAKE_STMT context for async queries.
*
* @param sf The SF_CONNECT context.
* @param query_id the query id of the async query.
*
* @return sfstmt SNOWFLAKE_STMT context for async queries.
*/
SF_STMT* STDCALL snowflake_create_async_query_result(SF_CONNECT *sf, const char *query_id);

/**
* Get the status of a query
*
* @param sfstmt The SF_STMT context.
*
* @return The query status.
*/
SF_QUERY_STATUS STDCALL snowflake_get_query_status(SF_STMT *sfstmt);

/**
* Frees the memory used by a SF_QUERY_RESULT_CAPTURE struct.
* Note that this only frees the struct itself, and *not* the underlying
Expand Down Expand Up @@ -775,6 +815,14 @@ snowflake_stmt_get_attr(SF_STMT *sfstmt, SF_STMT_ATTRIBUTE type, void **value);
*/
SF_STATUS STDCALL snowflake_execute(SF_STMT *sfstmt);

/**
* Executes a statement asynchronously.
* @param sfstmt SNOWFLAKE_STMT context.
*
* @return 0 if success, otherwise an errno is returned.
*/
SF_STATUS STDCALL snowflake_async_execute(SF_STMT *sfstmt);

/**
* Executes a statement with capture.
* @param sfstmt SNOWFLAKE_STMT context.
Expand Down
3 changes: 3 additions & 0 deletions include/snowflake/platform.h
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,9 @@ void STDCALL sf_memory_error_handler();
// this should be called by application before any calls of sfclient
void STDCALL sf_exception_on_memory_failure();

void STDCALL sf_sleep_ms(int sleep_ms);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be in out public api?



#ifdef __cplusplus
}
#endif
Expand Down
Loading
Loading