-
Notifications
You must be signed in to change notification settings - Fork 36
FEAT: BCP implementation in mssql-python driver using rust #402
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 8 commits
bcad33c
284dd2f
387d1e9
72f6953
1d16f97
f8f6737
f7a6606
4ca4969
2d9faf7
62f4795
e60cf2d
1501ff8
ebd4386
cf42d22
ffad1d6
b0c536f
ce1e64d
f698b97
099f09f
854d983
6fad210
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -2451,6 +2451,99 @@ | |||||
| ) | ||||||
| return True | ||||||
|
|
||||||
| def _bulkcopy(self, table_name: str, data, **kwargs): | ||||||
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||
| """ | ||||||
| Perform bulk copy operation using Rust-based implementation. | ||||||
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||
|
|
||||||
| Args: | ||||||
| table_name: Target table name | ||||||
| data: Iterable of tuples/lists containing row data | ||||||
| **kwargs: Additional options passed to the Rust bulkcopy method | ||||||
| - batch_size: Number of rows per batch (default: 1000) | ||||||
| - timeout: Timeout in seconds (default: 30) | ||||||
| - column_mappings: List of tuples mapping source column index to target column name | ||||||
|
|
||||||
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||
| Returns: | ||||||
| Dictionary with rows_copied, batch_count, and elapsed_time | ||||||
|
|
||||||
| Raises: | ||||||
| ImportError: If mssql_py_core is not installed | ||||||
| ValueError: If parameters are invalid | ||||||
| RuntimeError: If connection string is not available | ||||||
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||
| """ | ||||||
| try: | ||||||
| import mssql_py_core | ||||||
| except ImportError as exc: | ||||||
| raise ImportError( | ||||||
| "Bulk copy requires mssql_py_core Rust library. " | ||||||
| "Install from BCPRustWheel directory." | ||||||
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||
| ) from exc | ||||||
|
|
||||||
| # Validate inputs | ||||||
| if not table_name or not isinstance(table_name, str): | ||||||
| raise ValueError("table_name must be a non-empty string") | ||||||
|
|
||||||
| # Extract and validate kwargs with defaults | ||||||
| batch_size = kwargs.get("batch_size", 1000) | ||||||
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||
| timeout = kwargs.get("timeout", 30) | ||||||
subrata-ms marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
| if batch_size <= 0: | ||||||
| raise ValueError(f"batch_size must be positive, got {batch_size}") | ||||||
| if timeout <= 0: | ||||||
| raise ValueError(f"timeout must be positive, got {timeout}") | ||||||
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||
|
|
||||||
| # Get and parse connection string | ||||||
| if not hasattr(self.connection, "connection_str"): | ||||||
| raise RuntimeError("Connection string not available for bulk copy") | ||||||
|
|
||||||
| params = { | ||||||
| k.strip().lower(): v.strip() | ||||||
|
||||||
| k.strip().lower(): v.strip() | |
| k.strip().lower(): v |
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Fixed
Show fixed
Hide fixed
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
subrata-ms marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Outdated
Copilot
AI
Jan 16, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log statement could potentially expose sensitive connection details. While it doesn't directly log credentials, logging connection details should be done with caution. Consider using sanitized logging similar to other parts of the codebase that handle connection strings.
| logger.debug("Bulk copy connecting to %s/%s", context["server"], context["database"]) | |
| logger.debug("Bulk copy establishing connection") |
Uh oh!
There was an error while loading. Please reload this page.