FIX: Cursor.describe invalid data

[dlevy-msft-sql]

Work Item / Issue Reference

GitHub Issue: #352

---

Summary

This PR fixes cursor.description to properly return SQL type codes per DB-API 2.0 specification, while maintaining backwards compatibility with libraries like pandas and polars that compare against Python types.

The original issue was reported when using polars pl.read_database() which failed with:

ComputeError: could not append value: 2013-01-01 of type: date to the builder

---

Key Changes

1. New SqlTypeCode Class

A dual-compatible type code that compares equal to both:

• SQL type integers (e.g., desc[1] == -9) for DB-API 2.0 compliance
• Python types (e.g., desc[1] == str) for pandas/polars/library compatibility

cursor.execute("SELECT id, name FROM users")
desc = cursor.description

# Style 1: Compare with Python types (pandas/polars compatibility)
if desc[0][1] == int:
    print("Integer column")

# Style 2: Compare with SQL type codes (DB-API 2.0)
if desc[0][1] == 4:  # SQL_INTEGER
    print("Integer column")

# Get raw SQL type code
type_code = int(desc[0][1])  # Returns 4 for SQL_INTEGER

2. SQL Server Spatial Type Support

Added handling for SQL Server User-Defined Types (SQL_SS_UDT = -151):

• geography (e.g., GPS coordinates, regions)
• geometry (e.g., shapes, planar coordinates)
• hierarchyid (e.g., org charts, tree structures)

3. ODBC 3.x Date/Time Type Codes

Added support for:

• SQL_TYPE_DATE (91)
• SQL_TYPE_TIME (92)
• SQL_TYPE_TIMESTAMP (93)
• SQL_TYPE_TIMESTAMP_WITH_TIMEZONE (95)

4. Hash/Equality Contract Fix

Made SqlTypeCode intentionally unhashable since __eq__ compares to multiple types with different hash values. Attempting to hash raises TypeError with a helpful message guiding users to use int(type_code) as dict keys instead.

---

Files Changed

File	Changes
mssql_python/cursor.py	Added SqlTypeCode class, updated _initialize_description(), fixed _build_converter_map()
mssql_python/constants.py	Added SQL_SS_UDT, SQL_SS_TIME2, SQL_SS_XML constants
mssql_python/pybind/ddbc_bindings.cpp	Added C++ handling for spatial types, datetime2, smalldatetime
mssql_python/__init__.py	Exported SqlTypeCode
mssql_python/mssql_python.pyi	Added type stubs for SqlTypeCode
mssql_python/connection.py	Updated converter methods to accept SqlTypeCode
tests/test_002_types.py	Added 18 SqlTypeCode unit tests
tests/test_004_cursor.py	Added cursor description tests, thread safety tests
tests/test_017_spatial_types.py	NEW: 37 spatial type tests (geography, geometry, hierarchyid)
tests/test_018_polars_integration.py	NEW: 6 polars integration tests
requirements.txt	Added polars for integration testing
CHANGELOG.md	Documented changes

---

Usage Examples

SqlTypeCode - Dual Compatibility

cursor.execute("SELECT id, name FROM users")
desc = cursor.description

# Style 1: Compare with Python types (pandas/polars compatibility)
if desc[0][1] == int:
    print("Integer column")

# Style 2: Compare with SQL type codes (DB-API 2.0)
if desc[0][1] == 4:  # SQL_INTEGER
    print("Integer column")

# Get raw SQL type code
type_code = int(desc[0][1])  # Returns 4 for SQL_INTEGER

Polars Integration (Original Issue)

import polars as pl
import mssql_python

conn = mssql_python.connect("Server=myserver;Database=mydb;...")

# This now works correctly!
df = pl.read_database(
    query="SELECT * FROM Sales.Customers",
    connection=conn
)
print(df)

Spatial Types

# Geography (GPS coordinates)
cursor.execute("""
    INSERT INTO locations (geo) 
    VALUES (geography::STGeomFromText('POINT(-122.349 47.651)', 4326))
""")

# Fetch as binary
cursor.execute("SELECT geo FROM locations")
geo_bytes = cursor.fetchone()[0]  # Returns bytes

# Or fetch as WKT text
cursor.execute("SELECT geo.STAsText() FROM locations")
wkt = cursor.fetchone()[0]  # Returns "POINT (-122.349 47.651)"

---

Testing

Test Coverage

• ✅ All existing tests pass (1425 passed)
• ✅ 18 new SqlTypeCode unit tests
• ✅ 37 new spatial type tests (geography, geometry, hierarchyid)
• ✅ 6 new polars integration tests
• ✅ Thread safety tests for _column_metadata
• ✅ Binary parameter round-trip tests for spatial types
• 📊 Diff Coverage: 93%
• 📊 Overall Coverage: 77%

Polars Integration Tests

Test	Description
test_polars_read_database_basic	Basic data types (INT, NVARCHAR, FLOAT)
test_polars_read_database_with_dates	DATE columns - the original failure case
test_polars_read_database_all_common_types	All common SQL Server types
test_polars_read_database_with_nulls	NULL value handling
test_polars_read_database_large_result	1000 row result set
test_cursor_description_type_code_polars_compatibility	Direct type comparison verification

---

Breaking Changes

None. The SqlTypeCode class maintains full backwards compatibility:

• ✅ cursor.description[i][1] == str still works
• ✅ cursor.description[i][1] == -9 also works
• ✅ int(cursor.description[i][1]) returns raw SQL type code

---

Related Issues

• Fixes Cursor.description returning incorrect values #352 - Cursor.description returning incorrect values

[Copilot]

Pull request overview

This PR fixes issue #352 by implementing two main improvements: (1) correcting cursor.description to return SQL type codes (integers) instead of Python type objects, ensuring DB-API 2.0 specification compliance, and (2) adding support for SQL Server spatial data types (geography, geometry, hierarchyid) by handling the SQL_SS_UDT type code (-151).

Key changes:

• Fixed cursor.description[i][1] to return SQL type integer codes (e.g., 4, -9) instead of Python types (int, str) per DB-API 2.0 spec
• Added SQL_SS_UDT (-151) support for SQL Server User-Defined Types including spatial data types
• Updated output converter lookup to use SQL type codes consistently throughout the codebase

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
mssql_python/cursor.py	Changed cursor.description to return SQL type codes instead of Python types; added _column_metadata storage; updated _build_converter_map to use SQL type codes; added mappings for UDT, XML, DATETIME2, SMALLDATETIME types
mssql_python/constants.py	Added SQL_SS_UDT = -151 constant for SQL Server User-Defined Types
mssql_python/pybind/ddbc_bindings.cpp	Added C++ constants for SQL_SS_UDT, SQL_DATETIME2, SQL_SMALLDATETIME; implemented UDT handling in SQLGetData_wrap, FetchBatchData, FetchMany_wrap, and FetchAll_wrap for LOB streaming
tests/test_004_cursor.py	Added lob_wvarchar_column to test schema; updated test_cursor_description to expect SQL type codes; added comprehensive geography type test suite (14 new tests); separated LOB and non-LOB fetch tests; fixed output converter test for UTF-16LE encoding
tests/test_003_connection.py	Simplified converter integration tests to use SQL type constants directly instead of dynamic type detection

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 93%

🎯 Overall Coverage 77%

📈 Total Lines Covered: 5556 out of 7213 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

• mssql_python/init.py (100%)
• mssql_python/connection.py (88.2%): Missing lines 50,1020
• mssql_python/constants.py (100%)
• mssql_python/cursor.py (90.6%): Missing lines 134,158,1132,1479,2547
• mssql_python/pybind/ddbc_bindings.cpp (100%)

Summary

• Total: 100 lines
• Missing: 7 lines
• Coverage: 93%

mssql_python/connection.py

Lines 46-54

---

Lines 1016-1024

---

mssql_python/cursor.py

Lines 130-138

---

Lines 154-162

---

Lines 1128-1136

---

Lines 1475-1483

---

Lines 2543-2551

---

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.hpp: 58.8%
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.cpp: 69.5%
mssql_python.pybind.ddbc_bindings.h: 69.7%
mssql_python.pybind.connection.connection.cpp: 75.3%
mssql_python.row.py: 79.5%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.__init__.py: 84.9%
mssql_python.cursor.py: 85.2%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

tests/test_004_cursor.py:1

• This change corrects the exception type from a generic Exception to mssql_python.Error, which is more specific and accurate for database errors. This is already fixed in the updated code.

"""

tests/test_004_cursor.py:13063

• Simplified cleanup by removing unnecessary try-except block around DROP TABLE statement. This is already fixed in the updated code.

        cursor.execute(f"DROP TABLE IF EXISTS {table_name}")

tests/test_004_cursor.py:1

• Added proper encoding handling for string comparison in output converter test. The driver passes string values as UTF-16LE encoded bytes to output converters, so the test now correctly encodes the comparison value. This is already fixed in the updated code.

"""

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[saurabh500]

SqlTypeCode should be used instead of SQLTypeCode. I am assuming this is a public type and the naming would matter more. PascalCase for Python classes is recommended and applicable to acronyms also.

[dlevy-msft-sql]

SqlTypeCode should be used instead of SQLTypeCode. I am assuming this is a public type and the naming would matter more. PascalCase for Python classes is recommended and applicable to acronyms also.

Fixed.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.