Fix SET NOT NULL with deleted NULL rows

[dentiny]

Closes #1158

The bug is, when we tries to set NOT NULL and check feasibility, we directly check from stats whether we have null rows.

ducklake/src/storage/ducklake_table_entry.cpp

Line 609 in d897bc5

if (col_stats.has_null_count && col_stats.null_count > 0) {

In the current implementation, we only populate stats when new data inserted -- we didn't updates stats when rows deleted. so the stats doesn't really reflect the latest actual status.

In this PR, I did a few things:

• Check transaction-local change
• Check stats, if indicates no NULL we could trust it
• Otherwise, the stats might not be latest enough, so we fallback to scan table

[dentiny]

ccache C:\PROGRA~1\MICROS~2\18\ENTERP~1\VC\Tools\MSVC\1451~1.362\bin\Hostx64\x64\cl.exe  /nologo /TP -DDUCKDB_BUILD_LIBRARY -DDUCKDB_CUSTOM_PLATFORM=windows_amd64 -DDUCKDB_EXTENSION_AUTOINSTALL_DEFAULT=1 -DDUCKDB_EXTENSION_AUTOLOAD_DEFAULT=1 -D_SILENCE_ALL_MS_EXT_DEPRECATION_WARNINGS=1 -ID:\a\ducklake\ducklake\duckdb\src\include -ID:\a\ducklake\ducklake\duckdb\third_party\fsst -ID:\a\ducklake\ducklake\duckdb\third_party\fmt\include -ID:\a\ducklake\ducklake\duckdb\third_party\hyperloglog -ID:\a\ducklake\ducklake\duckdb\third_party\fastpforlib -ID:\a\ducklake\ducklake\duckdb\third_party\skiplist -ID:\a\ducklake\ducklake\duckdb\third_party\ska_sort -ID:\a\ducklake\ducklake\duckdb\third_party\fast_float -ID:\a\ducklake\ducklake\duckdb\third_party\re2 -ID:\a\ducklake\ducklake\duckdb\third_party\miniz -ID:\a\ducklake\ducklake\duckdb\third_party\utf8proc\include -ID:\a\ducklake\ducklake\duckdb\third_party\concurrentqueue -ID:\a\ducklake\ducklake\duckdb\third_party\pcg -ID:\a\ducklake\ducklake\duckdb\third_party\pdqsort -ID:\a\ducklake\ducklake\duckdb\third_party\tdigest -ID:\a\ducklake\ducklake\duckdb\third_party\mbedtls\include -ID:\a\ducklake\ducklake\duckdb\third_party\httplib -ID:\a\ducklake\ducklake\duckdb\third_party\jaro_winkler -ID:\a\ducklake\ducklake\duckdb\third_party\vergesort -ID:\a\ducklake\ducklake\duckdb\third_party\yyjson\include -ID:\a\ducklake\ducklake\duckdb\third_party\zstd\include /DWIN32 /D_WINDOWS /EHsc /wd4244 /wd4267 /wd4200 /wd26451 /wd26495 /D_CRT_SECURE_NO_WARNINGS /utf-8 /O2 /Ob2 /DNDEBUG -MT /W0 /showIncludes /Fothird_party\fmt\CMakeFiles\duckdb_fmt.dir\format.cc.obj /Fdthird_party\fmt\CMakeFiles\duckdb_fmt.dir\duckdb_fmt.pdb /FS -c D:\a\ducklake\ducklake\duckdb\third_party\fmt\format.cc
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(326): error C2653: 'stdext': is not a class or namespace name
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(326): error C2061: syntax error: identifier 'checked_array_iterator'
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(327): error C2988: unrecognizable template declaration/definition
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(327): error C2143: syntax error: missing ';' before '<'
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(327): error C2059: syntax error: '<'
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(327): error C2143: syntax error: missing ';' before '{'
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(327): error C2447: '{': missing function header (old-style formal list?)
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(336): error C2988: unrecognizable template declaration/definition
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(336): error C2143: syntax error: missing ';' before '<'
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(336): error C7525: inline variables require at least '/std:c++17'
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(336): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(336): error C2059: syntax error: '<'
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(336): error C2653: 'Container': is not a class or namespace name
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(337): error C2065: 'Container': undeclared identifier
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(337): error C2923: 'std::back_insert_iterator': 'Container' is not a valid template type argument for parameter '_Container'
D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(337): note: see declaration of 'Container'
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(337): error C2143: syntax error: missing ';' before '{'
Error: D:\a\ducklake\ducklake\duckdb\third_party\fmt\include\fmt/format.h(337): error C2447: '{': missing function header (old-style formal list?)
[12/624] Building CXX object third_party\libpg_query\CMakeFiles\duckdb_pg_query.dir\src_backend_nodes_value.cpp.obj
[13/624] Building CXX object third_party\libpg_query\CMakeFiles\duckdb_pg_query.dir\src_common_keywords.cpp.obj
[14/624] Building CXX object third_party\libpg_query\CMakeFiles\duckdb_pg_query.dir\src_backend_parser_scan.cpp.obj
[15/624] Building CXX object third_party\mbedtls\CMakeFiles\duckdb_mbedtls.dir\mbedtls_wrapper.cpp.obj
[16/624] Building CXX object third_party\libpg_query\CMakeFiles\duckdb_pg_query.dir\src_backend_parser_gram.cpp.obj
ninja: build stopped: subcommand failed.

The error message doesn't seem related to my PR.

[guillesd]

This one is ready!

[pdet]

Thanks, LGTM