BugFix: Where clause with nested struct comparison incorrectly handles nulls within the struct #201
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As the title says.
fixes https://github.com/duckdblabs/duckdb-internal/issues/3063
Normally when a comparison takes place in a where clause, a validity mask is not passed for the resulting vector. Instead,
NULL
if one of the left or right vectors has a null value, then the condition is not "matched". However, this only works for non-nested column types (i.e int, string, uuid, etc.) where the validity mask is valid for the column value. For nested types, however, there can be a valid list with aNULL
value. These cases are missed during the normal null checking, so a "match" is produced. This caused logically inconsistent results in queries like the followingbroken behaviour
The fix is to pass a validity mask to the expression executor in the physical filter. This validity mask is only used for filters on nested types, so hopefully it should regress filter functions too much.