Allow null values in multi-dimensional arrays #33786
Conversation
Allow constructing multi-dimensional arrays with null values instead of panicking. Similarly to PostgreSQL, treat null elements as zero-dimensional arrays. Fixes MaterializeInc/database-issues#9757 Signed-off-by: Moritz Hoffmann <[email protected]>
| // `true` if any element null or has a dimension of 0. | ||
| let mut have_empty = false; | ||
| let mut dim = None; | ||
| for datum in datums { |
There was a problem hiding this comment.
If you update the dim variable in all cases you can handle the wrong cardinality error in one place and also don't need to keep around have_empty:
let mut dim = None;
for datum in datums {
let actual = match datum {
Datum::Null => 0,
Datum::Array(arr) => arr.len(),
d => panic!("unexpected datum {d}"),
};
if let Some(expected) = dim && actual != expected {
// All input arrays must have the same dimensionality.
return Err(InvalidArrayError::WrongCardinality { actual, expected }.into());
}
dim = Some(actual);
}
let dim = dim.unwrap_or(0);
// Per PostgreSQL, if all input arrays are zero dimensional, so is the output.
if dim == 0 {
return Ok(temp_storage.try_make_datum(|packer| packer.try_push_array(&[], &[]))?)
}| SELECT * FROM jsons WHERE random_id = CAST(payload->>'my_field' AS uuid[])[random_index] | ||
| ) | ||
|
|
||
| # Regression test for issue #9757 |
There was a problem hiding this comment.
| # Regression test for issue #9757 | |
| # Regression test for issue MaterializeInc/database-issues#9757 |
| query T | ||
| SELECT ARRAY[NULL::BIGINT[], ARRAY[]::BIGINT[], NULL::BIGINT[]]; | ||
| ---- | ||
| {} |
There was a problem hiding this comment.
I don't know if this was intentional but from reading the original code I realized that you also fixed an extra bug. Materialize on main accepts the following expression:
materialize=> select array[array[1, 2], array[3, 4, 5], array[6]];
array
---------------------
{{1,2},{3,4},{5,6}}
(1 row)
But it should be rejected. This is what pg does and I think your PR also catches now:
postgres=# select array[array[1, 2], array[3, 4, 5], array[6]];
ERROR: multidimensional arrays must have array expressions with matching dimensions
Can we add one more regression test that doesn't use null arrays?
There was a problem hiding this comment.
Nope, unintentional, and means that try_push_array might be wrong. I'll check tomorrow.
There was a problem hiding this comment.
Shouldn't have come past this check: https://github.com/MaterializeInc/materialize/blob/main/src/repr/src/row.rs#L2262-L2275
Allow constructing multi-dimensional arrays with null values instead of panicking. Similarly to PostgreSQL, treat null elements as zero-dimensional arrays.
Fixes MaterializeInc/database-issues#9757