Skip to content

ChunkTableSource doesn't handle projection pushdown #66

@wmTJc9IK0Q

Description

@wmTJc9IK0Q

When using a RowTableSource, unprojected columns are properly handled by SetRowValue:

duckdb-go/row.go

Lines 22 to 29 in 2ad3914

func SetRowValue[T any](row Row, colIdx int, val T) error {
projectedIdx := row.projection[colIdx]
if projectedIdx < 0 || projectedIdx >= len(row.chunk.columns) {
return nil
}
vec := row.chunk.columns[projectedIdx]
return setVectorVal(&vec, row.r, val)
}

However, DataChunk which is exposed to the FillChunk method on ChunkTableSource does not have any knowledge of which columns are projected. This results in errors like invalid column count: expected 1, got 1 when calling chunk.SetValue for too many columns.

Potential solutions:

  • Give DataChunk internal knowledge about projected column indices and ignore unprojected columns when SetValue is called. This would mimic the Row type.
    • Cons: this field would not be relevant in all use cases of DataChunk and could cause confusion
  • Wrap DataChunk with a table UDF specific type that handles this
    • Pros: Doesn't have the con from above
    • Cons: Breaking change to table UDFs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions