Skip to content

Conversation

@rluvaton
Copy link
Member

@rluvaton rluvaton commented Oct 20, 2025

This PR include the following PRs changes please make sure to review them first

Which issue does this PR close?

N/A

Rationale for this change

This will make scalar to array in DataFusion faster

What changes are included in this PR?

Include the changes of:

Are these changes tested?

yes

Are there any user-facing changes?

yes

@github-actions github-actions bot added the arrow Changes to the arrow crate label Oct 20, 2025
@rluvaton rluvaton changed the title feat: add new_repeated and try_new_repeated to ByteArray feat: add new_repeated to ByteArray Oct 20, 2025
@rluvaton rluvaton marked this pull request as draft October 20, 2025 15:28
# Conflicts:
#	arrow-buffer/src/buffer/mutable.rs
@rluvaton rluvaton marked this pull request as ready for review October 21, 2025 18:26
let s: &[u8] = value.as_ref().as_ref();
let value_offsets = OffsetBuffer::from_repeated_length(s.len(), repeat_count);
let bytes: Buffer = {
let mut mutable_buffer = MutableBuffer::with_capacity(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 0, do we not know how many elements?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is the responsibility of the repeat

And 0 will use null pointer so it will no allocate anything unnecessary

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@alamb
Copy link
Contributor

alamb commented Nov 3, 2025

Let's get @mbrobbel 's comment addressed and we can merge this in

@rluvaton
Copy link
Member Author

rluvaton commented Nov 3, 2025

@alamb this can be merged

Copy link
Member

@mbrobbel mbrobbel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rluvaton, @vegarsti and @alamb

@mbrobbel mbrobbel merged commit b8a1926 into apache:main Nov 3, 2025
26 checks passed
@rluvaton rluvaton deleted the add-repeat-helpear-for-bytes-array branch November 3, 2025 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants