Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] [Streaming Indexing] Fix intermittent "The bulk request must be terminated by a newline [\\n]" failures #16214

Open
reta opened this issue Oct 7, 2024 · 1 comment
Assignees
Labels
bug Something isn't working Indexing Indexing, Bulk Indexing and anything related to indexing v2.18.0 Issues and PRs related to version 2.18.0

Comments

@reta
Copy link
Collaborator

reta commented Oct 7, 2024

Describe the bug

While bulk ingesting more or less large documents (> 1Kb), the server may reply intermittent with:

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\\n]"}],"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\\n]"},"status":400})

The issue comes from the fact that Netty could emit chunks partially but have knobs to control this behaviour. This configuration is not exposed by Reactor Netty yet (see please reactor/reactor-netty#3452) but expected to come in the next minor release.

Related component

Indexing

To Reproduce

See please https://build.ci.opensearch.org/job/gradle-check/47073/testReport/org.opensearch.rest/ReactorNetty4StreamingIT/testStreamingRequestOneBatchBySize/

Expected behavior

The large documents should be bulk indexed normally

Additional Details

Plugins
transport-reactor-netty4

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • CI

Additional context
reactor/reactor-netty#3452

@reta reta added bug Something isn't working untriaged labels Oct 7, 2024
@reta reta self-assigned this Oct 7, 2024
@github-actions github-actions bot added the Indexing Indexing, Bulk Indexing and anything related to indexing label Oct 7, 2024
@reta reta added v2.18.0 Issues and PRs related to version 2.18.0 and removed untriaged labels Oct 7, 2024
@reta
Copy link
Collaborator Author

reta commented Oct 7, 2024

@thomas-long-f3 fyi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Indexing Indexing, Bulk Indexing and anything related to indexing v2.18.0 Issues and PRs related to version 2.18.0
Projects
None yet
Development

No branches or pull requests

1 participant