Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document new experimental ingestion streaming APIs #8123

Merged
merged 16 commits into from
Sep 13, 2024

Conversation

reta
Copy link
Contributor

@reta reta commented Aug 29, 2024

Description

Document new experimental ingestion streaming APIs

Issues Resolved

Closes #8111

Version

2.17.0

Frontend features

If you're submitting documentation for an OpenSearch Dashboards feature, add a video that shows how a user will interact with the UI step by step. A voiceover is optional.

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

@kolchfa-aws kolchfa-aws added v2.17.0 release-notes PR: Include this PR in the automated release notes experimental labels Aug 29, 2024
@kolchfa-aws
Copy link
Collaborator

Thank you, @reta! I made some edits and pushed into your PR. Please verify that everything is still technically accurate. Also, I changed the bulk request to curl so please verify that the request is formatted correctly. Will this bulk request work in Dev Tools of OS Dashboards? (not sure if it'll work with the 2 headers).

@kolchfa-aws
Copy link
Collaborator

Also, please provide an issue where the users can track the progress of the feature. Or we can just direct them to the OS forum.

Signed-off-by: Andriy Redko <[email protected]>
@reta
Copy link
Contributor Author

reta commented Sep 2, 2024

Thank you, @reta! I made some edits and pushed into your PR. Please verify that everything is still technically accurate.

Thanks a lot @kolchfa-aws , I also made the required changes, looks good to me! Thank you!

Copy link
Collaborator

@kolchfa-aws kolchfa-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @reta! I think we can still keep the json highlighting on the curl code block. Technically, it's not JSON, but I like the way the curly braces are matched and the field names are all the same color :) Applying "Done but waiting to merge" label to put this in the 2.17 editorial queue starting 9/11. This PR will be merged before the release.

@kolchfa-aws kolchfa-aws added 6 - Done but waiting to merge PR: The work is done and ready to merge 5 - Editorial review PR: Editorial review in progress labels Sep 3, 2024
Signed-off-by: Andriy Redko <[email protected]>
@reta
Copy link
Contributor Author

reta commented Sep 3, 2024

Thank you, @reta! I think we can still keep the json highlighting on the curl code block.

Got it, brought it back, thanks @kolchfa-aws !

Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reta @kolchfa-aws Please see my comments and changes and let me know if you have any questions. Thanks!

_api-reference/document-apis/bulk-streaming.md Outdated Show resolved Hide resolved
_api-reference/document-apis/bulk-streaming.md Outdated Show resolved Hide resolved
_api-reference/document-apis/bulk-streaming.md Outdated Show resolved Hide resolved
_api-reference/document-apis/bulk-streaming.md Outdated Show resolved Hide resolved
_api-reference/document-apis/bulk-streaming.md Outdated Show resolved Hide resolved
`wait_for_active_shards` | String | Specifies the number of active shards that must be available before OpenSearch processes the bulk request. Default is 1 (only the primary shard). Set to `all` or a positive integer. Values greater than 1 require replicas. For example, if you specify a value of 3, the index must have two replicas distributed across two additional nodes for the request to succeed.
`batch_interval` | Time | Specifies how long bulk operations should be accumulated into batch before sending over to data nodes.
`batch_size` | Time | Specifies how many bulk operations should be accumulated into batch before sending over to data nodes. Default `1`.
{% comment %}_source | List | asdf
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this and the following two rows be here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They should (by API specs)

_api-reference/document-apis/bulk-streaming.md Outdated Show resolved Hide resolved
_api-reference/document-apis/bulk-streaming.md Outdated Show resolved Hide resolved
_api-reference/document-apis/bulk-streaming.md Outdated Show resolved Hide resolved
_api-reference/document-apis/bulk-streaming.md Outdated Show resolved Hide resolved
@kolchfa-aws kolchfa-aws merged commit fd2e9fe into opensearch-project:main Sep 13, 2024
6 checks passed
noahstaveley pushed a commit to noahstaveley/documentation-website that referenced this pull request Sep 23, 2024
…t#8123)

* Document new experimental ingestion streaming APIs

Signed-off-by: Andriy Redko <[email protected]>

* Doc review

Signed-off-by: Fanit Kolchina <[email protected]>

* Small rewording

Signed-off-by: Fanit Kolchina <[email protected]>

* Address review comments

Signed-off-by: Andriy Redko <[email protected]>

* Address review comments

Signed-off-by: Andriy Redko <[email protected]>

* Address  review comments

Signed-off-by: Andriy Redko <[email protected]>

* Address  review comments

Signed-off-by: Andriy Redko <[email protected]>

* Address  review comments

Signed-off-by: Andriy Redko <[email protected]>

* Address  review comments

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Co-authored-by: Fanit Kolchina <[email protected]>
Signed-off-by: Noah Staveley <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Editorial review PR: Editorial review in progress 6 - Done but waiting to merge PR: The work is done and ready to merge experimental release-notes PR: Include this PR in the automated release notes v2.17.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DOC] Document new experimental ingestion streaming APIs
3 participants