Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch index settings asynchronously #135

Merged
merged 1 commit into from
May 5, 2023

Conversation

msfroh
Copy link
Collaborator

@msfroh msfroh commented May 2, 2023

In some cases, like using ./gradlew run, I've run into issues where OpenSearch crashes, complaining that we're making a blocking call on a listener thread, because we fetch index settings to see if a result transformer has been configured for the current index.

I'm kind of surprised that we haven't run into this in production use-cases, but it may just be because assertions are not enabled in production. Regardless, blocking calls on listener threads are a bad idea, since that's how you run out of listener threads.

This change makes the index settings call asynchronous and chains the remaining logic off of that.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

In some cases, like using ./gradlew run, I've run into issues where
OpenSearch crashes, complaining that we're making a blocking call on a
listener thread, because we fetch index settings to see if a result
transformer has been configured for the current index.

I'm kind of surprised that we haven't run into this in production
use-cases, but it may just be because assertions are not enabled in
production. Regardless, blocking calls on listener threads are a bad
idea, since that's how you run out of listener threads.

This change makes the index settings call asynchronous and chains the
remaining logic off of that.

Signed-off-by: Michael Froh <[email protected]>
Copy link
Member

@sejli sejli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I did notice that some of the formatting was changed on some files, particularly ConfigurationUtils.java. This repo should probably include the Spotless Gradle plugin for a formatting check.

@msfroh msfroh merged commit fdadd8c into opensearch-project:main May 5, 2023
@msfroh msfroh deleted the async_settings branch May 6, 2023 00:22
@mingshl mingshl added backport 2.x Backport to 2.x branch enhancement change or upgrade that increases software capabilities beyond original client specifications labels Jul 12, 2023
@opensearch-trigger-bot
Copy link

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-135-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 fdadd8cb8c70d7732650b52614756d8fa2569a97
# Push it to GitHub
git push --set-upstream origin backport/backport-135-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-135-to-2.x.

mingshl pushed a commit to mingshl/search-processor that referenced this pull request Jul 12, 2023
In some cases, like using ./gradlew run, I've run into issues where
OpenSearch crashes, complaining that we're making a blocking call on a
listener thread, because we fetch index settings to see if a result
transformer has been configured for the current index.

I'm kind of surprised that we haven't run into this in production
use-cases, but it may just be because assertions are not enabled in
production. Regardless, blocking calls on listener threads are a bad
idea, since that's how you run out of listener threads.

This change makes the index settings call asynchronous and chains the
remaining logic off of that.

Signed-off-by: Michael Froh <[email protected]>
mingshl added a commit that referenced this pull request Jul 12, 2023
In some cases, like using ./gradlew run, I've run into issues where
OpenSearch crashes, complaining that we're making a blocking call on a
listener thread, because we fetch index settings to see if a result
transformer has been configured for the current index.

I'm kind of surprised that we haven't run into this in production
use-cases, but it may just be because assertions are not enabled in
production. Regardless, blocking calls on listener threads are a bad
idea, since that's how you run out of listener threads.

This change makes the index settings call asynchronous and chains the
remaining logic off of that.

Signed-off-by: Michael Froh <[email protected]>
Co-authored-by: Michael Froh <[email protected]>
mingshl pushed a commit to mingshl/search-processor that referenced this pull request Jul 12, 2023
In some cases, like using ./gradlew run, I've run into issues where
OpenSearch crashes, complaining that we're making a blocking call on a
listener thread, because we fetch index settings to see if a result
transformer has been configured for the current index.

I'm kind of surprised that we haven't run into this in production
use-cases, but it may just be because assertions are not enabled in
production. Regardless, blocking calls on listener threads are a bad
idea, since that's how you run out of listener threads.

This change makes the index settings call asynchronous and chains the
remaining logic off of that.

Signed-off-by: Michael Froh <[email protected]>
mingshl added a commit that referenced this pull request Jul 12, 2023
In some cases, like using ./gradlew run, I've run into issues where
OpenSearch crashes, complaining that we're making a blocking call on a
listener thread, because we fetch index settings to see if a result
transformer has been configured for the current index.

I'm kind of surprised that we haven't run into this in production
use-cases, but it may just be because assertions are not enabled in
production. Regardless, blocking calls on listener threads are a bad
idea, since that's how you run out of listener threads.

This change makes the index settings call asynchronous and chains the
remaining logic off of that.

Signed-off-by: Michael Froh <[email protected]>
Co-authored-by: Michael Froh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch enhancement change or upgrade that increases software capabilities beyond original client specifications
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants