Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support forecast tasks in profile API; enable index field modifications #1316

Merged
merged 1 commit into from
Sep 18, 2024

Conversation

kaituo
Copy link
Collaborator

@kaituo kaituo commented Sep 18, 2024

Description

This PR adds support for forecast task targets in profile API and enables modification of categorical and custom result index fields

  • Support forecast task targets in profile API: Previously, the profile API returned empty results for forecast task targets. This update adds support for them. For details, refer to ForecastTaskProfileRunner.
  • Allow modification of categorical and custom result index fields in forecasting: These fields are not editable in Anomaly Detection (AD). Implemented a lastUiBreakingChangeTime field in the config index to manage changes. Whenever the categorical or custom result index fields are updated, lastUiBreakingChangeTime is refreshed. The UI will not display results before this time to prevent inconsistencies. For details, refer to Config and AbstractTimeSeriesActionHandler.

Testing Done:

  • Added ITs for both scenarios.

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@opensearch-trigger-bot opensearch-trigger-bot bot added infra Changes to infrastructure, testing, CI/CD, pipelines, etc. backport 2.x labels Sep 18, 2024
@kaituo kaituo added backport 2.17 enhancement New feature or request and removed infra Changes to infrastructure, testing, CI/CD, pipelines, etc. labels Sep 18, 2024
Copy link

codecov bot commented Sep 18, 2024

Codecov Report

Attention: Patch coverage is 82.85714% with 6 lines in your changes missing coverage. Please review.

Project coverage is 79.43%. Comparing base (3f0fc8c) to head (7feb12f).
Report is 16 commits behind head on main.

Files with missing lines Patch % Lines
.../rest/handler/AbstractTimeSeriesActionHandler.java 25.00% 1 Missing and 2 partials ⚠️
.../handler/AbstractAnomalyDetectorActionHandler.java 50.00% 0 Missing and 1 partial ⚠️
...opensearch/forecast/ForecastTaskProfileRunner.java 66.66% 0 Missing and 1 partial ⚠️
.../rest/handler/AbstractForecasterActionHandler.java 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##               main    #1316      +/-   ##
============================================
+ Coverage     71.83%   79.43%   +7.60%     
- Complexity     4898     5605     +707     
============================================
  Files           518      533      +15     
  Lines         22879    23363     +484     
  Branches       2245     2319      +74     
============================================
+ Hits          16434    18559    +2125     
+ Misses         5410     3693    -1717     
- Partials       1035     1111      +76     
Flag Coverage Δ
plugin 79.43% <82.85%> (+7.60%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/main/java/org/opensearch/ad/model/ADTask.java 98.00% <100.00%> (+0.01%) ⬆️
.../java/org/opensearch/ad/model/AnomalyDetector.java 86.33% <100.00%> (-0.09%) ⬇️
...search/ad/rest/RestIndexAnomalyDetectorAction.java 100.00% <100.00%> (+74.28%) ⬆️
...va/org/opensearch/forecast/model/ForecastTask.java 98.00% <100.00%> (+0.41%) ⬆️
...java/org/opensearch/forecast/model/Forecaster.java 78.48% <100.00%> (+7.40%) ⬆️
...earch/forecast/rest/RestIndexForecasterAction.java 79.48% <100.00%> (+60.56%) ⬆️
...n/java/org/opensearch/timeseries/model/Config.java 87.01% <100.00%> (-0.39%) ⬇️
...arch/timeseries/transport/BooleanNodeResponse.java 100.00% <100.00%> (+100.00%) ⬆️
...ensearch/timeseries/transport/BooleanResponse.java 100.00% <100.00%> (+100.00%) ⬆️
.../handler/AbstractAnomalyDetectorActionHandler.java 97.87% <50.00%> (+77.87%) ⬆️
... and 3 more

... and 137 files with indirect coverage changes

This PR adds support for forecast task targets in profile API and enables modification of categorical and custom result index fields
* Support forecast task targets in profile API: Previously, the profile API returned empty results for forecast task targets. This update adds support for them. For details, refer to ForecastTask.
* Allow modification of categorical and custom result index fields in forecasting: These fields are not editable in Anomaly Detection (AD). Implemented a lastUiBreakingChangeTime field in the config index to manage changes. Whenever the categorical or custom result index fields are updated, lastUiBreakingChangeTime is refreshed. The UI will not display results before this time to prevent inconsistencies. For details, refer to Config and AbstractTimeSeriesActionHandler.

Testing Done:
* Added ITs for both scenarios.

Signed-off-by: Kaituo Li <[email protected]>
@kaituo kaituo merged commit 062db14 into opensearch-project:main Sep 18, 2024
19 of 20 checks passed
@opensearch-trigger-bot
Copy link

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/anomaly-detection/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/anomaly-detection/backport-2.x
# Create a new branch
git switch --create backport/backport-1316-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 062db1467cd59a4cf46daa03e9c0a2f2ee200607
# Push it to GitHub
git push --set-upstream origin backport/backport-1316-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/anomaly-detection/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-1316-to-2.x.

@opensearch-trigger-bot
Copy link

The backport to 2.17 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/anomaly-detection/backport-2.17 2.17
# Navigate to the new working tree
pushd ../.worktrees/anomaly-detection/backport-2.17
# Create a new branch
git switch --create backport/backport-1316-to-2.17
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 062db1467cd59a4cf46daa03e9c0a2f2ee200607
# Push it to GitHub
git push --set-upstream origin backport/backport-1316-to-2.17
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/anomaly-detection/backport-2.17

Then, create a pull request where the base branch is 2.17 and the compare/head branch is backport/backport-1316-to-2.17.

@yuye-aws
Copy link
Member

Are we backporting this PR to 2.17 and 2.x?

@kaituo
Copy link
Collaborator Author

kaituo commented Sep 26, 2024

@yuye-aws backporting to 2.x, won't do it for 2.17: #1319

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants