Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix tasks namespace #520

Merged
merged 5 commits into from
Sep 10, 2024
Merged

Conversation

Xtansia
Copy link
Collaborator

@Xtansia Xtansia commented Aug 21, 2024

Description

Describe what this change achieves.

Issues Resolved

List any issues this PR will resolve, e.g. Closes [...].

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

github-actions bot commented Aug 21, 2024

Changes Analysis

Commit SHA: dec5cc6
Comparing To SHA: a768a73

API Changes

Summary

├─┬Paths
│ ├─┬/_reindex
│ │ └─┬POST
│ │   └─┬Responses
│ │     └─┬200
│ │       └─┬application/json
│ │         └─┬Schema
│ │           ├──[➖] type (27034:19)❌ 
│ │           ├──[➖] properties (27036:15)❌ 
│ │           ├──[➖] properties (27038:15)❌ 
│ │           ├──[➖] properties (27040:15)❌ 
│ │           ├──[➖] properties (27042:15)❌ 
│ │           ├──[➖] properties (27046:15)❌ 
│ │           ├──[➖] properties (27050:15)❌ 
│ │           ├──[➖] properties (27048:15)❌ 
│ │           ├──[➖] properties (27052:15)❌ 
│ │           ├──[➖] properties (27054:15)❌ 
│ │           ├──[➖] properties (27056:15)❌ 
│ │           ├──[➖] properties (27058:15)❌ 
│ │           ├──[➖] properties (27060:15)❌ 
│ │           ├──[➖] properties (27062:15)❌ 
│ │           ├──[➖] properties (27064:15)❌ 
│ │           ├──[➖] properties (27066:15)❌ 
│ │           ├──[➖] properties (27068:15)❌ 
│ │           ├──[➕] oneOf (27006:17)
│ │           └──[➕] oneOf (28114:7)
│ ├─┬/{index}/_update_by_query
│ │ └─┬POST
│ │   └─┬Responses
│ │     └─┬200
│ │       └─┬application/json
│ │         └─┬Schema
│ │           ├──[➖] type (28123:19)❌ 
│ │           ├──[➖] properties (28125:15)❌ 
│ │           ├──[➖] properties (28133:15)❌ 
│ │           ├──[➖] properties (28127:15)❌ 
│ │           ├──[➖] properties (28131:15)❌ 
│ │           ├──[➖] properties (28135:15)❌ 
│ │           ├──[➖] properties (28137:15)❌ 
│ │           ├──[➖] properties (28139:15)❌ 
│ │           ├──[➖] properties (28151:15)❌ 
│ │           ├──[➖] properties (28153:15)❌ 
│ │           ├──[➖] properties (28155:15)❌ 
│ │           ├──[➖] properties (28157:15)❌ 
│ │           ├──[➖] properties (28141:15)❌ 
│ │           ├──[➖] properties (28143:15)❌ 
│ │           ├──[➖] properties (28145:15)❌ 
│ │           ├──[➖] properties (28147:15)❌ 
│ │           ├──[➖] properties (28149:15)❌ 
│ │           ├──[➕] oneOf (28065:17)
│ │           └──[➕] oneOf (28114:7)
│ ├─┬/_tasks/{task_id}
│ │ └─┬GET
│ │   └─┬Responses
│ │     └─┬200
│ │       └─┬application/json
│ │         └─┬Schema
│ │           └─┬response
│ │             └──[🔀] $ref (52732:9)❌ 
│ └─┬/{index}/_delete_by_query
│   └─┬POST
│     └─┬Responses
│       └─┬200
│         └─┬application/json
│           └─┬Schema
│             ├──[➖] type (25837:19)❌ 
│             ├──[➖] properties (25839:15)❌ 
│             ├──[➖] properties (25841:15)❌ 
│             ├──[➖] properties (25843:15)❌ 
│             ├──[➖] properties (25847:15)❌ 
│             ├──[➖] properties (25849:15)❌ 
│             ├──[➖] properties (25851:15)❌ 
│             ├──[➖] properties (25853:15)❌ 
│             ├──[➖] properties (25855:15)❌ 
│             ├──[➖] properties (25857:15)❌ 
│             ├──[➖] properties (25859:15)❌ 
│             ├──[➖] properties (25861:15)❌ 
│             ├──[➖] properties (25863:15)❌ 
│             ├──[➖] properties (25865:15)❌ 
│             ├──[➖] properties (25867:15)❌ 
│             ├──[➖] properties (25869:15)❌ 
│             ├──[➖] properties (25871:15)❌ 
│             ├──[➕] oneOf (25839:17)
│             └──[➕] oneOf (28114:7)
└─┬Components
  ├──[➖] schemas (28204:7)❌ 
  ├──[➖] schemas (52595:7)❌ 
  ├──[➖] schemas (52620:7)❌ 
  ├──[➕] schemas (28132:7)
  ├──[➕] schemas (28196:7)
  ├──[➕] schemas (52731:7)
  ├──[➕] schemas (52624:7)
  ├──[➕] schemas (52653:7)
  ├──[➕] schemas (52642:7)
  ├──[➕] schemas (52635:7)
  ├──[➕] schemas (28110:7)
  ├──[➕] schemas (28202:7)
  ├──[➕] schemas (28114:7)
  ├──[➕] schemas (29611:7)
  ├──[➕] schemas (52628:7)
  ├──[➕] schemas (52617:7)
  ├─┬tasks._common:TaskInfos
  │ ├─┬ONEOF
  │ │ ├──[➕] title (52708:18)
  │ │ └─┬Schema
  │ │   └──[🔀] $ref (52654:9)❌ 
  │ └─┬ONEOF
  │   └──[➕] title (52704:18)
  ├─┬_common:BaseNode
  │ ├──[➖] required (28199:11)❌ 
  │ ├──[➖] required (28200:11)❌ 
  │ ├──[➖] required (28202:11)❌ 
  │ └──[➖] required (28198:11)❌ 
  ├─┬_common:NodeRole
  │ ├──[➖] enum (29030:11)❌ 
  │ ├──[➖] enum (29017:11)❌ 
  │ ├──[➖] enum (29028:11)❌ 
  │ ├──[➖] enum (29029:11)❌ 
  │ ├──[➖] enum (29026:11)❌ 
  │ ├──[➖] enum (29016:11)❌ 
  │ ├──[➖] enum (29020:11)❌ 
  │ ├──[➖] enum (29025:11)❌ 
  │ ├──[➖] enum (29024:11)❌ 
  │ ├──[➖] enum (29018:11)❌ 
  │ ├──[➖] enum (29019:11)❌ 
  │ ├──[➖] enum (29021:11)❌ 
  │ ├──[➖] enum (29022:11)❌ 
  │ ├──[➖] enum (29023:11)❌ 
  │ ├──[➖] enum (29027:11)❌ 
  │ ├──[➖] type (29014:13)❌ 
  │ ├──[➕] oneOf (29025:11)
  │ ├──[➕] oneOf (29031:11)
  │ └──[➕] oneOf (29010:11)
  ├─┬tasks._common:TaskInfo
  │ └─┬status
  │   └──[🔀] $ref (52635:20)❌ 
  └─┬tasks._common:TaskListResponseBase
    └─┬nodes
      └─┬Schema
        └──[🔀] $ref (52643:9)❌ 

Document Element Total Changes Breaking Changes
paths 58 52
components 44 26
  • BREAKING Changes: 78 out of 102
  • Modifications: 4
  • Removals: 74
  • Additions: 24
  • Breaking Removals: 74
  • Breaking Modifications: 4

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/10782874511/artifacts/1911697885

API Coverage

Before After Δ
Covered (%) 537 (52.6 %) 537 (52.6 %) 0 (0 %)
Uncovered (%) 484 (47.4 %) 484 (47.4 %) 0 (0 %)
Unknown 26 26 0

@Xtansia Xtansia force-pushed the fix/tasks-namespace branch 3 times, most recently from 2d4ac35 to 608065b Compare September 9, 2024 05:15
@Xtansia Xtansia marked this pull request as ready for review September 9, 2024 05:15
@@ -1927,6 +1908,8 @@ components:
- remote_cluster_client
- transform
- voting_only
x-deprecated-enums:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with x-deprected-enums is that it doesn't express since when. Would this be possible?

- value: master
   x-deprecated-since: '2.0'

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unfortunate that enum: only accepts a string list, so we can't properly dedupe the values and associate info directly. We could do something like:

    NodeRole:
      type: string
      enum:
        - client
        - cluster_manager
        - coordinating_only
        - data
        - data_cold
        - data_content
        - data_frozen
        - data_hot
        - data_warm
        - ingest
        - master
        - ml
        - remote_cluster_client
        - transform
        - voting_only
      x-deprecated-enums:
        master:
          message: 'Please use inclusive language.'
          since: '2.0'

Or if we want to generalize adding any extensions against specific values:

    NodeRole:
      type: string
      enum:
        - client
        - cluster_manager
        - coordinating_only
        - data
        - data_cold
        - data_content
        - data_frozen
        - data_hot
        - data_warm
        - ingest
        - master
        - ml
        - remote_cluster_client
        - transform
        - voting_only
      x-enum:
        cluster_manager:
          x-version-added: '2.0'
        master:
          x-deprecation-message: 'Please use inclusive language.'
          x-version-deprecated: '2.0'

Obviously would ideally need a validator/linter for it to ensure values match up etc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we transform these parts of the spec in the merger and could have special treatment for enums?

Otherwise I think x-enum is a better and more extensible idea.

Copy link
Collaborator

@nhtruong nhtruong Sep 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The x-vendor-extensions are only applicable to OpenAPI properties, not JSON Schemas. Further more, I don't think this level of granularity is necessary. We can add a description property for the schema that says The use of "master" as a node role has been deprecated. Use "cluster_manager" instead

Copy link
Collaborator

@nhtruong nhtruong Sep 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think that this repo needs branches for major versions, as it will get more and more complex to keep track of all the diffs between major versions in the same spec.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nhtruong Are you saying this PR is good as is? I'm ok with it, feel free to hit approve.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean instead of x-deprecated-enums, which is not a supported property of JSON-schema, we should replace it with a description to keep it simple as the extra resolution of info is not worth the complexity. If we start adding x-deprecated-enums, then there will be x-deprecated-properties and such. The server still supports these deprecated values, we just need a way to inform the reader that they are deprecated one way or the other, and I think having it in the description will suffice.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nhtruong The x- are allowed on all schema objects according to OAS 3.1 (because "OpenAPI properties" are just schemas): https://spec.openapis.org/oas/v3.1.0.html#fixed-fields-19

The main reasoning for this is that we annotate these values as @Deprecated in Java (or [Obsolete] in .NET), so just putting it in the description is unhelpful.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should do https://stackoverflow.com/a/78232576.

Signed-off-by: Thomas Farr <[email protected]>
Signed-off-by: Thomas Farr <[email protected]>
Copy link

github-actions bot commented Sep 9, 2024

Spec Test Coverage Analysis

Total Tested
563 272 (48.31 %)

@Xtansia Xtansia requested a review from dblock September 9, 2024 22:22
dblock
dblock previously approved these changes Sep 9, 2024
Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

try {
this.ajv.addSchema(schema, key);
} catch (e) {
throw new Error(`Failed to add schema ${key}: \`${JSON.stringify(schema)}\``, { cause: e })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could use a test.

@dblock
Copy link
Member

dblock commented Sep 9, 2024

@nhtruong you good with the enum solution?

Signed-off-by: Thomas Farr <[email protected]>
@nhtruong nhtruong merged commit a51361e into opensearch-project:main Sep 10, 2024
20 checks passed
@Xtansia Xtansia deleted the fix/tasks-namespace branch September 10, 2024 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants