-
Notifications
You must be signed in to change notification settings - Fork 472
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spec Insert #8472
base: main
Are you sure you want to change the base?
Spec Insert #8472
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
opensearch-openapi.yaml | ||
rspec_examples.txt |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
--require spec_helper |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
require: rubocop-rake | ||
AllCops: | ||
Include: | ||
- 'lib/**/*.rb' | ||
- 'Rakefile' | ||
NewCops: enable | ||
|
||
Metrics/CyclomaticComplexity: | ||
Enabled: false | ||
Metrics/MethodLength: | ||
Enabled: false | ||
Metrics/ParameterLists: | ||
Enabled: false | ||
Metrics/AbcSize: | ||
Enabled: false | ||
Metrics/PerceivedComplexity: | ||
Enabled: false | ||
|
||
Layout/EmptyLineAfterGuardClause: | ||
Enabled: false | ||
|
||
Style/MultilineBlockChain: | ||
Enabled: false | ||
Style/SingleLineMethods: | ||
Enabled: false |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
3.1.0 |
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,91 @@ | ||||||
# README: Spec Insert | ||||||
Check failure on line 1 in spec_insert/README.md GitHub Actions / style-job
|
||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
- [What is this?](#what-is-this) | ||||||
- [Installation](#Installation) | ||||||
- [How to use](#how-to-use) | ||||||
- [Insert Query Parameters](#insert-query-parameters) | ||||||
- [Insert Path Parameters](#insert-path-parameters) | ||||||
- [Insert Paths and HTTP Methods](#insert-paths-and-http-methods) | ||||||
- [Ignored files and folders](#ignored-files-and-folders) | ||||||
|
||||||
## What is this? | ||||||
Check failure on line 10 in spec_insert/README.md GitHub Actions / style-job
|
||||||
This program allows you to insert API components generated from the OpenSearch Specification into this repository's markdown files. It's still underdevelopment, and many features are not yet implemented. This document will be updated as the program evolves. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
## Installation | ||||||
1. Clone this repository. | ||||||
2. Change to the `spec_insert` directory. | ||||||
3. Install Ruby 3.1.0 or later. | ||||||
4. Install the required gems by running `bundle install`. | ||||||
|
||||||
## How to use | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
Edit your markdown file and insert the following snippet where you want the API components to be inserted: | ||||||
```markdown | ||||||
<!-- spec_insert_start | ||||||
api: <API_NAME> | ||||||
component: <COMPONENT_NAME> | ||||||
other_param: <OTHER_PARAM> | ||||||
--> | ||||||
|
||||||
This is where the API component will be inserted. | ||||||
Everything between the `spec_insert_start` and `spec_insert_end` tags will be overwritten. | ||||||
|
||||||
<!-- spec_insert_end --> | ||||||
``` | ||||||
|
||||||
Then run the following Rake commands to download the latest OpenSearch Specification and insert the API components into the markdown files: | ||||||
```shell | ||||||
rake download_spec | ||||||
rake insert_spec | ||||||
``` | ||||||
|
||||||
### Insert Query Parameters | ||||||
To insert query parameters table of the `cat.indices` API, use the following snippet: | ||||||
```markdown | ||||||
<!-- spec_insert_start | ||||||
api: cat.indices | ||||||
component: query_parameters | ||||||
--> | ||||||
<!-- spec_insert_end --> | ||||||
``` | ||||||
|
||||||
- This will insert the query parameters of the `cat.indices` API into the markdown file 3 default columns: `Parameter`, `Type`, and `Description`. There are 5 columns that can be inserted: `Parameter`, `Type`, `Description`, `Required`, and `Default`. When `Required`/`Default` is not chosen, the info will be written in the `Description` column. | ||||||
- This component accepts `include_global` (boolean, default to `false`) argument to include global query parameters in the table. | ||||||
- This component accepts `include_deprecated` (boolean, default to `true`) argument to include deprecated parameters in the table. | ||||||
- This component accepts `pretty` (boolean, default to `false`) argument to render the table in the pretty format instead of the compact format. | ||||||
|
||||||
```markdown | ||||||
<!-- spec_insert_start | ||||||
api: cat.indices | ||||||
component: query_parameters | ||||||
include_global: true | ||||||
include_deprecated: false | ||||||
pretty: true | ||||||
--> | ||||||
<!-- spec_insert_end --> | ||||||
``` | ||||||
|
||||||
### Insert Path Parameters | ||||||
|
||||||
To insert path parameters table of the `indices.create` API, use the following snippet: | ||||||
```markdown | ||||||
<!-- spec_insert_start | ||||||
api: indices.create | ||||||
component: path_parameters | ||||||
--> | ||||||
<!-- spec_insert_end --> | ||||||
``` | ||||||
|
||||||
This table behaves the same as the query parameters table except that it does not accept the `include_global` argument. | ||||||
|
||||||
### Insert Paths and HTTP Methods | ||||||
|
||||||
To insert paths and HTTP methods of the `search` API, use the following snippet: | ||||||
```markdown | ||||||
<!-- spec_insert_start | ||||||
api: search | ||||||
component: paths_and_http_methods | ||||||
--> | ||||||
<!-- spec_insert_end --> | ||||||
``` | ||||||
|
||||||
### Ignored files and folders | ||||||
The program will ignore all markdown files whose names are in ALL CAPS. On top of that, you can also add files and folders you want to the [ignored.txt](./ignored.txt) file. Each line in the file should be the name of a file or folder you want to ignore. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe we should name |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
# frozen_string_literal: true | ||
|
||
require 'rake' | ||
|
||
desc 'Download the OpenSearch API specification' | ||
task :download_spec do | ||
sh 'curl -L -X GET ' \ | ||
'https://github.com/opensearch-project/opensearch-api-specification' \ | ||
'/releases/download/main-latest/opensearch-openapi.yaml ' \ | ||
'-o opensearch-openapi.yaml' | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Download to a temp or Don't use openapi_url = "https://github.com/opensearch-project/opensearch-api-specification/releases/download/main-latest/opensearch-openapi.yaml"
openapi_path = "_site/opensearch-openapi.yaml"
File.write(openapi_path, URI.open(openapi_url).read) |
||
|
||
desc 'Insert the OpenSearch API specification info into the documentation' | ||
task :insert_spec do | ||
require_relative 'lib/spec_inserter' | ||
require_relative 'lib/doc_processor' | ||
|
||
SpecInserter.new( | ||
root_folder: '../', | ||
spec_file: './opensearch-openapi.yaml', | ||
ignored: './ignored.txt' | ||
).insert_spec | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. rename the method to have a |
||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
# SPDX-License-Identifier: Apache-2.0 | ||
# | ||
# The OpenSearch Contributors require contributions made to | ||
# this file be licensed under the Apache-2.0 license or a | ||
# compatible open source license. | ||
|
||
# frozen_string_literal: true | ||
|
||
source 'https://rubygems.org' | ||
|
||
gem 'rake', '~> 13' | ||
gem 'activesupport', '~> 7' | ||
gem 'mustache', '~> 1' | ||
|
||
group :development, :test do | ||
gem 'rspec' | ||
gem 'rubocop', '~> 1.44', require: false | ||
gem 'rubocop-rake', require: false | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# All files and folders listed below are ignored by the spec_insert program. | ||
# MD files whose names are in ALL CAPS are always ignored. | ||
|
||
spec_insert/ | ||
release-notes/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
# SPDX-License-Identifier: Apache-2.0 | ||
# | ||
# The OpenSearch Contributors require contributions made to | ||
# this file be licensed under the Apache-2.0 license or a | ||
# compatible open source license. | ||
|
||
# frozen_string_literal: true | ||
|
||
require_relative 'parameter' | ||
require_relative 'operation' | ||
|
||
# A collection of operations that comprise a single API Action | ||
# AKA operation-group | ||
class Action | ||
# @param [SpecHash] spec Parsed OpenAPI spec | ||
def self.actions=(spec) | ||
operations = spec.paths.flat_map do |url, ops| | ||
ops.filter_map { |verb, op| Operation.new(op, url, verb) unless op['x-ignorable'] } | ||
end | ||
@actions = operations.group_by(&:group).values.map { |ops| Action.new(ops) }.index_by(&:full_name) | ||
end | ||
|
||
# @return [Hash<String, Action>] API Actions indexed by operation-group | ||
def self.actions | ||
raise 'Actions not set' unless @actions | ||
@actions | ||
end | ||
|
||
# @return [Array<Operation>] Operations in the action | ||
attr_reader :operations | ||
|
||
# @param [Array<Operation>] operations | ||
def initialize(operations) | ||
@operations = operations | ||
@operation = operations.first | ||
@spec = @operation&.spec | ||
end | ||
|
||
# @return [Array<Parameter>] Input arguments. | ||
def arguments; @arguments ||= Parameter.from_operations(@operations.map(&:spec)); end | ||
|
||
# @return [String] Full name of the action (i.e. namespace.action) | ||
def full_name; @operation&.group; end | ||
|
||
# return [String] Name of the action | ||
def name; @operation&.action; end | ||
|
||
# @return [String] Namespace of the action | ||
def namespace; @operation&.namespace; end | ||
|
||
# @return [Array<String>] Sorted unique HTTP verbs | ||
def http_verbs; @operations.map(&:http_verb).uniq.sort; end | ||
|
||
# @return [Array<String>] Unique URLs | ||
def urls; @operations.map(&:url).uniq; end | ||
|
||
# @return [String] Description of the action | ||
def description; @spec&.description; end | ||
|
||
# @return [Boolean] Whether the action is deprecated | ||
def deprecated; @spec&.deprecated; end | ||
|
||
# @return [String] Deprecation message | ||
def deprecation_message; @spec['x-deprecation-message']; end | ||
|
||
# @return [String] API reference | ||
def api_reference; @operation&.external_docs&.url; end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
# SPDX-License-Identifier: Apache-2.0 | ||
# | ||
# The OpenSearch Contributors require contributions made to | ||
# this file be licensed under the Apache-2.0 license or a | ||
# compatible open source license. | ||
|
||
# frozen_string_literal: true | ||
|
||
# An API Operation | ||
class Operation | ||
# @return [Openapi3Parser::Node::Operation] Operation Spec | ||
attr_reader :spec | ||
# @return [String] URL | ||
attr_reader :url | ||
# @return [String] HTTP Verb | ||
attr_reader :http_verb | ||
# @return [String] Operation Group | ||
attr_reader :group | ||
# @return [String] API Action | ||
attr_reader :action | ||
# @return [String] API Namespace | ||
attr_reader :namespace | ||
|
||
# @param [Openapi3Parser::Node::Operation] spec Operation Spec | ||
# @param [String] url | ||
# @param [String] http_verb | ||
def initialize(spec, url, http_verb) | ||
@spec = spec | ||
@url = url | ||
@http_verb = http_verb.upcase | ||
@group = spec['x-operation-group'] | ||
@action, @namespace = @group.split('.').reverse | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,94 @@ | ||
# frozen_string_literal: true | ||
|
||
module ArgLocation | ||
PATH = :path | ||
QUERY = :query | ||
end | ||
|
||
# Represents a parameter of an API action | ||
class Parameter | ||
# @return [String] The name of the parameter | ||
attr_reader :name | ||
# @return [String] The description of the parameter | ||
attr_reader :description | ||
# @return [Boolean] Whether the parameter is required | ||
attr_reader :required | ||
# @return [SpecHash] The JSON schema of the parameter | ||
attr_reader :schema | ||
# @return [String] Argument type in documentation | ||
attr_reader :doc_type | ||
# @return [String] The default value of the parameter | ||
attr_reader :default | ||
# @return [Boolean] Whether the parameter is deprecated | ||
attr_reader :deprecated | ||
# @return [String] The deprecation message | ||
attr_reader :deprecation_message | ||
# @return [String] The OpenSearch version when the parameter was deprecated | ||
attr_reader :version_deprecated | ||
# @return [ArgLocation] The location of the parameter | ||
attr_reader :location | ||
|
||
def initialize(name:, description:, required:, schema:, default:, deprecated:, deprecation_message:, | ||
version_deprecated:, location:) | ||
@name = name | ||
@description = description | ||
@required = required | ||
@schema = schema | ||
@doc_type = get_doc_type(schema).gsub('String / List', 'List').gsub('List / String', 'List') | ||
@default = default | ||
@deprecated = deprecated | ||
@deprecation_message = deprecation_message | ||
@version_deprecated = version_deprecated | ||
@location = location | ||
end | ||
|
||
# @param [SpecHash | nil] schema | ||
# @return [String | nil] Documentation type | ||
def get_doc_type(schema) | ||
return nil if schema.nil? | ||
union = schema.anyOf || schema.oneOf | ||
return union.map { |sch| get_doc_type(sch) }.join(' / ') unless union.nil? | ||
return 'Integer' if schema.type == 'integer' | ||
return 'Float' if schema.type == 'number' | ||
return 'Boolean' if schema.type == 'boolean' | ||
return 'String' if schema.type == 'string' | ||
return 'NULL' if schema.type == 'null' | ||
return 'List' if schema.type == 'array' | ||
'Object' | ||
end | ||
|
||
# @param [SpecHash] Full OpenAPI spec | ||
def self.global=(spec) | ||
@global = spec.components.parameters.filter { |_, p| p['x-global'] }.map { |_, p| from_parameters([p], 1) } | ||
end | ||
|
||
# @return [Array<Parameter>] Global parameters | ||
def self.global | ||
raise 'Global parameters not set' unless @global | ||
@global | ||
end | ||
|
||
# @param [Array<SpecHash>] operations List of operations of the same group | ||
# @return [Array<Parameter>] List of parameters of the operation group | ||
def self.from_operations(operations) | ||
operations.flat_map(&:parameters).filter { |param| !param['x-global'] } | ||
.group_by(&:name).values.map { |params| from_parameters(params, operations.size) } | ||
end | ||
|
||
# @param [Array<SpecHash>] params List of parameters of the same name | ||
# @param [Integer] opts_count Number of operations involved | ||
# @return [Parameter] Single parameter distilled from the list | ||
def self.from_parameters(params, opts_count) | ||
param = params.first || SpecHash.new | ||
schema = param&.schema || SpecHash.new | ||
Parameter.new(name: param.name, | ||
description: param.description || schema.description, | ||
required: params.filter(&:required).size >= opts_count, | ||
schema: schema, | ||
default: param.default || schema.default, | ||
deprecated: param.deprecated || schema.deprecated, | ||
deprecation_message: param['x-deprecation-message'] || schema['x-deprecation-message'], | ||
version_deprecated: param['x-version-deprecated'] || schema['x-version-deprecated'], | ||
location: params.any? { |p| p.in == 'path' } ? ArgLocation::PATH : ArgLocation::QUERY) | ||
end | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should apply to specs to, use
rubocop -a ; rubocop --auto-gen-config
in general.