[API-spec] Add a custom configuration for admin-only APIs and add tests for those

[DarshitChanpura]

Description

At present the test suite added via opensearch-project/opensearch-api-specification#439 test for common scenario, which is 403 when calling admin-only API.

This issue requests creation of a separate docker config to test admin-only APIs as TLS and rest-admin.

Expand to see custom roles_mapping.yml

---
# In this file users, backendroles and hosts can be mapped to Security roles.
# Permissions for OpenSearch roles are configured in roles.yml

_meta:
  type: "rolesmapping"
  config_version: 2

# Define your roles mapping here

## Demo roles mapping

all_access:
  reserved: false
  backend_roles:
  - "admin"
  users:
  - "admin"
  description: "Maps admin to all_access"

own_index:
  reserved: false
  users:
  - "*"
  description: "Allow full access to an index named like the username"

logstash:
  reserved: false
  backend_roles:
  - "logstash"

kibana_user:
  reserved: false
  backend_roles:
  - "kibanauser"
  description: "Maps kibanauser to kibana_user"

readall:
  reserved: false
  backend_roles:
  - "readall"

manage_snapshots:
  reserved: false
  backend_roles:
  - "snapshotrestore"

kibana_server:
  reserved: true
  users:
  - "kibanaserver"

security_rest_api_access:
  reserved: true
  users:
  - "admin"

security_rest_api_full_access:
  reserved: true
  users:
  - "admin"

Expand to see custom opensearchyml

---
cluster.name: docker-cluster

# Bind to all interfaces because we don't know what IP address Docker will assign to us.
network.host: 0.0.0.0

# # minimum_master_nodes need to be explicitly set when bound on a public IP
# # set to 1 to allow single node clusters
# discovery.zen.minimum_master_nodes: 1

# Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again.
# discovery.type: single-node


######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn: ['CN=kirk,OU=client,O=client,L=test,C=de']
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [ 'all_access', 'security_rest_api_access', 'security_rest_api_full_access']
plugins.security.restapi.admin.enabled: true
plugins.security.unsupported.restapi.allow_securityconfig_modification: true
plugins.security.nodes_dn_dynamic_config_enabled: true
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
  .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
  .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
  .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
  .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*,
  .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
  .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
  .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models,
  .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
  .plugins-flow-framework-state]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########

Expand to see custom docker-compose.yml

services:
  opensearch-node1:
    image: opensearchstaging/opensearch:3.0.0
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - discovery.seed_hosts=opensearch-node1,opensearch-node2
      - cluster.initial_master_nodes=opensearch-node1,opensearch-node2
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    volumes:
      # - ./config/opensearch.yml:/usr/share/opensearch/config/opensearch.yml
      # - ./config/roles_mapping.yml:/usr/share/opensearch/config/opensearch-security/roles_mapping.yml
      # - ./config/esnode.pem:/usr/share/opensearch/config/esnode.pem
      # - ./config/esnode-key.pem:/usr/share/opensearch/config/esnode-key.pem
      # - ./config/kirk.pem:/usr/share/opensearch/config/kirk.pem
      # - ./config/kirk-key.pem:/usr/share/opensearch/config/kirk-key.pem
      # - ./config/root-ca.pem:/usr/share/opensearch/config/root-ca.pem
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net
  opensearch-node2: # This is the same settings as the opensearch-node1
    image: opensearchstaging/opensearch:3.0.0
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node2
      - discovery.seed_hosts=opensearch-node1,opensearch-node2
      - cluster.initial_master_nodes=opensearch-node1,opensearch-node2
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      # - ./config/opensearch.yml:/usr/share/opensearch/config/opensearch.yml
      # - ./config/roles_mapping.yml:/usr/share/opensearch/config/opensearch-security/roles_mapping.yml
      # - ./config/esnode.pem:/usr/share/opensearch/config/esnode.pem
      # - ./config/esnode-key.pem:/usr/share/opensearch/config/esnode-key.pem
      # - ./config/kirk.pem:/usr/share/opensearch/config/kirk.pem
      # - ./config/kirk-key.pem:/usr/share/opensearch/config/kirk-key.pem
      # - ./config/root-ca.pem:/usr/share/opensearch/config/root-ca.pem
    networks:
      - opensearch-net
  opensearch-dashboards:
    image: opensearchstaging/opensearch-dashboards:3.0.0
    container_name: opensearch-dashboards
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
    networks:
      - opensearch-net
    # volumes:
    #   - ./opensearch_dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
networks:
  opensearch-net:

These are sample configs that would enable normal admin to be a rest-admin and allow for testing.
The main drawback to this is maintaining a list of custom certificates in a repo outside security plugin.

Additional Items

Add schemas for Forbidden (400), MethodNotImplemented (501) and BadRequest (400) responses. At the time of writing this, the support was not yet added for these statuses.

Expand to see the sample schemas :

schema: null
# $ref: '../schemas/security._common.yaml#/components/schemas/MethodNotImplemented'

schema: null
# $ref: '../schemas/security._common.yaml#/components/schemas/Forbidden'

schema: null
# $ref: '../schemas/security._common.yaml#/components/schemas/BadRequest'



# BadRequest:
#   type: object
#   properties:
#     status:
#       type: string
#       enum:
#         - 400
#     message:
#       type: string
#       description: Message returned as part of BAD_REQUEST response.

# Forbidden:
#   type: object
#   properties:
#     status:
#       type: string
#     message:
#       type: string
#       description: Message returned as part of Forbidden response.

# MethodNotImplemented:
#   type: object
#   properties:
#     status:
#       type: string
#       enum:
#         - 501
#     message:
#       type: string
#       description: Message returned as part of NOT_IMPLEMENTED response.

[stephen-crawford]

[Triage] Hi @DarshitChanpura, thank you for filing this issue. This seems like a smart change. Going to mark as triaged.