OIDC auth, custom configs (opensearch-project#63)

* Update dashboards.yml. Fix owner and permissions for home folder Signed-off-by: Sergey Shubin <[email protected]> * The validity period of certificates is set to a variable Signed-off-by: Sergey Shubin <[email protected]> * change HOME directory for {{ os_user }} and {{ os_dashboards_user }} and set them /bin/false shell Signed-off-by: Sergey Shubin <[email protected]> * auth_type (internal, openid). Custom configs, IaC 1. Added the ability to log in via OpenID 2. Added the ability to install custom configuration files for the cluster 3. Added the ability to reconfigure the cluster (in particular, update certificates) when expanding it 4. Added the ability not to change certificates if the cluster composition has not changed, but only the settings have changed. Signed-off-by: Sergey Shubin <[email protected]> * readme. description for OpenID, IaC, custom configuration files Signed-off-by: Sergey Shubin <[email protected]> * refactoring. see opensearch-project#63 Signed-off-by: Sergey Shubin <[email protected]>
peterzhuamazon · Jul 13, 2022 · a9170a4 · a9170a4
1 parent d45969a
commit a9170a4
Show file tree

Hide file tree

Showing 14 changed files with 712 additions and 23 deletions.
diff --git a/README.md b/README.md
@@ -33,6 +33,8 @@ This ansible playbook supports the following,
 - Configure TLS/SSL for OpenSearch transport layer(Nodes to Nodes communication) and REST API layer
 - Generate self-signed certificates to configure TLS/SSL for opensearch
 - Configure the Internal Users Database with limited users and user-defined passwords
+- Configuration of authentication and authorization via OpenID
+- Overriding default settings with your own
 - Install and configure the Apache2.0 opensource OpenSearch Dashboards
 
 ### Prerequisite
@@ -97,13 +99,53 @@ cluster_type: single-node
 
 You should set the reserved users(`admin` and `kibanaserver`) password using `admin_password` and `kibanaserver_password` variables.
 
+If you define your own internal users (in addition to the reserved `admin` and `kibanaserver`) in custom configuration 
+files, then passwords to them should be set via variables on the principle of `<username>_password`
+
 It will install and configure the opensearch. Once the deployment completed, you can access the opensearch Dashboards with user `admin` and password which you provided for variable `admin_password`.
 
     # Deploy with ansible playbook - run the playbook as non-root user which have sudo privileges,
     ansible-playbook -i inventories/opensearch/hosts opensearch.yml --extra-vars "admin_password=Test@123 kibanaserver_password=Test@6789" --become
 
 **Note**: Change the user details in `ansible_user` parameter in `inventories/opensearch/hosts`  inventory file.
 
+### OpenID authentification
+To enable authentication via OpenID, you need to change the `auth_type` variable in the inventory file 
+`inventories/opensearch/group_vars/all/all.yml` by setting the value `oidc` and prescribe the necessary settings 
+in the `oidc:` block.
+
+### Custom configuration files
+
+To override the default settings files, you need to put your settings in the `files` directory. The files should be 
+named exactly the same as the original ones (internal_users.yml, roles.yml, tenants.yml, etc.)
+
+Especially note the file `files/internal_users.yml`. If it exists and the `copy_custom_security_configs: true` setting is enabled, 
+then only in this case the task of setting passwords for internal users from variables is started. If the file `internal_users.yml` 
+is not located in the `files` directory, but, for example, in one of its subdirectories, then playbook will not work correctly
+
+### IaC (Infrastructure-as-Code)
+
+If you want to use the role not only for the initial deployment of the cluster, but also for further management of it, 
+then set the `iac_enable` parameter to `true`. 
+
+By default, if the /tmp/opensearch-nodecerts directory with certificates exists on the server from which the playbook 
+is launched, it is assumed that the configuration has not changed and some settings are not copied to the target servers.
+
+Conversely, if the /tmp/opensearch-nodecerts directory does not exist on the server from which the playbook is launched, 
+then new certificates and settings are generated and they are copied to the target servers.
+
+If you use this repository not only for the initial deployment of the cluster, but also for its automatic configuration 
+via CI/CD, then new certificates will be generated every time the pipeline is launched, overwriting existing ones, which 
+is not always necessary if the cluster is already in production.
+
+When iac_enable enabling, and all the cluster servers have all the necessary certificates, they will not be copied again. 
+If at least on one server (for example, when adding a new server to the cluster) if there is not at least one certificate 
+from the list, then all certificates on all cluster servers will be updated
+
+Also, if the option is enabled, the settings files will be updated with each execution (previously, the settings were 
+updated only if the /tmp/opensearch-nodecerts directory was missing on the server from which the playbook was launched 
+and new certificates were generated)
+
 ## Contributing
 
 See [developer guide](DEVELOPER_GUIDE.md) and [how to contribute to this project](CONTRIBUTING.md).

diff --git a/files/internal_users.yml b/files/internal_users.yml
@@ -0,0 +1,26 @@
+---
+# This is the internal user database
+# The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh
+
+_meta:
+  type: "internalusers"
+  config_version: 2
+
+# Define your internal users here
+
+admin:
+  hash: "{{ admin_password }}"
+  reserved: true
+  backend_roles:
+  - "admin"
+  description: "admin user"
+
+kibanaserver:
+  hash: "{{ kibanaserver_password }}"
+  reserved: true
+  description: "kibanaserver user"
+
+logstash:
+  hash: "{{ logstash_password }}"
+  reserved: true
+  description: "logstash user"
diff --git a/files/roles.yml b/files/roles.yml
@@ -0,0 +1,54 @@
+---
+_meta:
+  type: "roles"
+  config_version: 2
+
+
+indexes_full_access:
+  reserved: false
+  index_permissions:
+    - index_patterns:
+        - "*"
+      allowed_actions:
+        - "*"
+  tenant_permissions:
+  - tenant_patterns:
+    - "*"
+    allowed_actions:
+    - "kibana_all_write"
+# ----------------------------------------------------
+indexes_security_search_full_access:
+  reserved: true
+  index_permissions:
+    - index_patterns:
+        - "kube-apiserver-audit-*"
+        - "syslog-*"
+      allowed_actions:
+        - "indices:data/read/search*"
+        - "read"
+        - "view_index_metadata"
+  tenant_permissions:
+  - tenant_patterns:
+    - "SECURITY"
+    allowed_actions:
+    - "kibana_all_write"
+# ----------------------------------------------------
+indexes_web_search_full_access:
+  reserved: true
+  index_permissions:
+    - index_patterns:
+        - "ingress-nginx-*"
+        - "mywebapp-*"
+      allowed_actions:
+        - "indices:data/read/search*"
+        - "read"
+        - "view_index_metadata"
+  tenant_permissions:
+  - tenant_patterns:
+    - "WEB"
+    allowed_actions:
+    - "kibana_all_write"
+# ----------------------------------------------------
+# Restrict users so they can only view visualization and dashboard on OpenSearchDashboards
+kibana_read_only:
+  reserved: true
diff --git a/files/roles_mapping.yml b/files/roles_mapping.yml
@@ -0,0 +1,59 @@
+---
+# In this file users, backendroles and hosts can be mapped to Security roles.
+# Permissions for OpenSearch roles are configured in roles.yml
+
+_meta:
+  type: "rolesmapping"
+  config_version: 2
+
+kibana_server:
+  reserved: true
+  users:
+  - "kibanaserver"
+
+logstash:
+  reserved: true
+  users:
+  - "logstash"
+
+# Define your roles mapping here
+all_access:
+  reserved: false
+  backend_roles:
+  - "admin"
+  - "opensearch_admin"
+  description: "Maps admin to all_access"
+# ----------------------------------------------------
+indexes_full_access:
+  reserved: false
+  backend_roles:
+  - "opensearch_admin"
+  description: "Maps admin to indexes_full_access"
+# ----------------------------------------------------
+own_index:
+  reserved: false
+  users:
+  - "*"
+  description: "Allow full access to an index named like the username"
+# ----------------------------------------------------
+readall:
+  reserved: false
+  backend_roles:
+  - "opensearch_index_read_all"
+# ----------------------------------------------------
+indexes_security_search_full_access:
+  reserved: true
+  backend_roles:
+  - "opensearch_index_read_all"
+  - "opensearch_index_read_security"
+  description: "Maps users to indexes_security_search_full_access"
+# ----------------------------------------------------
+indexes_web_search_full_access:
+  reserved: true
+  backend_roles:
+  - "opensearch_index_read_all"
+  - "opensearch_index_read_web"
+  description: "Maps users to indexes_web_search_full_access"
+
+
+
diff --git a/files/tenants.yml b/files/tenants.yml
@@ -0,0 +1,12 @@
+---
+_meta:
+  type: "tenants"
+  config_version: 2
+
+# Define your tenants here
+SECURITY:
+  reserved: false
+  description: "Tenant for security logs (e.g. kubernetes audit or opensearch audit)"
+WEB:
+  reserved: false
+  description: "Tenant for web-app logs"
diff --git a/inventories/opensearch/group_vars/all/all.yml b/inventories/opensearch/group_vars/all/all.yml
@@ -30,3 +30,65 @@ cluster_type: multi-node
 os_user: opensearch
 
 os_dashboards_user: opensearch-dashboards
+
+# Number of days that certificates are valid
+cert_valid_days: 730
+
+# Auth type: 'internal' or 'oidc' (OpenID). Default: internal
+auth_type: internal
+
+# OIDC settings
+oidc:
+  description: "Authenticate via IdP"
+  # OpenID server URI
+  connect_url: https://oidc.example.com/auth/realms/<realm name>/.well-known/openid-configuration
+  # The JWT token field that contains the user name
+  subject_key: preferred_username
+  # the JWT token field that contains a list of user roles
+  roles_key: roles
+  # Scopes
+  scopes: "openid profile email"
+  # The address of Dashboards to redirect the user to after successful authentication
+  dashboards_url: http(s)://<dashboards address>.example.com
+  # IdP client ID
+  client_id: opensearch
+  # IdP client secret
+  client_secret: "00000000-0000-0000-0000-000000000000"
+
+# Overwrite demo configurations with your own
+copy_custom_security_configs: false
+
+# To override demo configurations, you can use your own configuration files.
+# Place them in the "files" directory. Specify the path to the files
+custom_security_plugin_configs:
+  - files/tenants.yml
+  - files/roles.yml
+  - files/roles_mapping.yml
+  - files/internal_users.yml
+
+# By default, if the /tmp/opensearch-nodecerts directory with certificates
+# exists on the server from which the playbook is launched, it is assumed
+# that the configuration has not changed and some settings are not copied
+# to the target servers.
+#
+# Conversely, if the /tmp/opensearch-nodecerts directory does not exist on
+# the server from which the playbook is launched, then new certificates and
+# settings are generated and they are copied to the target servers.
+#
+# If you use this repository not only for the initial deployment of the
+# cluster, but also for its automatic configuration via CI/CD, then new
+# certificates will be generated every time the pipeline is launched,
+# overwriting existing ones, which is not always necessary if the cluster is
+# already in production.
+#
+# When iac_enable enabling, and all the cluster servers have all the necessary
+# certificates, they will not be copied again. If at least on one server (for
+# example, when adding a new server to the cluster) if there is not at least one
+# certificate from the list, then all certificates on all cluster servers will
+# be updated
+#
+# Also, if the option is enabled, the settings files will be updated with each
+# execution (previously, the settings were updated only if the
+# /tmp/opensearch-nodecerts directory was missing on the server from which the
+# playbook was launched and new certificates were generated)
+iac_enable: false
diff --git a/roles/linux/dashboards/defaults/main.yml b/roles/linux/dashboards/defaults/main.yml
@@ -17,3 +17,6 @@ os_nodes_dashboards: |-
         {%- endfor %}
 
 systemctl_path: /etc/systemd/system
+
+# Auth type: 'internal' or 'oidc' (OpenID). Default: internal
+auth_type: internal
diff --git a/roles/linux/dashboards/tasks/dashboards.yml b/roles/linux/dashboards/tasks/dashboards.yml
@@ -6,24 +6,26 @@
     dest: "/tmp/opensearch-dashboards.tar.gz"
   register: download
 
-- name: Dashboards Install | Create opensearch user
+- name: Dashboards Install | Create opensearch dashboard user
   user:
     name: "{{ os_dashboards_user }}"
     state: present
-    shell: /bin/bash
-  when: download.changed
+    shell: /bin/false
+    create_home: true
+    home: "{{ os_dashboards_home }}"
+  when: download.changed or iac_enable
 
 - name: Dashboards Install | Create home directory
   file:
     path: "{{ os_dashboards_home }}"
     state: directory
     owner: "{{ os_dashboards_user }}"
     group: "{{ os_dashboards_user }}"
-  when: download.changed
+  when: download.changed or iac_enable
 
 - name: Dashboards Install | Extract the tar file
   command: chdir=/tmp/ tar -xvzf opensearch-dashboards.tar.gz -C "{{ os_dashboards_home }}" --strip-components=1
-  when: download.changed
+  when: download.changed or iac_enable
 
 - name: Dashboards Install | Copy Configuration File
   template:

diff --git a/roles/linux/dashboards/templates/opensearch_dashboards.yml b/roles/linux/dashboards/templates/opensearch_dashboards.yml
@@ -11,3 +11,15 @@ opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
 opensearch_security.readonly_mode.roles: ["kibana_read_only"]
 # Use this setting if you are running dashboards without https
 opensearch_security.cookie.secure: false
+
+
+# OpenID settings
+{% if auth_type == 'oidc' %}
+opensearch_security.auth.type: openid
+opensearch_security.openid.base_redirect_url: "{{ oidc.dashboards_url }}"
+opensearch_security.openid.client_id: "{{ oidc.client_id }}"
+opensearch_security.openid.scope: "{{ oidc.scopes }}"
+opensearch_security.openid.client_secret: "{{ oidc.client_secret }}"
+opensearch_security.openid.connect_url: "{{ oidc.connect_url }}"
+opensearch_security.openid.verify_hostnames: true
+{% endif %}
diff --git a/roles/linux/opensearch/defaults/main.yml b/roles/linux/opensearch/defaults/main.yml
@@ -20,3 +20,6 @@ os_sec_plugin_tools_path: /usr/share/opensearch/plugins/opensearch-security/tool
 os_api_port: 9200
 
 systemctl_path: /etc/systemd/system
+
+# Auth type: 'internal' or 'oidc' (OpenID). Default: internal
+auth_type: internal
diff --git a/roles/linux/opensearch/tasks/opensearch.yml b/roles/linux/opensearch/tasks/opensearch.yml
@@ -10,20 +10,22 @@
   user:
     name: "{{ os_user }}"
     state: present
-    shell: /bin/bash
-  when: download.changed
+    shell: /bin/false
+    create_home: true
+    home: "{{ os_home }}"
+  when: download.changed or iac_enable
 
 - name: OpenSearch Install | Create home directory
   file:
     path: "{{ os_home }}"
     state: directory
     owner: "{{ os_user }}"
     group: "{{ os_user }}"
-  when: download.changed
+  when: download.changed or iac_enable
 
 - name: OpenSearch Install | Extract the tar file
   command: chdir=/tmp/ tar -xvzf opensearch.tar.gz -C "{{ os_home }}" --strip-components=1
-  when: download.changed
+  when: download.changed or iac_enable
 
 - name: OpenSearch Install | Copy Configuration File
   blockinfile: