feat/custom mounts

ansible/.gitignore

@@ -100,3 +100,5 @@ roles/*
 !roles/topology/**
 !roles/raid/
 !roles/raid/**
+!roles/custom_mounts/
+!roles/custom_mounts/**

ansible/roles/custom_mounts/README.md

@@ -0,0 +1,185 @@
+custom_mounts
+=====
+
+This Ansible role automates the mounting of CIFS, NFS, and CephFS/Ceph RBD, using either fstab or autofs.
+
+Requirements
+------------
+
+None.
+
+Role Variables
+--------------
+
+- `ceph_mt_conf_src_dir`: Path to the Ceph `.conf` file. Default: `{{ role_path }}/files`
+- `ceph_common_version`: Version of Ceph to install. Default: `"2:19.2.2"`. If set to `false`, the latest version will be installed. Ceph will not be updated if already installed.
+- `ceph_repo_release`: Ceph repository release name corresponding to the version. Default: `squid`. Ceph will not be updated if already installed.
+
+- `custom_mounts`: Mounts are defined in a dictionary format. Each key represents a unique mount configuration. The role supports multiple mount types and methods, each with specific requirements and behaviors.
+
+```yaml
+custom_mounts:
+  <mount_name>:                  # Arbitrary unique name for the mount
+    method: <fstab|autofs>
+    type: <cifs|nfs|cephfs|ceph_rbd>
+    fs_path: <source_path>
+    mount_point: <target_path>
+    [master_mount_point]: <autofs base path>  # optional autofs only
+    [autofs_options]: <autofs map options>    # optional autofs only
+    fs_name: <ceph cluster name>              # ceph only
+    pool_name: <rbd.pool.name>                # ceph_rdb only
+    image_name: <rdb_image>                   # ceph_rdb only
+    fstype: <xfs|ext4>                        # ceph_rdb only
+    credentials:                              # cifs or ceph
+      username: <user>                        # cifs only
+      password: <pass>                        # cifs only
+      domain: <domain>                        # cifs only
+      file_path: <path to credential file>    # cifs only
+      client_name: <client_name>              # ceph only      
+      secret: <ceph client key>               # ceph only
+      ceph_conf: <ceph.conf name>             # optional ceph only
+    [mount_owner]: <owner>                    # optional fstab only
+    [mount_group]: <group>                    # optional fstab only
+    [mount_mode]: <permissions>               # optional fstab only
+    [mount_opts]: <mount options>             # optional
+    [state]: <mounted|unmounted>              # optional fstab only
+    [dump]: <0|1>                             # optional fstab only
+    [passno]: <0|1>                           # optional fstab only
+```
+
+Mount Type Differences
+----------------------
+
+1. **CIFS/SMB** `type: cifs` (Windows-style network shares)
+- Compatible with method: **autofs** and **fstab**.
+- Requires `fs_path` in UNC format: `//host/share`.
+- Requires a `credentials:` section (`username`, `password`, `domain`, `file_path`).
+  - Credentials are stored in a file and referenced via `file_path`.
+- Mount options may include `vers`, `uid`, `gid`, `mfsymlinks`, etc.
+- Credentials are stored in a file and referenced via `file_path`.
+  - If `master_mount_point` is not used, the base directory of `fs_path` is used as `master_mount_point`.
+- When method: **autofs**, 
+  - If `master_mount_point` is used, the `fs_path` is treated as a sub-location inside `master_mount_point`.
+  - `autofs_options` are optional to be added to `/etc/auto.master.d/mountkey.autofs` e.g. `--timeout 60`
+  -  Mount points settings `mount_owner`, `mount_group`, `mount_mode` will not persist after autofs mounting. These need to be set as mount_opts e.g. `mount_opts: uid=5000,gid=5002,file_mode=0770,dir_mode=0770`
+
+
+2. **NFS** `type: nfs`  (Unix-style network shares)
+- Compatible with method: **autofs** and **fstab**.
+- `fs_path` format: `host:/export/path`.
+- Typically does not require credentials.
+- Mount options may include `rw`, `nofail`, `_netdev`, etc.
+- If **autofs** is used with `master_mount_point`, the `fs_path` is treated as a sub-location inside `master_mount_point`.  
+  - If `master_mount_point` is not used, the base directory of `fs_path` is used as `master_mount_point`.
+
+3. **CephFS** `type: cephfs`
+- Only supports Requires: `fstab` (not `autofs`).
+- Requires `fs_name` (e.g., `fast_cephfs`) and access to a Ceph configuration file.
+- Credentials include `client_name`, `secret`, and `ceph_conf`.  
+  - The `ceph.conf` file is copied from the Ansible control host.  
+  - The keyring file is templated per client.
+
+4. **Ceph RDB** `type: ceph_rdb`
+- Only supports method: `fstab` (not `autofs`).
+- Same requirements as CephFS
+- Requires: 
+  - `pool_name`: ceph pool name e.g. rbd.fast_rbd 
+  - `image_name`: e.g. test_rdb_image
+  - `fstype`: e.g. `xfs` 
+
+NB,for method **autofs** mount points settings `mount_owner`, `mount_group`, `mount_mode` will not persist after autofs mounting. 
+
+Dependencies
+------------
+
+None.
+
+Example Playbook
+----------------
+
+```yaml
+- hosts: Add custom_mounts
+  become: true
+  tags: custom_mounts
+  tasks:
+    - include_role:
+        name: custom_mounts
+        tasks_from: "{{ 'install_packages.yml' if appliances_mode == 'build' else 'main.yml' }}"
+```
+
+Example custom_mounts
+----------------
+
+```yaml
+custom_mounts:
+  mount_cifs:
+    method: fstab
+    type: cifs
+    fs_path: '//192.168.124.218/install_share'
+    mount_point: /mnt/install_share 
+    mount_owner: "{{ analysis_user }}"
+    mount_group: "{{ analysis_group }}"
+    mount_mode: "0770"
+    mount_opts: "mfsymlinks,vers=3.02,gid=1002,forcegid,uid=5000,forceuid,dir_mode=0770,"
+    state: mounted
+    dump: 0
+    passno: 0
+  autofs_cifs_share:
+    method: autofs
+    type: cifs
+    fs_path: '//192.168.124.218/dev-pengu-fs'
+    mount_point: /mnt/logs
+    credentials:
+      username: XX_PenGU
+      password: "dfsf"
+      domain: CYMRU
+      file_path: /etc/.cifs_credentials
+  nfs_fstab:
+    method: fstab
+    type: nfs
+    fs_path: '192.168.124.203:/iso_sr_2'
+    mount_point: /mnt/iso_sr_2_nfs_fstab
+  autofs_nfs:
+    method: autofs
+    type: nfs
+    fs_path: '192.168.124.203:/iso_share'
+    mount_point: iso_share
+    master_mount_point: /mnt/autofs_nfs
+    autofs_options: "--timeout 60"
+  cephfs_fstab1:
+    method: fstab
+    type: cephfs
+    fs_name: fast_fs
+    fs_path: /volumes/_nogroup/syslogs_subvol/03bb4b # path after the filesystem
+    mount_point: /mnt/cephfs
+    mount_opts: "_netdev"
+    credentials:
+      ceph_conf: ceph.conf
+      client_name: fast_fs_rw
+      secret: keykeykey
+  ceph_rbd_data:
+    method: fstab
+    type: ceph_rbd
+    pool_name: rbd.fast_rbd 
+    image_name: test_rdb_image  
+    fstype: xfs 
+    mount_point: /mnt/test_rdb_image 
+    mount_opts: "_netdev"
+    state: mounted
+    dump: 0
+    passno: 0
+    credentials:
+      ceph_conf: ceph.conf
+      client_name: fast_rbd # without .client 
+      secret: dsfdfv4
+```
+
+License
+-------
+
+Apache v2
+
+Author Information
+------------------
+
+Jonathan Jenkins [email protected]

ansible/roles/custom_mounts/defaults/main.yml

@@ -0,0 +1,72 @@
+---
+
+ceph_mt_conf_src_dir: "{{ appliances_environment_root }}/files"
+# ceph_mt_conf_src_dir: "{{ role_path }}/files"
+ceph_common_version: "2:19.2.2"  # if false will be latest
+ceph_repo_release: squid
+
+custom_mounts: {}
+# custom_mounts:
+#   mount_cifs: # name of mount 
+#     method: fstab      # autofs or fstab
+#     type: cifs          # cifs or cephfs or ceph_rbd
+#     # fs_name: "192.168.124.218" # ceph only 
+#     fs_path: '//192.168.124.218/install_share' # cifs 
+#     mount_point: /mnt/install_share 
+#     mount_owner: "{{ analysis_user }}" # optional owner of mount point default:root
+#     mount_group: "{{ analysis_group }}" # optional group of mount point  default:root
+#     mount_mode: "0770"         # optional permissions  of mount point default:0775
+#     mount_opts: "mfsymlinks,vers=3.02,gid=1002,forcegid,uid=5000,forceuid,dir_mode=0770,_netdev,nofail" #o ptional
+#     state: mounted # optional default('mounted') 
+#     dump: 0 # optional
+#     passno: 0 # optional
+#   autofs_cifs_share: # arbitary name of name of mount mus be unique  
+#     method: autofs      # autofs or fstab
+#     type: cifs          # only cifs or nfs compatable with autofs
+#     # fs_name: bioinformatics_share # ceph only
+#     fs_path: '//192.168.124.218/dev-pengu-fs' # cifs and nfs this must be the full source
+#     mount_point: /mnt/logs 
+#     mount_owner: "{{ analysis_user }}" 
+#     mount_group: "{{ analysis_group }}"
+#     mount_mode: "0770"
+#     mount_opts: "noatime,ro"
+#     dump:
+#     credentials:
+#       username: PHW_PenGU
+#       password: "dfsf"
+#       domain: CYMRU
+#       file_path: /etc/.cifs_credentials
+#   nfs_fstab: # name of mount 
+#     method: fstab      # autofs or fstab
+#     type: nfs          
+#     fs_path: '192.168.124.203:/iso_sr_2' # for nfs and cifs must be src_path
+#     mount_point: /mnt/iso_sr_2_nfs_fstab
+#     mount_owner: "{{ analysis_user }}"
+#     mount_group: "{{ analysis_group }}"
+#     mount_mode: "0770"
+#     mount_opts: "rw"
+#   autofs_nfs: 
+#     method: autofs      
+#     type: nfs          # cifs or cephfs or ceph_rbd 
+#     fs_path: '192.168.124.203:/iso_share' # cifs
+#     mount_point: iso_share # If master_mount_point used should be single name not path
+#     master_mount_point: /mnt/autofs_nfs # optional for autofs
+#     autofs_options: "--timeout 60" # optional for autofs to at to auto.master.d/ options 
+#     mount_owner: "{{ analysis_user }}" # optional
+#     mount_group: "{{ analysis_group }}" # optional
+#     mount_mode: "0770" # optional
+#     mount_opts: "rw" # optional
+# ceph_rbd_data:
+#     method: fstab
+#     type: ceph_rbd   # must use method fstab
+#     pool_name: rbd.fast_rbd # ceph rdb only
+#     image_name: test_rdb_image  # ceph rdb only
+#     fstype: xfs # ceph rdb only
+#     mount_point: /mnt/test_rdb_image 
+#     mount_opts: "_netdev"
+#     state: mounted
+#     dump: 0
+#     passno: 0
+#     credentials:
+#       client_name: fast_rbd # without .client 
+#       secret: dsfdfv4

ansible/roles/custom_mounts/handlers/main.yml

@@ -0,0 +1,7 @@
+---
+
+- name: restart autofs
+  become: true
+  ansible.builtin.service:
+    name: autofs
+    state: restarted

ansible/roles/custom_mounts/tasks/autofs_main.yml

@@ -0,0 +1,78 @@
+---
+
+- name: "{{ abs_mount_point }} - Warn and fail if ceph_rbd is used with autofs"
+  ansible.builtin.fail:
+    msg: >
+      Ceph volumes (type=ceph_rbd or cephfs) should not be mounted using autofs.
+      Please use method=fstab instead.
+  when: autofs_item.value.type == "ceph_rbd"
+
+
+# - name: "{{ abs_mount_point }} - Include ceph client setup"
+#   ansible.builtin.include_tasks: "ceph_setup.yml"
+#   when: autofs_item.value.type in ["cephfs"]
+#   vars:
+#     ceph_item: "{{ autofs_item }}"
+
+
+- name: "{{ abs_mount_point }} - Include cifs/smb client setup"
+  ansible.builtin.include_tasks: "cifs_setup.yml"
+  when: autofs_item.value.type in ["cifs"]
+  vars:
+    cifs_item: "{{ autofs_item }}"
+
+
+- name: "{{ abs_mount_point }} - Set autofs mount point variables"
+  set_fact:
+    autofs_master_mount_point: >-
+      {{ (autofs_item.value.master_mount_point
+          if autofs_item.value.master_mount_point is defined
+          else (autofs_item.value.mount_point | dirname) | default('/-', true)
+        ) | trim }}
+    autofs_mount_point: "{{ (autofs_item.value.mount_point | basename) | trim }}" # not a path 
+    autofs_master_mount_point_slug: >-
+      {{ (
+          autofs_item.value.master_mount_point
+          if autofs_item.value.master_mount_point is defined
+          else (autofs_item.value.mount_point | dirname) | default('fileroot', true)
+        ) | regex_replace('^/', '') | replace('/', '-') | trim }}
+
+
+- name: "{{ abs_mount_point }} - Ensure mount directories exist"
+  become: true
+  ansible.builtin.file:
+    path: "{{ autofs_master_mount_point }}"
+    state: directory
+    owner: "{{ item.value.mount_owner | default(omit) }}"
+    group: "{{ item.value.mount_group | default(omit) }}"
+    mode: "{{ item.value.mount_mode | default(omit) }}"
+
+
+- name: "{{ abs_mount_point }} - Ensure autofs master map fragment is correct"
+  become: true
+  ansible.builtin.lineinfile:
+    path: "/etc/auto.master.d/{{ autofs_master_mount_point_slug }}.autofs"
+    line: "{{ autofs_master_mount_point }} /etc/auto.{{ autofs_master_mount_point_slug }} {{ (autofs_item.value.autofs_options | default('')) | trim }}"
+    regexp: '^{{ autofs_master_mount_point | regex_escape() }}\s+.*$'
+    state: present
+    create: true # Creates the file if it does not exist.
+    mode: '0644'
+  notify: restart autofs
+
+- name: "{{ abs_mount_point }} - Ensure autofs map file entry is correct"
+  become: true
+  vars:
+    line_var: >-
+      {% if autofs_item.value.type == 'cifs' %}
+      {{ autofs_mount_point }} -fstype=cifs,{{ autofs_item.value.mount_opts }},credentials={{ autofs_item.value.credentials.file_path }} :{{ autofs_item.value.fs_path }}
+      {% elif autofs_item.value.type == 'nfs' %}
+      {{ autofs_mount_point }} -fstype=nfs,{{ autofs_item.value.mount_opts }} {{ autofs_item.value.fs_path }}
+      {% endif %}
+  ansible.builtin.lineinfile:
+    path: "/etc/auto.{{ autofs_master_mount_point_slug }}"
+    regexp: '^{{ autofs_mount_point | regex_escape() }}\s+.*$'
+    line: "{{(line_var) | trim}}"
+    state: present
+    create: true
+    mode: '0644'
+  notify: restart autofs