Merge pull request #13 from netboxlabs/main

main to dev

Author: netboxlabs-nvp

docs/configure-awx-aap.md

@@ -1,18 +1,18 @@
-# Configure AWX or Tower/AAP
+# Configure AWX/Tower/AAP
 
-*You only need to do this configuration step if you intend to use AWX or Tower/AAP to handle your Proxmox automation.*
+*You only need to do this configuration step if you intend to use AWX/Tower/AAP to handle your Proxmox automation.*
 
-Certainly, you do not need to do Ansible automation by using webhooks and event rules (triggering) in NetBox.  [This weblog](https://netboxlabs.com/blog/getting-started-with-network-automation-netbox-ansible/) shows you how you can use [Ansible](https://www.ansible.com/) with NetBox, as network source of truth, to induce changes in your environment -- by using a pull method for your automation from any client on your network.  In this example, you'll be able to run `ansible-playbook`, alongside a dynamic inventory (NetBox) to induce automation, or in this case automating changes to Proxmox VMs.
+Certainly, you do not need to do Ansible automation by using webhooks and event rules (triggering) in NetBox.  [This weblog](https://netboxlabs.com/blog/getting-started-with-network-automation-netbox-ansible/) shows you how you can use [Ansible](https://www.ansible.com/) with NetBox, as network source of truth, to induce changes in your environment -- by facilitating automation from any client on your network.  In that example, you'd be able to run `ansible-playbook`, alongside a dynamic inventory (NetBox) to induce automation, and from there you could add Proxmox VM automation.
 
-However, many other NetBox users want to use NetBox as NSoT (network source of truth) to facilitate their Proxmox VM automation.  Changes to Proxmox VMs in NetBox will result in automation being kicked off, in this case via [AWX](https://github.com/ansible/awx), or perhaps for a Red Hat commercial customer, through [Tower/AAP](https://www.redhat.com/en/solutions/it-automation?sc_cid=7015Y000003sm3kQAA&gad_source=1&gclid=CjwKCAiAl4a6BhBqEiwAqvrqugh1f-1RfeP-NQxOKYhSbwJqUPVqGqR1A0ScrGMdNhLUbdTayU-EOhoCg00QAvD_BwE&gclsrc=aw.ds).  By using webhooks and event rules in NetBox, AWX or Tower/AAP are more than capable of inducing Proxmox automation.  In fact, using AWX or Tower/AAP is the preferred method for large environments -- where Proxmox VM deployment is a part of an underlying CI/CD.
+However, many other NetBox users want to use NetBox as NSoT (network source of truth) to facilitate their Proxmox VM automation.  Changes to (Proxmox) VMs in NetBox will result in automation being kicked off, in this case via [AWX](https://github.com/ansible/awx), or perhaps for a Red Hat commercial customer, through [Tower/AAP](https://www.redhat.com/en/solutions/it-automation?sc_cid=7015Y000003sm3kQAA&gad_source=1&gclid=CjwKCAiAl4a6BhBqEiwAqvrqugh1f-1RfeP-NQxOKYhSbwJqUPVqGqR1A0ScrGMdNhLUbdTayU-EOhoCg00QAvD_BwE&gclsrc=aw.ds).  By using webhooks and event rules in NetBox, AWX/Tower/AAP are more than capable of inducing Proxmox automation.  In fact, using AWX/Tower/AAP is the preferred method for large environments -- where Proxmox VM deployment might be a part of an underlying CI/CD process.
 
-For those who are unfamiliar, AWX is the upstream (community, i.e. free) version of AAP.  Functionally, AWX works the same way as Tower/AAP, but without the commercial support.  AWX is an excellent alternative as you work through NetBox/Proxmox automation, but there can be a heavy lift when it comes to configuring AWX for the first time.  This section talks through the steps you'll need to be able to run AWX and to begin your Proxmox VM automation journey with NetBox.
+For those who are unfamiliar, AWX is the upstream (community, i.e. free) version of AAP.  Functionally, AWX works the same way as Tower/AAP, but without the commercial support.  AWX is an excellent alternative as you work through NetBox/Proxmox automation, but it can be a heavy lift when configuring AWX for the first time.  This section talks through the steps you'll need to be able to run AWX and to begin your Proxmox VM automation journey with NetBox.
 
 ### Installing AWX with docker-compose
 
-AWX (or Tower/AAP) are typically installed in an environment where Kuberenetes (k8s) is available.  However, should you have Docker/docker-compose running on your local system, you should be able to install AWX [this way](https://github.com/ansible/awx/blob/devel/tools/docker-compose/README.md).
+AWX/Tower/AAP are typically installed in an environment where Kuberenetes (k8s) is available.  However, should you have Docker/docker-compose running on your local system, you should be able to install AWX [this way](https://github.com/ansible/awx/blob/devel/tools/docker-compose/README.md).
 
-Once you have installed AWX (or Tower/AAP) in your environment, and are able to login, as an 'admin' user through the UI, you can start configuring AWX (or Tower/AAP) to facilitate your Proxmox VM automation.  *Note that you can add whatever user(s)/group(s) that you want to AWX, but make sure that whatever user(s)/group(s) you add to AWX have the appropriate permissions to manage the following.*
+Once you have installed AWX/Tower/AAP in your environment, and are able to login, as an 'admin' user through the UI, you can start configuring AWX/Tower/AAP to facilitate your Proxmox VM automation.  *Note that you can add whatever user(s)/group(s) that you want to AWX, but make sure that whatever user(s)/group(s) you add to AWX have the appropriate permissions to manage the following.*
 
 #### Create Github (or your Git of choice) Credential in AWX
 
@@ -30,11 +30,11 @@ Navigate to Resources > Inventories.  'Demo Inventory' should be sufficient for
 
 #### Create Execution Environment in AWX
 
-Typically, when `ansible` or `ansible-playbook` is/are executed from the command line, this is done via a Python3 `venv`.  However, with AWX, there is no such capability to interact with a command line to leverage `venv` to do a `pip install` of Python module dependencies.
+Typically, when `ansible` or `ansible-playbook` is/are executed from the command line, this is done via a Python3 `venv`.  However, with AWX, there is no such capability to interact with a command line to leverage `venv` to do a `pip install` of NetBox/Python module dependencies.
 
-As a result, you will need to use an [Execution Environment](https://ansible.readthedocs.io/projects/awx/en/latest/userguide/execution_environments.html) in AWX.  Your Execution Environment is a container image that will include all of the (Python) module dependencies that you'll need to facilitate Proxmox automation, and this container image will live in your container registry of choice.  
+As a result, you will need to use an [Execution Environment](https://ansible.readthedocs.io/projects/awx/en/latest/userguide/execution_environments.html) in AWX.  Your Execution Environment is a container image that will include all of the (NetBox/Python) module dependencies that you'll need to facilitate Proxmox automation, and this container image will live in your container registry of choice.  
 
-*You (probably) only need to create an Exection Environment once for `netbox-proxmox-automation` with AWX.*
+*You (probably) only need to create an Exection Environment once for `netbox-proxmox-automation` with AWX/Tower/AAP.*
 
 In the end, your Execution Environment should look like this in AWX.
 

docs/index.md

@@ -6,17 +6,19 @@ When you think of the challenges of a widely used network documentation solution
 
 This automation handles creation, removal, and changes of/to Proxmox VMs.  The underlying automation uses [webhooks](https://demo.netbox.dev/static/docs/additional-features/webhooks/) and [event rules](https://netboxlabs.com/docs/netbox/en/stable/features/event-rules/) in NetBox.  When you induce a change in NetBox, this will set the desired VM state(s) in Proxmox.
 
-When you create VM objects in NetBox, the following will take place in Proxmox:
+When you create/update/delete VM objects in NetBox, the following will take place in Proxmox:
 
 - when you create a VM object in NetBox (name, status == Staged, chosen Proxmox VM template name), this will clone a VM in Proxmox of the same name, from the defined template
-- when you add a SSH key and/or primary IP address to a NetBox VM object (status == Staged), this will update the VM settings in Proxmox -- adding ipconfig0 and ssh key settings
-- when you add disks (scsi0 - scsiN) to a NetBox VM object (status == Staged), this will:
+- when you add a SSH key to a NetBox VM object (status == Staged), a SSH key will be added to the VM settings in Proxmox
+- when you add a primary IP address to a NetBox VM object (status == Staged), this will update the VM settings in Proxmox for ipconfig0
+- when you add or resize VM disks (scsi0 - scsiN) for a NetBox VM object (status == Staged), this will:
   - resize scsi0 on the Proxmox VM to the size that was defined in NetBox
   - create scsi1 - scsiN on the Proxmox VM and set them to their specified sizes
+  - resize scsi1 - scsiN on the Proxmox VM and resize them to their specified sizes (*NOTE: Proxmox does not allow you to shrink disks!*)
 - when you remove a disk or disks from a NetBox VM object, this will remove the corresponding disks from the Proxmox VM (*NOTE: this does not include scsi0 as that is the OS disk*)
 
 Further:
 
-- when you set a VM's state to 'active' in NetBox, this will start a VM in Proxmox
-- when you set a VM's state to 'offline' in NetBox, this still stop a VM in Proxmox
+- when you set a VM's state to 'Active' in NetBox, this will start a VM in Proxmox
+- when you set a VM's state to 'Offline' in NetBox, this still stop a VM in Proxmox
 - when you remove a VM from NetBox, this will stop and remove a VM in Proxmox.

docs/netbox-event-rules-and-webhooks-awx-aap.md

@@ -9,7 +9,7 @@ A webhook in NetBox will consume the payload of data from an event rule.  An eve
 
 For the sake of automation, every event rule that you create in NetBox requires either a Webhook or a Script.
 
-Regardless of whether you are using a Flask (or other) application for Proxmox automation, or you are using AWX or Tower/AAP, this automation should trigger anytime that a Proxmox VM is changed in NetBox such that:
+Regardless of whether you are using a Flask (or other) application for Proxmox automation, or you are using AWX/Tower/AAP, this automation should trigger anytime that a Proxmox VM is changed in NetBox such that:
 
 - a Proxmox VM has been created in NetBox with a status of 'Staged'
 - a Proxmox VM in NetBox (with a status of 'Staged') has a changed network configuration
@@ -20,11 +20,11 @@ Regardless of whether you are using a Flask (or other) application for Proxmox a
 - a Proxmox VM in NetBox has been removed
 
 
-### AWX or Tower/AAP
+### AWX/Tower/AAP
 
-As noted earlier, AWX or Tower/AAP will perform Proxmox automation through separate (project) templates.  This section walks you through how (NetBox) webhooks and (NetBox) event rules are handled by AWX.
+As noted earlier, AWX/Tower/AAP will perform Proxmox automation through separate (project) templates.  This section walks you through how (NetBox) webhooks and (NetBox) event rules are handled by AWX.
 
-#### AWX or Tower/AAP Webhook
+#### AWX/Tower/AAP Webhook
 
 To use NetBox webhooks with AWX, each NetBox webhook for Proxmox VM management will point at a separate AWX (project) template.  In AWX, each (project) template has a unique ID.  When we execute a webhook in NetBox, in this case we're using AWX, the (NetBox) webhook will in turn point at the (project) template ID in AWX -- and tell AWX to launch the template, i.e. to run the automation.
 
@@ -79,7 +79,7 @@ Regardless of which AWX template you use as a (NetBox) webhook, you must include
 ![NetBox Proxmox VM resize disk AWX webhook image](./images/proxmox-vm-resize-disk-awx.png)
 
 
-#### AWX or Tower/AAP Event Rules
+#### AWX/Tower/AAP Event Rules
 
 Now let's take a look at the NetBox event rules that call an AWX webhook (project template) with Proxmox VM and VM disk object changes in Netbox.
 

docs/netbox-event-rules-and-webhooks-flask.md

@@ -9,7 +9,7 @@ A webhook in NetBox will consume the payload of data from an event rule.  An eve
 
 For the sake of automation, every event rule that you create in NetBox requires either a Webhook or a Script.
 
-Regardless of whether you are using a Flask (or other) application for Proxmox automation, or you are using AWX or Tower/AAP, this automation should trigger anytime that a Proxmox VM is changed in NetBox such that:
+Regardless of whether you are using a Flask (or other) application for Proxmox automation, or you are using AWX/Tower/AAP, this automation should trigger anytime that a Proxmox VM is changed in NetBox such that:
 
 - a Proxmox VM has been created in NetBox with a status of 'Staged'
 - a Proxmox VM in NetBox (with a status of 'Staged') has a changed network configuration

docs/netbox-ipam.md

@@ -2,10 +2,10 @@
 
 Given the heirarchical nature of NetBox, you will need to create the following objects before using `netbox-proxmox-automation` automation.  You should refer to the [NetBox planning guide](https://netboxlabs.com/docs/netbox/en/stable/getting-started/planning/) to address these dependencies before proceeding with `netbox-proxmox-automation`.
 
-Using NetBox's IPAM is a *requirement* of `netbox-proxmox-automation`.  This is because `netbox-proxmox-automation` is going to either assign a defined IP address to a specified inteface (or interfaces) on a Proxmox VM, or it's going to request an available IP address from NetBox's IPAM -- and assign the requested IP address to an interface (or interfaces) on a Proxmox VM.
+Using NetBox's IPAM is a *requirement* of `netbox-proxmox-automation`.  This is because `netbox-proxmox-automation` is going to either assign a defined IP address to a specified interface (or interfaces) on a Proxmox VM, or it's going to request an available IP address from NetBox's IPAM -- and assign the requested IP address to an interface (or interfaces) on a Proxmox VM.
 
 Ahead of using this automation, make sure to create the following IPAM-related objects in NetBox:
 
 - IPAM > RIRs
-- IPAM > Aggregates (relate each aggregate to RIR)
-- IPAM > Prefixes (use containers and set Active state for each active previx)
+- IPAM > Aggregates (relate each aggregate to an RIR)
+- IPAM > Prefixes (use containers and set Active state for each active prefix)

docs/netbox-key-and-permissions.md

@@ -4,9 +4,10 @@
 
 It is recommended that you do *not* create an API token for the NetBox 'admin' user.  Instead, create a new user in NetBox; then create a new permission for that API user -- that has sufficient read/write/modify permissions to modify the following object types in NetBox, at a minimum:
 
-  - Devices
+  - Devices (for VM cluster(s) hardware, if used)
   - Interfaces (devices and VMs)
   - VMs (groups, clusters, VMs)
+  - VM disks
 
 ### Create NetBox User + Group
 

docs/proxmox-api-user-and-key.md

@@ -46,18 +46,3 @@ proxmox-ve-shell# pveum user token add api_user@pve api_user_token -privsep 0 #
 
 proxmox-ve-shell# pveum acl modify / -user api_user@pve -role Administrator # allow api_user@pve to access everything -- given Administrator role rights
 ```
-
-For the command line above, note that you *will get the Proxmox API token via stdout only once*.  Make sure to copy and store this token in a safe place.  You will need it when we generate the Ansible `secrets.yml` configuration in the next step.
-
-Use `netbox-proxmox-discover-vms.yml` to discover and Proxmox VMs which aren't already in NetBox.  This procedure will also make incremental changes to existing Proxmox VMs in NetBox -- such as adding MAC address to network interfaces.
-
-## Usage
-
-```
-shell$ cd /path/to/netbox-proxmox-automation
-
-shell$ source venv/bin/activate
-
-(venv) shell$ ansible-playbook -i inventory netbox-proxmox-discover-vms.yml --ask-vault-pass
-```
-