Display/export playbook graph

[Hipska]

What is the idea ?

On the Playbook report page, a new section could show a graph indicating the relation of tasks versus their play(s)/role(s). Clicking a task on the graph could filter the list of task results of the playbook. It should be doable given the fact that ARA records the files.

The API and CLI could also support this by specifying one of the renderers.

Example UI

---
title: Ansible Playbook Grapher
---
%%{ init: { "flowchart": { "curve": "bumpX" } } }%%
flowchart LR
	%% Start of the playbook 'tests/fixtures/multi-plays.yml'
	playbook_a27d9bec("tests/fixtures/multi-plays.yml")
		%% Start of the play 'Play: all (0)'
		play_d6dd122d["Play: all (0)"]
		style play_d6dd122d stroke:#ba1a12,fill:#ba1a12,color:#ffffff
		playbook_a27d9bec --> |"1"| play_d6dd122d
		linkStyle 0 stroke:#ba1a12,color:#ba1a12
			pre_task_676cdcb1["[pre_task] Pretask"]
			style pre_task_676cdcb1 stroke:#ba1a12,fill:#ffffff
			play_d6dd122d --> |"1"| pre_task_676cdcb1
			linkStyle 1 stroke:#ba1a12,color:#ba1a12
			pre_task_44476583["[pre_task] Pretask 2"]
			style pre_task_44476583 stroke:#ba1a12,fill:#ffffff
			play_d6dd122d --> |"2"| pre_task_44476583
			linkStyle 2 stroke:#ba1a12,color:#ba1a12
			%% Start of the role '[role] fake_role'
			play_d6dd122d --> |"3"| role_f0c07194
			linkStyle 3 stroke:#ba1a12,color:#ba1a12
			role_f0c07194(["[role] fake_role"])
			style role_f0c07194 fill:#ba1a12,color:#ffffff,stroke:#ba1a12
				task_90876ffc["[task] fake_role : Debug 1"]
				style task_90876ffc stroke:#ba1a12,fill:#ffffff
				role_f0c07194 --> |"1 [when: ansible_distribution == 'Debian']"| task_90876ffc
				linkStyle 4 stroke:#ba1a12,color:#ba1a12
				task_bb701882["[task] fake_role : Debug 2"]
				style task_bb701882 stroke:#ba1a12,fill:#ffffff
				role_f0c07194 --> |"2 [when: ansible_distribution == 'Debian']"| task_bb701882
				linkStyle 5 stroke:#ba1a12,color:#ba1a12
				task_c00c5d61["[task] fake_role : Debug 3 with double quote "here" in the name"]
				style task_c00c5d61 stroke:#ba1a12,fill:#ffffff
				role_f0c07194 --> |"3 [when: ansible_distribution == 'Debian']"| task_c00c5d61
				linkStyle 6 stroke:#ba1a12,color:#ba1a12
			%% End of the role '[role] fake_role'
			%% Start of the role '[role] display_some_facts'
			play_d6dd122d --> |"4"| role_a168caef
			linkStyle 7 stroke:#ba1a12,color:#ba1a12
			role_a168caef(["[role] display_some_facts"])
			style role_a168caef fill:#ba1a12,color:#ffffff,stroke:#ba1a12
				task_737e2be9["[task] display_some_facts : ansible_architecture"]
				style task_737e2be9 stroke:#ba1a12,fill:#ffffff
				role_a168caef --> |"1"| task_737e2be9
				linkStyle 8 stroke:#ba1a12,color:#ba1a12
				task_61bbb3fb["[task] display_some_facts : ansible_date_time"]
				style task_61bbb3fb stroke:#ba1a12,fill:#ffffff
				role_a168caef --> |"2"| task_61bbb3fb
				linkStyle 9 stroke:#ba1a12,color:#ba1a12
				task_3b7308dc["[task] display_some_facts : Specific included task for Debian"]
				style task_3b7308dc stroke:#ba1a12,fill:#ffffff
				role_a168caef --> |"3"| task_3b7308dc
				linkStyle 10 stroke:#ba1a12,color:#ba1a12
			%% End of the role '[role] display_some_facts'
			task_c8b76065["[task] Add backport {{backport}}"]
			style task_c8b76065 stroke:#ba1a12,fill:#ffffff
			play_d6dd122d --> |"5"| task_c8b76065
			linkStyle 11 stroke:#ba1a12,color:#ba1a12
			task_f7cebcbb["[task] Install packages"]
			style task_f7cebcbb stroke:#ba1a12,fill:#ffffff
			play_d6dd122d --> |"6"| task_f7cebcbb
			linkStyle 12 stroke:#ba1a12,color:#ba1a12
			post_task_caafa665["[post_task] Posttask"]
			style post_task_caafa665 stroke:#ba1a12,fill:#ffffff
			play_d6dd122d --> |"7"| post_task_caafa665
			linkStyle 13 stroke:#ba1a12,color:#ba1a12
			post_task_b5ade468["[post_task] Posttask 2"]
			style post_task_b5ade468 stroke:#ba1a12,fill:#ffffff
			play_d6dd122d --> |"8"| post_task_b5ade468
			linkStyle 14 stroke:#ba1a12,color:#ba1a12
		%% End of the play 'Play: all (0)'
		%% Start of the play 'Play: database (0)'
		play_d780677e["Play: database (0)"]
		style play_d780677e stroke:#686864,fill:#686864,color:#ffffff
		playbook_a27d9bec --> |"2"| play_d780677e
		linkStyle 15 stroke:#686864,color:#686864
			%% Start of the role '[role] fake_role'
			play_d780677e --> |"1"| role_1e6bf323
			linkStyle 16 stroke:#686864,color:#686864
			role_1e6bf323(["[role] fake_role"])
			style role_1e6bf323 fill:#686864,color:#ffffff,stroke:#686864
				task_3cb17d25["[task] fake_role : Debug 1"]
				style task_3cb17d25 stroke:#686864,fill:#ffffff
				role_1e6bf323 --> |"1 [when: ansible_distribution == 'Debian']"| task_3cb17d25
				linkStyle 17 stroke:#686864,color:#686864
				task_1f6232f4["[task] fake_role : Debug 2"]
				style task_1f6232f4 stroke:#686864,fill:#ffffff
				role_1e6bf323 --> |"2 [when: ansible_distribution == 'Debian']"| task_1f6232f4
				linkStyle 18 stroke:#686864,color:#686864
				task_0361ffa3["[task] fake_role : Debug 3 with double quote "here" in the name"]
				style task_0361ffa3 stroke:#686864,fill:#ffffff
				role_1e6bf323 --> |"3 [when: ansible_distribution == 'Debian']"| task_0361ffa3
				linkStyle 19 stroke:#686864,color:#686864
			%% End of the role '[role] fake_role'
			%% Start of the role '[role] display_some_facts'
			play_d780677e --> |"2"| role_a8b4b712
			linkStyle 20 stroke:#686864,color:#686864
			role_a8b4b712(["[role] display_some_facts"])
			style role_a8b4b712 fill:#686864,color:#ffffff,stroke:#686864
				task_c1d17653["[task] display_some_facts : ansible_architecture"]
				style task_c1d17653 stroke:#686864,fill:#ffffff
				role_a8b4b712 --> |"1"| task_c1d17653
				linkStyle 21 stroke:#686864,color:#686864
				task_8353a2ef["[task] display_some_facts : ansible_date_time"]
				style task_8353a2ef stroke:#686864,fill:#ffffff
				role_a8b4b712 --> |"2"| task_8353a2ef
				linkStyle 22 stroke:#686864,color:#686864
				task_d8141ffa["[task] display_some_facts : Specific included task for Debian"]
				style task_d8141ffa stroke:#686864,fill:#ffffff
				role_a8b4b712 --> |"3"| task_d8141ffa
				linkStyle 23 stroke:#686864,color:#686864
			%% End of the role '[role] display_some_facts'
		%% End of the play 'Play: database (0)'
		%% Start of the play 'Play: webserver (0)'
		play_4d3ee472["Play: webserver (0)"]
		style play_4d3ee472 stroke:#43418b,fill:#43418b,color:#ffffff
		playbook_a27d9bec --> |"3"| play_4d3ee472
		linkStyle 24 stroke:#43418b,color:#43418b
			%% Start of the role '[role] nested_include_role'
			play_4d3ee472 --> |"1"| role_f611e648
			linkStyle 25 stroke:#43418b,color:#43418b
			role_f611e648(["[role] nested_include_role"])
			style role_f611e648 fill:#43418b,color:#ffffff,stroke:#43418b
				task_8d2e4414["[task] nested_include_role : Ensure postgresql is at the latest version"]
				style task_8d2e4414 stroke:#43418b,fill:#ffffff
				role_f611e648 --> |"1"| task_8d2e4414
				linkStyle 26 stroke:#43418b,color:#43418b
				task_d1bc52f0["[task] nested_include_role : Ensure that postgresql is started"]
				style task_d1bc52f0 stroke:#43418b,fill:#ffffff
				role_f611e648 --> |"2"| task_d1bc52f0
				linkStyle 27 stroke:#43418b,color:#43418b
				%% Start of the role '[role] display_some_facts'
				role_f611e648 --> |"3 [when: x is not defined]"| role_39ad2981
				linkStyle 28 stroke:#43418b,color:#43418b
				role_39ad2981(["[role] display_some_facts"])
				style role_39ad2981 fill:#43418b,color:#ffffff,stroke:#43418b
					task_110062e7["[task] display_some_facts : ansible_architecture"]
					style task_110062e7 stroke:#43418b,fill:#ffffff
					role_39ad2981 --> |"1"| task_110062e7
					linkStyle 29 stroke:#43418b,color:#43418b
					task_05309d40["[task] display_some_facts : ansible_date_time"]
					style task_05309d40 stroke:#43418b,fill:#ffffff
					role_39ad2981 --> |"2"| task_05309d40
					linkStyle 30 stroke:#43418b,color:#43418b
					task_d8106118["[task] display_some_facts : Specific included task for Debian"]
					style task_d8106118 stroke:#43418b,fill:#ffffff
					role_39ad2981 --> |"3"| task_d8106118
					linkStyle 31 stroke:#43418b,color:#43418b
				%% End of the role '[role] display_some_facts'
				%% Start of the role '[role] fake_role'
				role_f611e648 --> |"4"| role_60085133
				linkStyle 32 stroke:#43418b,color:#43418b
				role_60085133(["[role] fake_role"])
				style role_60085133 fill:#43418b,color:#ffffff,stroke:#43418b
					task_aa202401["[task] fake_role : Debug 1"]
					style task_aa202401 stroke:#43418b,fill:#ffffff
					role_60085133 --> |"1"| task_aa202401
					linkStyle 33 stroke:#43418b,color:#43418b
					task_bb8335d6["[task] fake_role : Debug 2"]
					style task_bb8335d6 stroke:#43418b,fill:#ffffff
					role_60085133 --> |"2"| task_bb8335d6
					linkStyle 34 stroke:#43418b,color:#43418b
					task_7e0c8ed3["[task] fake_role : Debug 3 with double quote "here" in the name"]
					style task_7e0c8ed3 stroke:#43418b,fill:#ffffff
					role_60085133 --> |"3"| task_7e0c8ed3
					linkStyle 35 stroke:#43418b,color:#43418b
				%% End of the role '[role] fake_role'
			%% End of the role '[role] nested_include_role'
			%% Start of the role '[role] display_some_facts'
			play_4d3ee472 --> |"2"| role_ff23a4e9
			linkStyle 36 stroke:#43418b,color:#43418b
			role_ff23a4e9(["[role] display_some_facts"])
			style role_ff23a4e9 fill:#43418b,color:#ffffff,stroke:#43418b
				task_8fc8f4bc["[task] display_some_facts : ansible_architecture"]
				style task_8fc8f4bc stroke:#43418b,fill:#ffffff
				role_ff23a4e9 --> |"1"| task_8fc8f4bc
				linkStyle 37 stroke:#43418b,color:#43418b
				task_6a9bc407["[task] display_some_facts : ansible_date_time"]
				style task_6a9bc407 stroke:#43418b,fill:#ffffff
				role_ff23a4e9 --> |"2"| task_6a9bc407
				linkStyle 38 stroke:#43418b,color:#43418b
				task_b6121d94["[task] display_some_facts : Specific included task for Debian"]
				style task_b6121d94 stroke:#43418b,fill:#ffffff
				role_ff23a4e9 --> |"3"| task_b6121d94
				linkStyle 39 stroke:#43418b,color:#43418b
			%% End of the role '[role] display_some_facts'
		%% End of the play 'Play: webserver (0)'
	%% End of the playbook 'tests/fixtures/multi-plays.yml'

Loading

Example CLI

ara playbook graph 16 -f graphviz

References

• ansible-playbook-grapher

[dmsimard]

Hi @Hipska and thanks for the issue.

I think ansible-playbook-grapher is great and fits in the definition of making playbooks easier to understand and troubleshoot :p

I remember we discussed it briefly a while back, and I pondered on that for a wihle so thanks for reminding me about it.

This is a bit verbose but here's my train of thought:

It would be useful to have a graph of the playbook attached to the report somehow.
However, I am not interested in adding a dependency on ansible-playbook-grapher as it is today, it pulls in too much stuff (including ansible-core!), sorry:

I won't go into all the details here but ara used to have a dependency on ansible back in version 0.x and it caused a lot of headaches. Technical and legal ones.

So how might we allow users to have playbook graphs in the UI without depending on ansible-playbook-grapher ?

I considered different options and feel like I have settled on an idea worth discussing: using ara_record.
ara_record is a generic way to attach any combination of key/values, for example recording the git commit sha from where the playbook is running:

We don't do much with the type of a record in the UI right now, but it is intended as a way for the consumer (i.e, the UI, the CLI) to know how to handle/parse/render the value of a key.
For example if the type is json or dict, we might pretty-print it or if the type is an url, it would be a link you could click on.

What if we had a graph type ? ansible-playbook-grapher outputs as an image (svg), a mermaid flowchart (mmd) and .json.

The raw json means we'd do our own rendering somehow and the mermaid flowchart requires mermaid.js (example in django here).
The svg is just an image, so then perhaps we should have an image type instead ?

That could be interesting? A generic way of attaching images (or screenshots?) to playbook reports.

It could look like this (pseudocode):

$ ansible-playbook-grapher -o playbook playbook.yml
$ export ARA_CALLBACK_PLUGINS=$(ansible.setup.callback_plugins)
$ ansible-playbook -i hosts playbook.yml

# playbook.yml
- name: Include playbook graph in the report
  ara_record:
    key: playbook-graph
    value: "{{ playbook_dir }}/playbook.svg"
    type: image

Then, in the UI, the user could see the image attached to the playbook. In this case it's a playbook graph but it could be any image, even images of cats.

This supposes we want to store images in the database in order to keep everything easily portable, but it's not necessarily a great idea. Otherwise they would be uploaded via django and stored on disk which means if you transfer your sqlite database somewhere, the attachments won't follow unless you also transfer them over. It is a bit more complicated.

It is not exactly what you have asked for but we could test what it looks like.

[Hipska]

I agree with wanting to keep ara package small and as less dependencies as possible. However, this surely counts for the Ansible plugins part (ara), but IMHO less for the server package (ara[server]). When putting this burden onto the controller node, You basically putting the extra overhead on each of the controllers, while it would be only once needed on the ara server. (Don't know if optional dependencies are possible?)

On the other hand, I also like the idea of putting this info into records and be more smart on how to handle them in the UI. I did not use them yet, but will look into it.