Trigger / schedule workflow to create issues if we are missing schemas for published versions

[ogenstad]

During the hackathon I created a small snippet that compares published Infrahub Docker images against the json schema versions we've published for each release. The script only prints output to the screen, currently it would list major and minor versions above version 1.0 and indicate if the schema is published or not.

The script looks like this:

import httpx
from packaging.version import InvalidVersion, Version

page = 1
page_size = 10
site = "https://registry.opsmill.io"
endpoint = "/api/v2.0/projects/opsmill/repositories/infrahub/artifacts"
query_string = "?with_tag=true&with_label=false&with_scan_overview=false&with_signature=false&with_immutable_status=false&with_accessory=false"

client = httpx.Client()

all_entries = []
missing_results = True
while missing_results:
    pagination = f"&page={page}&page_size={page_size}"
    url = f"{site}{endpoint}{query_string}{pagination}"
    res = client.get(url)
    assert res.status_code == 200
    if entries := res.json():
        all_entries.extend(entries)
    else:
        missing_results = False
    page += 1

artifact_tags = [entry["tags"] for entry in all_entries]

all_tags = []
for artifact in artifact_tags:
    for tag in artifact:
        all_tags.append(tag)

tag_names = [tag["name"] for tag in all_tags]

valid_semver = []

for entry in tag_names:
    try:
        version = Version(entry)

        if version.major >= 1 and not version.pre and not version.dev and not version.local and version.micro == 0:
            valid_semver.append(entry)
    except InvalidVersion:
        pass

existing_versions = []
missing_versions = []
for version in valid_semver:
    result = client.get(f"https://schema.infrahub.app/infrahub/schema/{version}.json")
    if result.status_code == 200:
        existing_versions.append(f"{version} exists")
    elif result.status_code == 404:
        missing_versions.append(f"{version} is missing")

print("Versions where the schema exists")
for existing in existing_versions:
    print(existing)

print("Versions where the schema is missing")
for missing in missing_versions:
    print(missing)

We could have a scheduled task that runs this that would create issues in this repository if we are missing any of the schemas. Alternatively trigger the execution of a pipeline in this repo to run whenever we publish a new docker image.

Running this script today gives this output:

Versions where the schema exists
1.0.0 exists
Versions where the schema is missing
1.0 is missing
1.1 is missing
1.1.0 is missing

Here we can choose if we should publish a schema both for 1.0.0 and 1.0 or simply ignore the 1.0 version. I also note that we have created images for 1.1.0 even though that version has yet to be released.