Track usage metrics

[huard]

We'll want a mechanism to track, monitor and aggregate usage to provide reports to our backers about the volume of data downloaded, the services used, the number of users, etc.

Specs:

• Daily aggregation
• When we receive the email for the DG report, export usage for the last three months to github repo (pavics-usage)
• Format CSV
• Header: date, egress, cpu, storage
• Units: YYYY-MM-DD, Gb, core-year, Gb

Bulk metrics (Mourad) -> /bulk

• Data egress (Gb leaving the platform)
• CPU usage (core-year)
• Data storage (Gb)

Container metrics (Long) -> /container

• Data egress (Gb leaving the platform)
• CPU usage (core-year)
• Data storage (Gb)

[tlvu]

From @huard to not lose in the sea of email:

Il y a plusieurs use case pour le monitoring de pavics:

Détecter des utilisations non-conformes (qqun qui tourne un bitcoin miner sur jupyterlab)

Fournir des stats d'utilisation pour le rapport du DG

• nombre de requêtes venant de l'interne (hors tests)
• nombre de requêtes venant de l'externe
• volume de données downloadé

On va aussi vouloir le nombre de download par trimestre de xclim.

[huard]

@diacha1 Pourrais-tu clarifier les attentes en terme de suivi de l'utilisation de la plateforme ?

[tlvu]

Container metrics (Long) -> /container

* Data egress (Gb leaving the platform)

* CPU usage (core-year)

* Data storage (Gb)

For me, you'll need to specify which containers or you want them all?

[huard]

Yes, we'll sort through them when creating the report.
So I suggest in the container case to have one per for each metric, with columns corresponding to containers.