[Bug][github_graphql] Incorrect dora calculation based on github deployments updated_at

[sungkangaw]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

We currently sync deployment data using github deployments. When the deployment is the latest deployment (active), there is no problem. But when syncing historical deployment data i.e. projects have run for some time and we are just setting up devlake project for it or recollect data in full refresh mode, we notice that incidents are not matched to the right deployments, lead time is not correct, consequently then also change failure rate and recovery time.

After looking through the code, we found out for github deployments, finished_date (which is the basis of all dora calculation) is mapped to github deployment updated_at here

incubator-devlake/backend/plugins/github_graphql/tasks/deployment_convertor.go

Line 93 in e061ef2

FinishedDate: &githubDeployment.UpdatedDate,

For latest deployment this is no issue.
But for old deployment, the updated_at can be the date that status is changed to inactive (changed when there is a new active deployment)

This result in wrong calculation of dora as updated_at is not the actual finished_date of the deployment.

What do you expect to happen

Correct calculation of dora.
We are not quite sure yet, but my proposal would be instead of using updated_at field of github deployment object. The graphql query should retrieve the last status where state = success and use updated_at of the status instead. This way the actual finished_date of when status is success is captured.
But please make a suggest if there is any better solution.

How to reproduce

1. Sync github deployments data. Make sure your repo have a few deployments already.
2. Check the finished_date of these deployments. It is usually around the time of the created_date of the next deployment and also check dora graph.

Anything else

No response

Version

v1.0.1

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[dosubot]

Hey @sungkangaw! 👋 I'm here to assist you with bug fixes, answer questions, and guide you on becoming a contributor. Let's squash those bugs together!

The issue you're experiencing with the DORA calculations is due to the finished_date being incorrectly mapped to the updated_at timestamp of GitHub deployments. This approach works for the latest deployments but causes inaccuracies for historical deployments, where updated_at may not reflect the actual completion time.

To address this, your proposal to modify the GraphQL query to retrieve the last status where state = success and use its updated_at timestamp as the finished_date is a valid solution. This change would ensure that the finished_date accurately represents the time when the deployment was successfully completed, leading to more accurate DORA metrics calculations.

Currently, the finished_date is set to the UpdatedDate of the githubDeployment in the deployment_convertor.go file ^[1]. Adjusting the logic to use the updated_at of the last successful status would require changes in the GraphQL query and possibly the data processing logic to ensure the correct timestamp is used.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}