Security: Codecov Bash Uploader Vulnerability

[lkysow]

Today (Apr 15, 2021) Codecov, a code coverage tool, announced that they had a vulnerability in their bash uploader script that allowed an attacker to potentially view all secrets in the CI/CD environments of Codecov users (https://about.codecov.io/security-update/). This vulnerability was introduced January 31, 2021 and was remediated April 1, 2021.

Atlantis uses Codecov in its CI scripts. We have rotated potentially exposed tokens and if you were not using the :latest Docker image tag during the period from Apr 1 until today then you will not have been affected. Please read on for more details.

There are two secrets in our CI environment that could have been exposed:

1. The GitHub token we use to run integration tests. This token is used to create webhooks on https://github.com/runatlantis/atlantis-tests and then we start up Atlantis in CI and create test PRs on that repository. That token had admin:repo_hook permissions. If stolen, an attacker could have created repository webhooks or modified our GitHub action workflows for the runatlantis organization.

   We've audited our repositories and found no such changes. We have now rotated this secret.

2. The Docker hub token we use to publish runatlantis/atlantis Docker images to our Docker hub account. If stolen, an attacker could have published malicious Docker images to our Docker hub account or modified existing images. For example they could have replaced the atlantis binary with their own that logged credentials.

   We've confirmed that none of our stable images were modified. It is possible that the :latest image was modified. We have no way to confirm that this did or did not occur because the :latest image is continually updated whenever the master branch changes.

   We audited our Docker Hub images and confirmed that the only images that have been pushed during the Jan 31-today window are: runatlantis/atlantis:v0.16.1, runatlantis/atlantis:v0.17.0-beta and runatlantis/atlantis:latest. We've confirmed that both runatlantis/atlantis:v0.16.1 and runatlantis/atlantis:v0.17.0-beta have not been modified by comparing their image SHAs and publish dates with the ones from our build pipeline. runatlantis/atlantis:latest tracks our master branch and so it is continually being updated. We've confirmed that the current tag matches our master branch, however it is possible that at some point during the window, an attacker pushed an update to that image which was then later overridden by a new push to master. There is no way for us to know if that occurred because the previous builds of runatlantis/atlantis:latest are overridden on new builds and Docker Hub does not provide logs for pushes. If you were or are currently using runatlantis/atlantis:latest during the period from Jan 31-today then the safest action would be to ensure you're using the latest :latest image and then rotate all tokens that Atlantis has access to.

Next Steps:

1. We have modified our CI scripts to validate the checksum from Codecov.
2. We will be taking steps in our CI scripts to ensure that secrets needed by certain steps are not available to others.
3. We understand that we need to increase the security and validation of our image building and publishing process in general. We are investigating that now and will provide updates when we have decided on the best course of action.

[lkysow]

Update: we've refactored our build pipeline to ensure only certain steps have access to secrets and we've fully removed the codecov integration.