Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

seiso should not raise error if configuration has already been deleted #42

Closed
ghost opened this issue Nov 13, 2020 · 5 comments · Fixed by #43
Closed

seiso should not raise error if configuration has already been deleted #42

ghost opened this issue Nov 13, 2020 · 5 comments · Fixed by #43

Comments

@ghost
Copy link

ghost commented Nov 13, 2020

Hello,

Some times, seiso raises this error when running with Gitlab CI and docker image appuio/oc:v3.11:

$ seiso configmaps -l app="${APPLICATION_NAME}" --older-than "${SEISO_OLDER_THAN}" --keep "${SEISO_KEEP_AT_MOST}" --delete
level=info msg="Seiso 0.7.0, commit a728ac2c7ea214bf9af854d4440d45528c13a5f6, date 2020-09-23T07:25:49Z"
Error: could not delete ConfigMaps for 'xxxx': configmaps "xxx-yyy-9dkg8m2bgh" not found

I think seiso could raise a Warning in this case but not an error, as the goal is to delete a config and it seems it's already deleted.

@ccremer
Copy link
Contributor

ccremer commented Nov 13, 2020

Hi.

That's interesting, because we are getting the list of candidates directly from the cluster. It seems like at the time where we are querying the cluster and deleting it after filtering, the candidate has been already deleted.... This looks like a race condition.

The fix should be easy, but how exactly do you use Seiso? what is the value of SEISO_OLDER_THAN and SEISO_KEEP_AT_MOST? Do you call seiso immediately after doing a deployment?

@ghost
Copy link
Author

ghost commented Nov 13, 2020

Depending on which environment is targeted, we've 3 different pair of values.

When I had this error, it was with the "development" environment configured with:

SEISO_KEEP_AT_MOST: 2
SEISO_OLDER_THAN: "3d"

seiso is run before any new deployment is run, but indeed, we can have some race condition with our integration and/or production environment launch the "seiso + deploy" job.

Maybe we should only run seiso for the development environment ?

Our gitlab-ci was inspired by the one provided by the appuio's example.

@ccremer
Copy link
Contributor

ccremer commented Nov 13, 2020

Hm strange, if it runs before the new deployment, I cannot image another reason why there's a race condition without having a closer look into your environment.

In any case, if you are able to wait a bit, the linked PR should ignore not-found-errors (but it will still be printed/logged as "Deleted"). That should remove the need to adapt your pipeline ™️

@ghost
Copy link
Author

ghost commented Nov 13, 2020

Cool, many thanks for the PR :-)

@ccremer
Copy link
Contributor

ccremer commented Nov 17, 2020

We have released 0.7.1 now, please check if that works for you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging a pull request may close this issue.

1 participant