Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added pod affinity if the default ReadWriteOnce is used. #13

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

keskival
Copy link

If accessMode ReadWriteOnce is used, then the volume can only be accessed in a write mode from a single node. This means that all pods which require write access to the volume need to be co-located to the same node. Failing that typically leads to a difficult to untangle locked volume mount situation where containers are stuck in ContainerCreating.

This change adds the necessary pod affinities if the related access modes are set in the values.yaml.

@keskival
Copy link
Author

keskival commented Dec 10, 2022

I am in progress of testing this, putting it here already for transparency.
Note that this relates to the issue here: #17

@SISheogorath
Copy link
Contributor

We literally just removed this in order to reduce unexpected affinities by just selecting a different volume access mode.

I guess the question becomes towards what end of the spectrum we want tonstart optimising the chart. Do we want to keep it generic and unopinionated or move towards an ideal, predefined setup?

See this conversation: mastodon/mastodon#20733 (comment)

@keskival keskival force-pushed the mastodon/22095/pod-affinity-for-readwriteonce branch from e51d535 to 9a54181 Compare December 10, 2022 19:26
@keskival
Copy link
Author

keskival commented Dec 10, 2022

Ok fair enough, but the default configuration isn't functional, and the jobs have analogous affinities set up.

In any case, this has been tested as far as template generation goes and produces templates in the expected form.

If ReadWriteOnce is used, not having a pod affinity would be an error which causes a catastrophic, random lock up.

I am fine with adding a visible warning to the values.yaml about this issue as well.

This is now confirmed working in my own cluster as well. Previously the defaults caused it to become locked up in a difficult to understand manner, which is what this PR fixes.

Added S3 conditioned disabling as well.

@keskival keskival force-pushed the mastodon/22095/pod-affinity-for-readwriteonce branch 3 times, most recently from b20acd7 to 2f3fe27 Compare December 10, 2022 23:36
If `accessMode` `ReadWriteOnce` is used, then the volume
can only be accessed in a write mode from a single node.
This means that all pods which require write access to
the volume need to be co-located to the same node.
Failing that typically leads to a difficult to untangle
locked volume mount situation where containers are stuck
in `ContainerCreating`.

This change adds the necessary pod affinities if the related
access modes are set in the `values.yaml`.
@keskival keskival force-pushed the mastodon/22095/pod-affinity-for-readwriteonce branch from 2f3fe27 to 682a5a4 Compare December 11, 2022 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants