You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While deploying xdsrelay in staging environment for a few critical services, we realized that the cache for eds had a drift.
The cache drift perhaps happened due to a bug. We need a way in xdsrelay to compare cache entried periodically and alarm when they dont match. The xdsrelay operators need a way to at least restart the server and reset the cache in order to mitigate when this happens.
There could always be bugs during initial launch, and such a method can keep the system safe. This feature should be based on a bootstrap config so that it can be enabled only when configured. It also needs a way to transform xdsrelay keys with upstream control plane keys.
The text was updated successfully, but these errors were encountered:
@jessicayuen We need some kind of monitor to make sure we alarm when cache drifts. We can make such a mechanism inhouse or think about this feature in the open from the outset, since this could be a helpful feature for operators to safely rollout the service. wdyt?
While deploying xdsrelay in staging environment for a few critical services, we realized that the cache for eds had a drift.
The cache drift perhaps happened due to a bug. We need a way in xdsrelay to compare cache entried periodically and alarm when they dont match. The xdsrelay operators need a way to at least restart the server and reset the cache in order to mitigate when this happens.
There could always be bugs during initial launch, and such a method can keep the system safe. This feature should be based on a bootstrap config so that it can be enabled only when configured. It also needs a way to transform xdsrelay keys with upstream control plane keys.
The text was updated successfully, but these errors were encountered: