Describe the bug

Flyte Propeller Followers/Replicas that have not won the leader election enqueue workflows but do not process them. This leads to a controllers memory leak and likely fills the queue with many stale workflows for if/when the follower/replica does become the leader. This current behavior isn't very confidence inspiring for a hot standby.

Below is an image of the queue depth increasing on the follower.

Expected behavior

I would expect that the follower/replica wouldn't enqueue workflows or would process them but throw them away.

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes