Skip to content

Conversation

@harshit-anyscale
Copy link
Contributor

Summary

This PR removes the RAY_SERVE_ALWAYS_RUN_PROXY_ON_HEAD_NODE environment variable as it has no practical effect in production.

Why this env var is redundant

The env var was intended to "always run a proxy on the head node even if it has no replicas." However, the controller already unconditionally ensures the head node is included in the proxy nodes set, making this flag ineffective.

Code flow analysis

In controller.py _update_proxy_nodes() (lines 419-430):

def _update_proxy_nodes(self):
    new_proxy_nodes = self.deployment_state_manager.get_active_node_ids()
    new_proxy_nodes = new_proxy_nodes - set(
        self.cluster_node_info_cache.get_draining_nodes()
    )
    new_proxy_nodes.add(self._controller_node_id)  # <-- UNCONDITIONAL
    self._proxy_nodes = new_proxy_nodes

The head node (_controller_node_id) is always added to _proxy_nodes unconditionally.

In proxy_state.py update() (where the env var was checked):

if RAY_SERVE_ALWAYS_RUN_PROXY_ON_HEAD_NODE:
    proxy_nodes.add(self._head_node_id)

This check happens after the controller already added the head node. Since proxy_nodes is a set, adding the same node again is a no-op.

Consequence

  • Setting RAY_SERVE_ALWAYS_RUN_PROXY_ON_HEAD_NODE=0 had no effect because the controller adds the head node regardless.
  • Setting RAY_SERVE_ALWAYS_RUN_PROXY_ON_HEAD_NODE=1 (the default) was also a no-op since the head node was already present.

Changes

  • Removed RAY_SERVE_ALWAYS_RUN_PROXY_ON_HEAD_NODE definition from constants.py
  • Removed the import and usage in proxy_state.py

@harshit-anyscale harshit-anyscale self-assigned this Jan 8, 2026
@harshit-anyscale harshit-anyscale requested a review from a team as a code owner January 8, 2026 05:54
@gemini-code-assist
Copy link
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@harshit-anyscale harshit-anyscale changed the title remove env var - ray serve proxy always on head node [4/n] remove env var - ray serve proxy always on head node Jan 8, 2026
Copy link
Contributor

@abrarsheikh abrarsheikh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lg2m. @akyang-anyscale please review

@ray-gardener ray-gardener bot added the serve Ray Serve Related Issue label Jan 8, 2026
@harshit-anyscale harshit-anyscale added the go add ONLY when ready to merge, run all tests label Jan 8, 2026
Copy link
Contributor

@akyang-anyscale akyang-anyscale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. looks like you may need to update the tests to pass in the head node id on .update(...)

@harshit-anyscale
Copy link
Contributor Author

lgtm. looks like you may need to update the tests to pass in the head node id on .update(...)

done, fixed it.

@abrarsheikh abrarsheikh merged commit c6b19e5 into master Jan 9, 2026
6 checks passed
@abrarsheikh abrarsheikh deleted the remove-env-var-always-on-head-node branch January 9, 2026 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

go add ONLY when ready to merge, run all tests serve Ray Serve Related Issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants