[core] Get cloud provider with ray on kubernetes by dayshah · Pull Request #51793 · ray-project/ray

dayshah · 2025-03-28T19:14:36Z

Why are these changes needed?

On GKE

dhyey@cloudshell:~ (dhyey-dev)$ kubectl exec -it $HEAD_POD -- python -c "import requests; print(requests.get('http://metadata.google.internal/computeMetadata/v1',headers={'Metadata-Flavor': 'Google'}))"
Defaulted container "ray-head" out of: ray-head, autoscaler
<Response [200]>
dhyey@cloudshell:~ (dhyey-dev)$ kubectl exec -it $HEAD_POD -- python -c "import requests; print(requests.get('http://169.254.169.254/latest/meta-data/'))"                                                                                       
Defaulted container "ray-head" out of: ray-head, autoscaler
<Response [404]>
dhyey@cloudshell:~ (dhyey-dev)$ kubectl exec -it $HEAD_POD -- python -c "import requests; print(requests.get('http://169.254.169.254/metadata/instance?api-version=2021-02-01'))"                                                                
Defaulted container "ray-head" out of: ray-head, autoscaler
<Response [404]>

On anyscale on eks (google metadata req results in ConnectionError)

>>> print(requests.get('http://169.254.169.254/latest/meta-data/'))
<Response [200]>
>>> print(requests.get('http://169.254.169.254/metadata/instance?api-version=2021-02-01'))
<Response [404]>

Note: Untested on azure

Related issue number

Checks

I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
- I've added any new APIs to the API Reference. For example, if I added a
  method in Tune, I've added it in doc/source/tune/api/ under the
  corresponding .rst file.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
- Release tests
- This PR is not tested :(

Signed-off-by: dayshah <dhyey2019@gmail.com>

jjyao · 2025-03-28T21:08:41Z

    max_workers: Optional[int] = None
    head_node_instance_type: Optional[str] = None
    worker_node_instance_types: Optional[List[str]] = None
+    cloud_provider_alt: Optional[str] = None


we cannot change the schema here without changing the server since server does the schema validation. Lets discuss offline how to change the schema.

updated with added field to UsageStatsToReport is that all?

and updated schema in test

updated based on discussion

Signed-off-by: dayshah <dhyey2019@gmail.com>

jjyao · 2025-04-02T05:29:12Z

+
+        import requests
+
+        def cloud_metadata_request(


Can we check if we are in GKE, EKS, AKS specifically since theoretically user can also install k8s on their ec2 machines.

I couldn't find any way of checking that from inside the ray container, @kevin85421 do you know if the kuberay-operator has any context?

talked offline, don't know of any way

If users install k8s on EC2 machines, they might use tools like kind. And I think the kind cluster environment is isolated from the host machine, so we can't detect which cloud provider we're in from inside the kind cluster.

I think you can look for some eks metadata tags like:

curl -s http://169.254.169.254/latest/meta-data/tags/instance/eks:cluster-name

unlikely to be perfect though.

Signed-off-by: dayshah <dhyey2019@gmail.com>

jjyao · 2025-04-15T05:14:59Z

+                if res.status_code != 404:
+                    if result.cloud_provider is None:
+                        result.cloud_provider = cloud_provider
+                    else:
+                        result.cloud_provider += f"_{cloud_provider}"
+                    return True


Is it true for those endpoints that as long as it's reachable, you are running on it?

yes, the endpoint isn't reachable otherwise

then we don't need to check 404? 404 means it's still reachable?

misunderstanding on my side, we can get a 404 when requesting from other clouds. You'll never get a 404 if the request is from that cloud. Can see example in description

Signed-off-by: dayshah <dhyey2019@gmail.com>

edoakes

please add tests by faking requests library

edoakes · 2025-04-22T18:27:26Z

+    else:
+        return "unknown"


we should wrap all the above checks in an unexpected exception handlers and return "unknown" in those cases (and log the unexpected exception)

or catch all exceptions inside cloud_metadata_request

done, catching in cloud_metadata_request and logging info

edoakes · 2025-04-22T18:28:49Z

+        else:
+            result.cloud_provider += f"_${get_cloud_from_metadata_requests()}"


what's this intending to do?

added a comment above
it's for if kubernetes was set already
then you want the cloud provider to be kubernetes_aws

github-actions · 2025-06-07T00:33:07Z

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

github-actions · 2025-06-23T00:34:43Z

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

Signed-off-by: dayshah <dhyey2019@gmail.com>

dayshah · 2025-07-03T07:30:18Z

please add tests by faking requests library

vibcoded some tests like the cool kids 😎

edoakes · 2025-07-03T13:41:01Z

Minimal tests failing: https://buildkite.com/ray-project/premerge/builds/43048#0197cf4d-ce0a-4cdc-84af-f7cb572bd370/188-1524

Signed-off-by: dayshah <dhyey2019@gmail.com>

dayshah · 2025-07-03T20:07:32Z

Minimal tests failing: https://buildkite.com/ray-project/premerge/builds/43048#0197cf4d-ce0a-4cdc-84af-f7cb572bd370/188-1524

fixed @edoakes

On GKE ``` dhyey@cloudshell:~ (dhyey-dev)$ kubectl exec -it $HEAD_POD -- python -c "import requests; print(requests.get('http://metadata.google.internal/computeMetadata/v1',headers={'Metadata-Flavor': 'Google'}))" Defaulted container "ray-head" out of: ray-head, autoscaler <Response [200]> dhyey@cloudshell:~ (dhyey-dev)$ kubectl exec -it $HEAD_POD -- python -c "import requests; print(requests.get('http://169.254.169.254/latest/meta-data/'))" Defaulted container "ray-head" out of: ray-head, autoscaler <Response [404]> dhyey@cloudshell:~ (dhyey-dev)$ kubectl exec -it $HEAD_POD -- python -c "import requests; print(requests.get('http://169.254.169.254/metadata/instance?api-version=2021-02-01'))" Defaulted container "ray-head" out of: ray-head, autoscaler <Response [404]> ``` On anyscale on eks (google metadata req results in ConnectionError) ``` >>> print(requests.get('http://169.254.169.254/latest/meta-data/')) <Response [200]> >>> print(requests.get('http://169.254.169.254/metadata/instance?api-version=2021-02-01')) <Response [404]> ``` Note: Untested on azure --------- Signed-off-by: dayshah <dhyey2019@gmail.com> Signed-off-by: ChanChan Mao <chanchanmao1130@gmail.com>

On GKE ``` dhyey@cloudshell:~ (dhyey-dev)$ kubectl exec -it $HEAD_POD -- python -c "import requests; print(requests.get('http://metadata.google.internal/computeMetadata/v1',headers={'Metadata-Flavor': 'Google'}))" Defaulted container "ray-head" out of: ray-head, autoscaler <Response [200]> dhyey@cloudshell:~ (dhyey-dev)$ kubectl exec -it $HEAD_POD -- python -c "import requests; print(requests.get('http://169.254.169.254/latest/meta-data/'))" Defaulted container "ray-head" out of: ray-head, autoscaler <Response [404]> dhyey@cloudshell:~ (dhyey-dev)$ kubectl exec -it $HEAD_POD -- python -c "import requests; print(requests.get('http://169.254.169.254/metadata/instance?api-version=2021-02-01'))" Defaulted container "ray-head" out of: ray-head, autoscaler <Response [404]> ``` On anyscale on eks (google metadata req results in ConnectionError) ``` >>> print(requests.get('http://169.254.169.254/latest/meta-data/')) <Response [200]> >>> print(requests.get('http://169.254.169.254/metadata/instance?api-version=2021-02-01')) <Response [404]> ``` Note: Untested on azure --------- Signed-off-by: dayshah <dhyey2019@gmail.com> Signed-off-by: Douglas Strodtman <douglas@anyscale.com>

Get cloud provider with ray on kubernetes

1fa96aa

Signed-off-by: dayshah <dhyey2019@gmail.com>

dayshah added go add ONLY when ready to merge, run all tests and removed go add ONLY when ready to merge, run all tests labels Mar 28, 2025

update

deac06d

Signed-off-by: dayshah <dhyey2019@gmail.com>

dayshah assigned jjyao Mar 28, 2025

dayshah added the go add ONLY when ready to merge, run all tests label Mar 28, 2025

dayshah marked this pull request as ready for review March 28, 2025 21:07

dayshah requested review from edoakes, jjyao and richardliaw as code owners March 28, 2025 21:07

jjyao reviewed Mar 28, 2025

View reviewed changes

Comment thread python/ray/_private/usage/usage_lib.py Outdated

dayshah added 4 commits March 28, 2025 14:14

update UsageStatsToReport

9446347

Signed-off-by: dayshah <dhyey2019@gmail.com>

update schema in test

4c486e4

Signed-off-by: dayshah <dhyey2019@gmail.com>

elif and fix test

bf42337

Signed-off-by: dayshah <dhyey2019@gmail.com>

fix

4efe777

Signed-off-by: dayshah <dhyey2019@gmail.com>

dayshah requested review from MortalHappiness and jjyao March 29, 2025 20:39

dayshah added 3 commits April 1, 2025 12:59

update based on discussion

42501d3

Signed-off-by: dayshah <dhyey2019@gmail.com>

unknown

eab2f0a

Signed-off-by: dayshah <dhyey2019@gmail.com>

Merge branch 'master' into usage-cloud-provider

18b39a6

jjyao reviewed Apr 2, 2025

View reviewed changes

fix test

21a3142

Signed-off-by: dayshah <dhyey2019@gmail.com>

dayshah requested a review from jjyao April 4, 2025 20:38

jjyao reviewed Apr 15, 2025

View reviewed changes

pure function

0682e58

Signed-off-by: dayshah <dhyey2019@gmail.com>

dayshah requested a review from jjyao April 15, 2025 05:39

Merge branch 'master' into usage-cloud-provider

025673a

edoakes reviewed Apr 22, 2025

View reviewed changes

mascharkh added the community-contribution Contributed by the community label May 15, 2025

mascharkh removed the community-contribution Contributed by the community label May 15, 2025

hainesmichaelc added community-backlog and removed community-backlog labels May 22, 2025

github-actions Bot added the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Jun 7, 2025

dayshah removed the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Jun 7, 2025

github-actions Bot added the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Jun 23, 2025

dayshah removed the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Jun 23, 2025

dayshah added 4 commits July 2, 2025 22:56

address comments

2803f73

Signed-off-by: dayshah <dhyey2019@gmail.com>

catch all and log

6a437cb

Signed-off-by: dayshah <dhyey2019@gmail.com>

Merge branch 'master' into usage-cloud-provider

e0a11cb

Signed-off-by: dayshah <dhyey2019@gmail.com>

vibecodin

e27f3e3

Signed-off-by: dayshah <dhyey2019@gmail.com>

dayshah requested a review from edoakes July 3, 2025 07:31

edoakes approved these changes Jul 3, 2025

View reviewed changes

no default param

8ba44c2

Signed-off-by: dayshah <dhyey2019@gmail.com>

edoakes merged commit 348063f into ray-project:master Jul 7, 2025
5 checks passed

dayshah deleted the usage-cloud-provider branch July 7, 2025 16:47

		else:
		result.cloud_provider += f"_${get_cloud_from_metadata_requests()}"

Conversation

dayshah commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are these changes needed?

Related issue number

Checks

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

edoakes left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jun 7, 2025

Uh oh!

github-actions Bot commented Jun 23, 2025

Uh oh!

dayshah commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

edoakes commented Jul 3, 2025

Uh oh!

dayshah commented Jul 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

dayshah commented Mar 28, 2025 •

edited

Loading

dayshah commented Jul 3, 2025 •

edited

Loading