Skip to content

[RLlib] Create resource bundle per learner #59620

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
simonsays1980 merged 6 commits into from
Jan 7, 2026
+18 −25
Merged

[RLlib] Create resource bundle per learner #59620

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
fccebe7
create resource bundle per learner
kamil-kaczmarek Dec 23, 2025
50810c6
Update rllib/algorithms/utils.py
kamil-kaczmarek Dec 23, 2025
b5c783b
Merge branch 'master' into kk/fix-learner-bundles
kamil-kaczmarek Dec 23, 2025
48e4405
Merge branch 'master' into kk/fix-learner-bundles
kamil-kaczmarek Dec 27, 2025
0023e50
Make num_cpus_per_learner more easily readable
Jan 7, 2026
12d7714
Fix `num_cpus_per_learner` from `num_cpus_per_learners` (thanks Simon…
Jan 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 18 additions & 25 deletions rllib/algorithms/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,47 +179,40 @@ def _get_offline_eval_runner_bundles(config):


def _get_learner_bundles(config):
try:
from ray.rllib.extensions.algorithm_utils import _get_learner_bundles as func

return func(config)
except Exception:
pass

if config.num_learners == 0:
if config.num_aggregator_actors_per_learner > 0:
return [{"CPU": config.num_aggregator_actors_per_learner}]
return [{"CPU": 1} for _ in range(config.num_aggregator_actors_per_learner)]
else:
return []

num_cpus_per_learner = (
config.num_cpus_per_learner
if config.num_cpus_per_learner != "auto"
else 1
if config.num_gpus_per_learner == 0
else 0
)
if config.num_cpus_per_learner != "auto":
num_cpus_per_learner = config.num_cpus_per_learner
elif config.num_gpus_per_learner == 0:
num_cpus_per_learner = 1
else:
num_cpus_per_learner = 0

# aggregator actors are co-located with learners and use 1 CPU each
bundles = [
{
"CPU": config.num_learners
* (num_cpus_per_learner + config.num_aggregator_actors_per_learner),
"GPU": config.num_learners * config.num_gpus_per_learner,
"CPU": num_cpus_per_learner + config.num_aggregator_actors_per_learner,
"GPU": config.num_gpus_per_learner,
}
for _ in range(config.num_learners)
Copy link
Contributor

@simonsays1980 simonsays1980 Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks more correct than before, but I am wondering, if this would still not ensure that AggregatorActors being scheduled on the same node as we do not use placement groups. Could theoretically an EnvRunner be scheduled on CPUs of the same node instead of an AggregatorActor?

]

return bundles


def _get_main_process_bundle(config):
if config.num_learners == 0:
num_cpus_per_learner = (
config.num_cpus_per_learner
if config.num_cpus_per_learner != "auto"
else 1
if config.num_gpus_per_learner == 0
else 0
)
if config.num_cpus_per_learner != "auto":
num_cpus_per_learner = config.num_cpus_per_learner
elif config.num_gpus_per_learner == 0:
num_cpus_per_learner = 1
else:
num_cpus_per_learner = 0

bundle = {
"CPU": max(num_cpus_per_learner, config.num_cpus_for_main_process),
"GPU": config.num_gpus_per_learner,
Expand Down