metadrive agent policy mapping: agent_n more than assigned number of agents

[aabdelnaby]

When training using the metadrive environment, I get the following error:

  File "/data1/aaabdeln/miniconda3/envs/marllib/lib/python3.8/site-packages/ray/util/iter.py", line 1151, in par_iter_next
    return next(self.local_it)
  File "/data1/aaabdeln/miniconda3/envs/marllib/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 378, in gen_rollouts
    yield self.sample()
  File "/data1/aaabdeln/miniconda3/envs/marllib/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 767, in sample
    batch = self.input_reader.next()
  File "/data1/aaabdeln/miniconda3/envs/marllib/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 103, in next
    batches = [self.get_data()]
  File "/data1/aaabdeln/miniconda3/envs/marllib/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 233, in get_data
    item = next(self._env_runner)
  File "/data1/aaabdeln/miniconda3/envs/marllib/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 599, in _env_runner
    _process_observations(
  File "/data1/aaabdeln/miniconda3/envs/marllib/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 749, in _process_observations
    episode._add_agent_rewards(rewards[env_id])
  File "/data1/aaabdeln/miniconda3/envs/marllib/lib/python3.8/site-packages/ray/rllib/evaluation/episode.py", line 260, in _add_agent_rewards
    self.policy_for(agent_id)] += reward
  File "/data1/aaabdeln/miniconda3/envs/marllib/lib/python3.8/site-packages/ray/rllib/evaluation/episode.py", line 153, in policy_for
    self.policy_mapping_fn(agent_id)
  File "/data1/aaabdeln/desktop/MARLlib/marllib/marl/algos/run_il.py", line 115, in <lambda>
    lambda agent_id: policy_ids[agent_name_ls.index(agent_id)])
ValueError: 'agent_1' is not in list`

Here is the script I am running:

from marllib import marl
from ray import tune
    
if __name__ == '__main__':



    env = marl.make_env(environment_name="metadrive", map_name="Intersection",num_agents=1, traffic_density = 0.3)

    # # initialize algorithm and load hyperparameters
    ippo = marl.algos.ippo(hyperparam_source="fintuned", lr = tune.loguniform(1e-4, 1e-2))

    # # build agent model based on env + algorithms + user preference if checked available
    model = marl.build_model(env, ippo,  {"core_arch": "mlp", "fc_layer": 4, "hidden_state_size": 512, "out_dim_fc_0": 256, "out_dim_fc_1": 128, "out_dim_fc_2":64, "out_dim_fc_3":32})

 
        # start learning + extra experiment settings if needed. remember to check ray.yaml before use
    ippo.fit(env, 
                model,
                local_mode=False, 
                num_gpus=2, 
                checkpoint_freq=200, 
                num_workers=5, 
                stop={'timesteps_total': 1000000000000}, 
                share_policy = "individual",
                local_dir = "./exp_results/tmp"
                )

it seems that the policy is getting agents that are not registered in the environment. The same error occurs for any number of agents