Running pre-trained AI

[Jack-McKee]

When I ran the pre-trained AI, the raw screenshots, and the lidar, they drove straight into the side wall repeatedly. The raw screenshots drove straight into the left wall and the lidar straight into the right wall for some reason. Also, It wouldn't let me switch the algorithm to REDQ or I would get an error from just updating the config file

[yannbouteiller]

Weird, are you using the tmrl-test track with the pre-trained policy? Or have you tried to train your own policy before testing?

For REDQ, I have not tested it in a while, we might have to double check that it still works (it requires a bunch of specific settings in config.json)

[Jack-McKee]

Also, I tried using the REDQSAC and ran into this error after updating just the config.json. Does this mean I need to update the config_objects.py also for it to use it?
PS C:\Users\Jack McKee> python -m tmrl --check-environment
Traceback (most recent call last):
File "", line 189, in _run_module_as_main
File "", line 148, in _get_module_details
File "", line 112, in get_module_details
File "C:\Users\Jack McKee\AppData\Local\Programs\Python\Python312\Lib\site-packages\tmrl_init.py", line 35, in
from tmrl.config.config_objects import CONFIG_DICT
File "C:\Users\Jack McKee\AppData\Local\Programs\Python\Python312\Lib\site-packages\tmrl\config\config_objects.py", line 38, in
assert ALG_NAME == "SAC", f"{ALG_NAME} is not implemented here."
AssertionError: REDQSAC is not implemented here.

[Jack-McKee]

I change the ALG in config to this
"ALG": {
"ALGORITHM": "REDQSAC",
"LEARN_ENTROPY_COEF":false,
"LR_ACTOR":0.0003,
"LR_CRITIC":0.00005,
"LR_ENTROPY":0.0003,
"GAMMA":0.995,
"POLYAK":0.995,
"TARGET_ENTROPY":-7.0,
"ALPHA":0.37,
"REDQ_N":10,
"REDQ_M":2,
"REDQ_Q_UPDATES_PER_POLICY_UPDATE":20
},

[yannbouteiller]

It says that the "Full" TrackMania environment example pipeline does not support REDQ-SAC at the moment (only the "LIDAR" example pipeline does).

It shouldn't be very hard to support REDQ-SAC in the Full pipeline though, since REDQ-SAC is basically SAC with more models. I'll look into this in the next couple days, but if you are proficient with python it should be easy.

[Jack-McKee]

I tested the SAC images to see if it would learn or take forever since I have an AMD GPU, but by training from scratch, it could finish the test map after 12 hours of training. My CPU sat at 100% the entire time though 😂.

[yannbouteiller]

This is because AMD GPUs are not supported and not used at all for training : the CPU is used instead.