[Fix] #1242 bug report broken tests for connect four in ci (#1243) #1139

Job	Run time
agilerl-tutorial-test (3.9, AgileRL)	14m 50s
agilerl-tutorial-test (3.10, AgileRL)	16m 12s
agilerl-tutorial-test (3.11, AgileRL)	15m 19s
tutorial-test (3.8, Tianshou)	3m 19s
tutorial-test (3.8, CustomEnvironment)	29s
tutorial-test (3.8, CleanRL)	5m 27s
tutorial-test (3.8, SB3/kaz)	4m 17s
tutorial-test (3.8, SB3/waterworld)	9m 17s
tutorial-test (3.8, SB3/test)	2m 31s
tutorial-test (3.9, Tianshou)	3m 34s
tutorial-test (3.9, CustomEnvironment)	31s
tutorial-test (3.9, CleanRL)	5m 34s
tutorial-test (3.9, SB3/kaz)	4m 48s
tutorial-test (3.9, SB3/waterworld)	7m 55s
tutorial-test (3.9, SB3/test)	2m 5s
tutorial-test (3.10, Tianshou)	3m 38s
tutorial-test (3.10, CustomEnvironment)	35s
tutorial-test (3.10, CleanRL)	7m 4s
tutorial-test (3.10, SB3/kaz)	4m 42s
tutorial-test (3.10, SB3/waterworld)	8m 32s
tutorial-test (3.10, SB3/test)	2m 3s
tutorial-test (3.11, Tianshou)	3m 25s
tutorial-test (3.11, CustomEnvironment)	27s
tutorial-test (3.11, CleanRL)	7m 10s
tutorial-test (3.11, SB3/kaz)	4m 47s
tutorial-test (3.11, SB3/waterworld)	7m 44s
tutorial-test (3.11, SB3/test)	2m 1s
	2h 28m 16s

Provide feedback