CI: Drop workaround to pass pymatgen related failed tests #5045

Job	Run time
validate-dependency-specification	10s
install-with-conda (3.9, true)	49s
install-with-pip	22s
install-with-conda (3.10, true)	52s
install-with-pip ([atomic_tools,docs,notebook,rest,tests,tui])	1m 17s
install-with-conda (3.11, true)	43s
install-with-conda (3.12, true)	55s
install-with-conda (false)	47s
create-conda-environment	42s
tests (3.9)	7m 13s
tests (3.10)	7m 13s
tests (3.11)	7m 12s
tests (3.12)	7m 12s
	35m 27s

Provide feedback