separate out optimizer dryruns to its separate group

[SeungjinYang]

Optimizer Dryrun tests can be flaky (with flakiness lasting for up to several hours at a time). Usually the method to deal with such issues is to mark the specific optimizer test as not required temporarily and rerun all other failed jobs and merge the PR in.

The problem here is that once a single test in a group fails, all other tests in the group are automatically cancelled. This usually means rerunning all failed jobs do not help because one of the optimizer tests will fail again, causing rest of the tests to be cancelled. The current workaround is to rerun each test (excluding the failed optimizer tests) one at a time which is unnecessarily time consuming.

This PR separates the optimizer tests to a separate test group so other tests can be rerun independently of optimizer test failures.

Since this introduces some code duplication, I factor out the common steps into a separate action.

Tested (run the relevant ones):

• Code formatting: install pre-commit (auto-check on commit) or bash format.sh
• Any manual or new tests for this PR (please specify below)
• All smoke tests: /smoke-test (CI) or pytest tests/test_smoke.py (local)
• Relevant individual tests: /smoke-test -k test_name (CI) or pytest tests/test_smoke.py::test_name (local)
• Backward compatibility: /quicktest-core (CI) or pytest tests/smoke_tests/test_backward_compat.py (local)

[kevinmingtarja]

It seems like there's a recent merge conflict, could you take a look @SeungjinYang?

[zpoint]

LGTM, thanks