[Feature] Macro-actions for LLMs

[vmoens]

Stack from ghstack (oldest at bottom):

• [Tutorial] LLM integration #2832
• -> [Feature] Macro-actions for LLMs #2831
• [Feature] vllm wrapper #2830
• [Feature] transformers policy #2825
• [Feature] batch_size, reward, done, attention_key in LLMEnv #2824
• [Feature] DensifyReward postproc #2823
• [Feature] DataLoadingPrimer.repeat #2822

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2831

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 18 New Failures

As of commit 151612a with merge base 413571b ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 2.
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 2.
• Generate documentation / build-docs (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 9e4289c0bc51192be1a42d87e3e99a41ac9616f0957410eb64d6476abbd17c09 /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 0b8f33e6dc46abf23a3867ea00473587d8901922bb330b18f1d3834de9d05049 /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 33dad777508d04d6a26996b03bc1de4a9121f8eae5572466870778ccaf108543 /exec failed with exit code 1
• Libs Tests on Linux / unittests-sklearn (3.10, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 2ec21816acbdb7ff68565daadb112e2dba8bc9f730af492ad2e7059b57afc2c9 /exec failed with exit code 2
• Lint / python-source-and-configs / linux-job (gh)
  RuntimeError: Command docker exec -t 1d57d45cda9ac7107201239d40717e1c77acb205192ea446803da6f662a87364 /exec failed with exit code 1
• RLHF Tests on Linux / unittests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 0cea5356be972cc1009e66db4fbfd96985b48944c10a940a948d7ef9bc16ce3c /exec failed with exit code 2
• SOTA Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t ba4f4fd307401cc869708ae6d6878e9bed498e37b1aee642e20a08e08f7f9a4a /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  RuntimeError: Command docker exec -t c8bfba18b137af03d898cf7206392a03f637ec4e236c0db68ba105727e6ae36c /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  RuntimeError: Command docker exec -t 1411ad3bb48575907128365eafb2838beb87db8528666bb55578f7e9e5c7a109 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  RuntimeError: Command docker exec -t 563feb41b97a9c5bcd1d1018f1b481b9facfae0087c3ad91887be4a50ca5b8bc /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  RuntimeError: Command docker exec -t f158224fecc1ce8b0c8d66f78f6a9f5d034fb21cbf608d5b6cdf949a84baeb06 /exec failed with exit code 1
• Unit-tests on Linux / tests-gpu (3.11, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 905b649c0e2abbe8b20cbaf6582b6e0fc014771748fd3752441ec6b708977fc4 /exec failed with exit code 1
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 74f549461628120366cf2765a5bf76e7f14bd8880f9f6fd2c8a9488792ef62e1 /exec failed with exit code 1
• Unit-tests on Linux / tests-optdeps (3.11, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 48ef6080c0f8585639fc810b67b6ff8df3e51bfacd7027bd43e9bc96ed84fbb1 /exec failed with exit code 2
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  RuntimeError: Command docker exec -t dd2c111ef83ba244012fa509cb5dcb17980fd6f788f358af832398a29c98a0cb /exec failed with exit code 1
• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.