refactor: hoist Arctic RL integration to top-level arctic_training/ folder

Per Charlie's feedback on the public SkyRL PR — the integration code (two
subclasses + an entrypoint) doesn't belong inside skyrl/ core, where the
SkyRL maintainers would have to keep it building. The legacy skyrl-tx/
folder pattern (top-level sibling of skyrl/, no maintenance from upstream
core) is the right precedent.

Move:
  skyrl/backends/arctic_rl/__init__.py        → arctic_training/arctic_rl_integration/__init__.py
  skyrl/backends/arctic_rl/arctic_trainer.py  → arctic_training/arctic_rl_integration/trainer.py
  skyrl/backends/arctic_rl/arctic_generator.py → arctic_training/arctic_rl_integration/generator.py
  skyrl/backends/arctic_rl/config.py          → arctic_training/arctic_rl_integration/config.py
  skyrl/train/entrypoints/main_arctic_rl.py   → arctic_training/arctic_rl_integration/entrypoint.py

The folder name `arctic_training/` matches Charlie's request (sibling of
`skyrl/`, like the legacy `skyrl-tx/`). The Python module name inside is
`arctic_rl_integration` — intentionally distinct from the upstream
`arctic_training` PyPI package (the DSS server library) so they can
coexist at import time. SkyRL's main_base.py routing becomes a 5-line
shim:

  if cfg.trainer.arctic_rl is not None:
      from arctic_rl_integration.entrypoint import main as arctic_rl_main
      arctic_rl_main()
      return

pyproject.toml gains a `where = [".", "arctic_training"]` and includes
`arctic_rl_integration*` so setuptools picks up the new package directory.

The ArcticRLTrainerConfig dataclass (the trainer.arctic_rl.* schema) stays
in skyrl/train/config/config.py — it's a pure schema, no behavior, near-
zero maintenance burden.

Validated on dev pod (kganesan-dev-h200-0-qsxhj, 8x H200): zero-env
1x4 colocated GSM8K, 3 stable steps, rewards 0.19 -> 0.25, step time
~19s. Matches prior baseline. Import smoke confirms arctic_rl_integration
and upstream arctic_training (DSS) coexist without collision.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: sfc-gh-kganesan

arctic_training/arctic_rl_integration/__init__.py

@@ -0,0 +1,30 @@
+"""Arctic RL backend for SkyRL.
+
+Provides ``ArcticPPOTrainer`` and ``ArcticGenerator`` that route all GPU
+work to an Arctic RL server, allowing any SkyRL recipe to switch backends
+by setting ``trainer.arctic_rl={}`` (and ``colocate_all: false``).
+
+Install::
+
+    uv sync --extra arctic-rl
+
+Usage in a recipe::
+
+    from arctic_rl_integration import ArcticPPOTrainer, ArcticGenerator
+
+The folder on disk is ``arctic_training/arctic_rl_integration/`` — Charlie
+asked for the integration code to sit at a top-level ``arctic_training/``
+sibling of ``skyrl/`` (matching the legacy ``skyrl-tx/`` placement). The
+inner module name ``arctic_rl_integration`` is intentionally distinct from
+the upstream ``arctic_training`` Python package (the DSS server library),
+which we depend on via pip — keeping the two namespaces separate avoids a
+collision at import time.
+
+Dependencies:
+    arctic_training — pip package providing ArcticRLClient/Server
+"""
+
+from arctic_rl_integration.trainer import ArcticPPOTrainer
+from arctic_rl_integration.generator import ArcticGenerator
+
+__all__ = ["ArcticPPOTrainer", "ArcticGenerator"]

skyrl/backends/arctic_rl/config.py …training/arctic_rl_integration/config.pyskyrl/backends/arctic_rl/config.py renamed to arctic_training/arctic_rl_integration/config.py skyrl/backends/arctic_rl/config.py renamed to arctic_training/arctic_rl_integration/config.py

@@ -8,13 +8,9 @@
 ARL-specific settings live under ``trainer.arctic_rl`` (see
 ``ArcticRLTrainerConfig``).
 
-Usage (colocated, 4 GPUs):
-    python -m skyrl.train.entrypoints.main_arctic_rl \\
-        trainer.arctic_rl.colocate=true \\
-        trainer.placement.policy_num_gpus_per_node=4 \\
-        generator.inference_engine.num_engines=4 \\
-        generator.inference_engine.gpu_memory_utilization=0.3 \\
-        trainer.policy.model.path=Qwen/Qwen3-0.6B
+This entrypoint is invoked indirectly by ``skyrl.train.entrypoints.main_base``
+when ``trainer.arctic_rl is not None``. Researchers should use the standard
+``main_base`` entrypoint and switch backends via config alone.
 """
 
 import os
@@ -30,8 +26,8 @@
 from skyrl.train.entrypoints.main_base import BasePPOExp
 from skyrl.train.utils import validate_cfg
 
-from skyrl.backends.arctic_rl import ArcticGenerator, ArcticPPOTrainer
-from skyrl.backends.arctic_rl.config import build_rl_config
+from arctic_rl_integration import ArcticGenerator, ArcticPPOTrainer
+from arctic_rl_integration.config import build_rl_config
 
 
 class ArcticRLExp(BasePPOExp):
@@ -90,7 +86,7 @@ def _setup_trainer(self):
         os.makedirs(self.cfg.trainer.export_path, exist_ok=True)
         os.makedirs(self.cfg.trainer.ckpt_path, exist_ok=True)
 
-        from skyrl.backends.arctic_rl.arctic_trainer import _ArcticInferenceEngineStub
+        from arctic_rl_integration.trainer import _ArcticInferenceEngineStub
         ie_stub = _ArcticInferenceEngineStub(client=self.arctic_client)
 
         tracker = self.get_tracker()

skyrl/train/entrypoints/main_arctic_rl.py …ning/arctic_rl_integration/entrypoint.pyskyrl/train/entrypoints/main_arctic_rl.py renamed to arctic_training/arctic_rl_integration/entrypoint.py skyrl/train/entrypoints/main_arctic_rl.py renamed to arctic_training/arctic_rl_integration/entrypoint.py

@@ -2,8 +2,15 @@
 requires = ["setuptools"]
 build-backend = "setuptools.build_meta"
 
+# ``skyrl/`` is the upstream package. ``arctic_training/`` is a top-level
+# folder hosting the Arctic RL integration as the ``arctic_rl_integration``
+# Python package — folder name matches Charlie's request (sibling of
+# ``skyrl/``, like the legacy ``skyrl-tx/``); module name is intentionally
+# distinct from the upstream PyPI package ``arctic_training`` to avoid an
+# import-time collision.
 [tool.setuptools.packages.find]
-include = ["skyrl*"]
+where = [".", "arctic_training"]
+include = ["skyrl*", "arctic_rl_integration*"]
 
 [project]
 name = "skyrl"

…l/backends/arctic_rl/arctic_generator.py …ining/arctic_rl_integration/generator.pyskyrl/backends/arctic_rl/arctic_generator.py renamed to arctic_training/arctic_rl_integration/generator.py skyrl/backends/arctic_rl/arctic_generator.py renamed to arctic_training/arctic_rl_integration/generator.py

@@ -478,9 +478,12 @@ def main() -> None:
 
     # Route to Arctic RL entrypoint if arctic_rl backend is configured.
     # This allows users to switch backends via config alone without changing
-    # the entrypoint command.
+    # the entrypoint command.  The integration code lives in a top-level
+    # ``arctic_training/`` folder (sibling of ``skyrl/``); the importable
+    # module name is ``arctic_rl_integration`` to avoid colliding with the
+    # upstream ``arctic_training`` package on PyPI.
     if cfg.trainer.arctic_rl is not None:
-        from skyrl.train.entrypoints.main_arctic_rl import main as arctic_rl_main
+        from arctic_rl_integration.entrypoint import main as arctic_rl_main
         arctic_rl_main()
         return