feat(rl): Cortex-serverless dispatch via arctic_platform.rl (zero integration change) - #50
Draft
sfc-gh-kganesan wants to merge 6 commits into
Draft
Conversation
Extend `ArcticRLClient` and the `CortexTransport` so the two pre-merged
integrations (`arctic-skyrl`'s `arctic_trainer._ArcticDispatch` and
Arctic-Platform's own `integrations/verl/adapter.py`) can swap onto the
unified `arctic_platform.client` package with a bare import change --
including `backend=cortex` for serverless runs.
Config (`config.py`):
- `extra="ignore"`; `_apply_legacy_aliases` collapses `backend=local` /
`dss-platform` / `neutrino` into `onprem` / `cortex`, aliases
`sample_gpus` -> `sampling_gpus`, silently drops legacy fields
(`log_prob_engine`, `sampling_engine`, `reference_model`,
`job_name`, `experiment_name`).
Client (`client.py`):
- `training_job_id` / `sampling_job_id` / `log_prob_job_id` / `get_server_state()`
properties so `main_arctic_rl.py`'s pre-init read + Ray-reconnect path
keep working verbatim.
- `fwd_bwd` / `fwd_no_grad` grow `**legacy_kwargs` folded into the body
(SkyRL passes `post_processors=[...]`; verl passes `reference_model=`,
`context=`, `router_replay=`).
- `sync_weights(cuda_ipc=False, low_memory=False)` -- SkyRL's colocated
path calls `sync_weights(cuda_ipc=True)`; Cortex ignores the flags.
- `wake_/sleep_ × {training, inference, log_prob}` +
`empty_training_cache`, `weight_norm`, `save_weights` take `**kwargs`
folded into the body (verl calls `wake_inference(tags=...)` /
`sleep_inference(level=...)`).
- `_flatten_metrics` bubbles `metrics.grad_norm` / `.loss` to the top
level and mirrors `loss` <-> `avg_loss` so `.get("grad_norm")` /
`.get("avg_loss")` work uniformly.
Transport (`transport.py`):
- Colocation lifecycle ops added to the canonical `OPS` set.
Cortex transport (`transports/cortex.py`):
- `fwd-no-grad` and `log-probs` handlers (rely on Neutrino GS endpoints
tracked separately).
- Colocation lifecycle ops registered as `_colo_noop` so SkyRL/verl's
colocated call sites don't have to branch on backend.
- `_normalize_train_body` translates the verl-GRPO
`{batch, meta, processing}` envelope into Cortex's RPC-style
`{args, kwargs, meta, processing, reference_model, ...}` body.
- `_shape_train_response` shims the loss-only Cortex response into the
on-prem shape (`{avg_loss, loss, metrics.grad_norm, post_process_outputs}`)
and aliases `model_outputs` -> `batch` so `response["batch"]["log_probs"]`
works for both transports.
On-prem Ray transport (`transports/onprem_ray.py`):
- `get_server_state()` returns the state actor so SkyRL's driver->worker
reconnect flow can reattach across Ray processes.
Tests:
- `test_skyrl_verl_compat.py` pins every SkyRL `_ArcticDispatch` and verl
`ArcticRLClientWrapper` call pattern with a recording transport (67
tests total).
- Existing `test_client_ops.py` updated for the new sync-weights body
and the expanded canonical `OPS` set.
Stacked on top of #47 (Cortex transport + DSSST1 wire codec); merge that
first.
Co-authored-by: Cursor <cursoragent@cursor.com>
17 tasks
`RayTransport.__init__` unconditionally spun up a fresh `create_arctic_rl_ray_server_state()` — fine for the driver, wrong for every subsequent reattach. verl's forwarder worker and SkyRL's `@ray.remote skyrl_entrypoint` both rebuild the client in a Ray-remote process from the driver: without a way to reuse the driver's state actor, the second process races a fresh actor set against the existing jobs. `create_arctic_rl_client(config, rl_server_state)` in the old async client accepted this exact handle; this restores it on the unified client via a `server_state=` kwarg threaded from `create_arctic_rl_client` -> `ArcticRLClient` -> `make_transport` -> `RayTransport`. Non-Ray transports ignore it. `create_arctic_rl_client(config, server_state)` keeps `server_state` positional to match the legacy 2-arg call site verbatim. Co-authored-by: Cursor <cursoragent@cursor.com>
…g to on-prem
The legacy `arctic_platform.rl.ArcticRLClientConfig` carried four
server-init knobs the unified `arctic_platform.client.ArcticRLClientConfig`
was silently dropping:
- `log_prob_ds_config` — separate DS engine tuning for the log-prob job
- `ds_worker_config` — DS worker knobs (use_liger, attn_impl,
zorro_train_*, logits_*)
- `arctic_inference_config`— Forest Cascade Attention + speculative decoding
- `full_determinism` — reproducibility flag
verl's `ArcticRLClientWrapper` constructs its `ArcticRLClientConfig` with
all four; with `extra="ignore"` on the unified model they got dropped
before reaching `_init_payload`, so the server DS worker + Arctic-Inference
came up in default mode -- silently wrong output.
Add the fields on the pydantic config and thread them through
`OnPremTransport._init_payload`:
- training job: `ds_config`, `ds_worker_config`, `training_config`,
`checkpoint_path`, `full_determinism` (already merged: seed).
- log-prob job: `log_prob_ds_config` overrides `ds_config`;
`ds_worker_config` + `full_determinism` also forwarded.
- sampling job: `vllm_config`, `arctic_inference_config`.
Tests pin the new fields on the config and the resulting per-job-type
payload shape.
Co-authored-by: Cursor <cursoragent@cursor.com>
4 tasks
…nges Both merged upstream integrations (NovaSky-AI/SkyRL#1837:integrations/arctic_rl/ and the Arctic-specific RemoteBackend used by verl-project/verl#6422 living at arctic_platform/integrations/verl/adapter.py) construct their client with from arctic_platform.rl import ArcticRLClientConfig, create_arctic_rl_client client = create_arctic_rl_client(config, server_state) The goal is Cortex serverless support without either integration changing. This commit adds Cortex as a first-class backend on that single entry point. ## What ### `arctic_platform/rl/config.py` - `backend` Literal widened `["local", "dss-platform"]` -> add `"cortex"`. - Cortex-only fields added: `cortex_host` / `_database` / `_schema` / `_endpoint` / `_pat_env_var` / `_base_url`, `max_seq_len`. All Optional; missing keys fall back to `CortexTransport` defaults / `CORTEX_*` env vars. - `_derive_host_port` short-circuits on `backend == "cortex"` (Cortex has no local host/port; SnowAPI routing is by base_url). - `_validate_local_gpu_counts` already scoped to `backend == "local"`; unchanged. ### `arctic_platform/rl/client.py` - `create_arctic_rl_client(config, server_state)` early-branches when `config.backend == "cortex"` and returns `_CortexClientShim` via `_cortex_dispatch.build_cortex_client`. - `ArcticRLHTTPClient` / `ArcticRLRayClient` imports moved *inside* the http / ray branches so the Cortex code path never triggers the on-prem server dep chain (uvicorn -> arctic_inference -> vllm). The type hint for `ArcticRLServerState` is behind `TYPE_CHECKING`. ### `arctic_platform/rl/_cortex_dispatch.py` (new) - `_to_unified_config(legacy)` builds an `arctic_platform.client.ArcticRLClientConfig` sized for Cortex: backend, model_name, GPU counts, seed, all seven cortex_* / max_seq_len fields, and reconnect job ids. On-prem-only knobs (`ds_config`, `arctic_inference_config`, `ds_worker_config`, `log_prob_ds_config`, `full_determinism`, `colocate`, etc.) are intentionally NOT forwarded — Cortex has no local placement to configure and silently passing them would mask real config drift. - `_CortexClientShim` wraps `arctic_platform.client.ArcticRLClient` (which is synchronous by design) and re-exposes the legacy async surface both integrations reach for: `async def fwd_bwd / fwd_no_grad / step / save_checkpoint / save_weights / generate / sync_weights / reset_prefix_cache / wake_/sleep_ × {training, inference, log_prob} / empty_training_cache / weight_norm / log_probs`. Sync methods (`reconnect_config`, `get_server_state`, `shutdown`) match the legacy signature. Properties `config` / `training_job_id` / `sampling_job_id` / `log_prob_job_id` pass through. - `reconnect_config()` round-trips through the legacy config with cortex job ids populated so verl's `reconnect_handle()` pattern keeps working. - `get_server_state()` returns `None` (Cortex has no local Ray state actor; verl's reconnect path re-attaches via `training_job_id`). ### `arctic_platform/rl/__init__.py` - Lazy-load the heavy exports via PEP 562 `__getattr__`. - Eager: `ArcticRLClientConfig`, `WeightSyncConfig` (pydantic-only). - Lazy: `create_arctic_rl_client`, `WeightSyncCoordinator`, and all six `processors.*` re-exports. First attribute access resolves + caches in `globals()` so subsequent lookups hit the normal fast path. - Rationale: prior to this, `from arctic_platform.rl import ArcticRLClientConfig` on a Cortex-only driver pulled `arctic_platform.rl.client` -> `http_client` -> `http_server` -> `arctic_inference.server.metrics` -> `vllm` at package-init time. The Cortex dispatch never actually executes any of that; the eager import was pure overhead that forced Cortex users to install the entire on-prem ML stack. Post-refactor, a Cortex driver needs `arctic-platform` + `pydantic` + `requests`. Nothing else. ### `tests/client/test_rl_cortex_dispatch.py` (new, 23 tests) - `TestConfigTranslation`: cortex backend translates 1:1; cortex fields threaded; missing fields fall back to unified defaults; on-prem fields NOT forwarded (silent-drift guard); reconnect job ids forwarded. - `TestShimAsyncSurface`: pins every async call site both integrations hit — `fwd_bwd` / `fwd_no_grad` w/ `reference_model=` + `post_processors=`, `step(learning_rate=...)`, `sync_weights(cuda_ipc, low_memory)`, `wake_inference(tags=...)`, `sleep_inference(level=...)`, full colocation lifecycle, `generate(prompts, sampling_params)`, `save_checkpoint(stage_info, path)`. - `TestShimSyncSurface`: `shutdown()` is sync; `reconnect_config()` returns a `_LegacyConfig` with cortex job ids populated; `get_server_state()` returns `None` on Cortex. - `TestPropertySurface`: `client.config` returns the legacy config instance (SkyRL reads `client.config.colocate`); job-id properties pass through. - `TestFactoryDispatch`: - `test_backend_cortex_returns_shim`: `create_arctic_rl_client` picks the Cortex path and translates the config correctly. - `test_cortex_path_never_imports_onprem_transports`: pins the invariant — after a cortex dispatch, `arctic_platform.rl.http_*` / `ray_*` modules are NOT in `sys.modules`. Regression here means Cortex drivers pay the on-prem dep cost again. - `test_onprem_transports_are_lazy_at_module_level`: pins the source- level invariant — `arctic_platform.rl.client` module must NOT expose `ArcticRLHTTPClient` / `ArcticRLRayClient` as attributes. ## Result - 92 tests pass locally (69 previously-existing + 23 new). Test file runs on any machine — no vllm, no ray, no arctic_inference required. - `from arctic_platform.rl import ArcticRLClientConfig` on a bare pydantic+requests env loads exactly 2 modules and 0 heavy deps. - Neither upstream integration needs any code change to gain Cortex support; flipping `config.backend = "cortex"` + populating `cortex_*` fields is the entire user-facing UX. Co-authored-by: Cursor <cursoragent@cursor.com>
sfc-gh-kganesan
changed the title
Two-part change: (1) an environment-variable-driven Cortex override on the
`arctic_platform.rl.create_arctic_rl_client` factory so launchers can flip
either integration to Cortex without any adapter change, and (2) a self-
contained fake Cortex GS + transport smoke suite that validates the full
client-side plumbing end-to-end.
## Why the env-var override
Reading the two merged upstream integrations more carefully:
- `NovaSky-AI/SkyRL:integrations/arctic_rl/config.py::build_rl_config` line
570 constructs `ArcticRLClientConfig(backend="local", ...)` with the
string literal — the yaml `trainer.arctic_rl.*` block has no `backend`
field to read from.
- `arctic_platform/integrations/verl/adapter.py::_initialize_client` line
559 does the same: `ArcticRLClientConfig(backend="local", ...)`.
So today, flipping `remote_backend.backend=cortex` / `trainer.arctic_rl.
backend=cortex` in yaml is a silent no-op. To actually reach the Cortex
dispatch path without touching either adapter, the launcher exports these
env vars and `create_arctic_rl_client` rewrites the (hardcoded-local)
config into a cortex one before dispatch. That is genuinely zero
integration change; the entire enablement is launcher-side env.
Recognized env:
- `ARCTIC_RL_BACKEND=cortex` (toggle; anything else = no-op)
- `CORTEX_BASE_URL`, `CORTEX_HOST` (target)
- `CORTEX_DATABASE`, `CORTEX_SCHEMA`, `CORTEX_ENDPOINT`, `CORTEX_PAT_ENV_VAR`
- `CORTEX_MAX_SEQ_LEN` (int; bad values warned, dropped)
Explicit fields on the incoming config win over env — env only fills gaps.
## What
### `arctic_platform/rl/client.py`
- New `_maybe_override_from_env(config)` helper: pydantic-only, reads the
toggle + fields, returns either the input config or a `model_copy(update=)`
rewritten to cortex. Idempotent when `config.backend` is already cortex.
- `create_arctic_rl_client` calls it before backend dispatch. `os` is now
a top-level import (pydantic-only path stays clean; no new heavy deps).
### `arctic_platform/rl/config.py`
- Micro-fix in `_derive_host_port`: short-circuit when the caller passed
both `host` and `port` explicitly. Previously the validator always
triggered `from arctic_platform.rl.ray_cluster import primary_ip`
(which pulls tensordict via `utils.batch`) on the on-prem branch,
even when there was nothing to derive. Tests running on a
cortex-shaped dev env (no tensordict) can now construct a
`backend="local"` config with explicit `host`/`port` — necessary for
the env-override tests that verify the fallback path stays local.
### `tests/client/test_rl_cortex_dispatch.py`
Six new tests under `TestEnvOverride`:
- `test_env_unset_is_noop` — no env vars → `_maybe_override_from_env`
returns the config identity-unchanged. Asserted at helper level so
the local dispatch branch doesn't need vllm/ray to run.
- `test_env_flips_local_to_cortex` — `ARCTIC_RL_BACKEND=cortex` +
`CORTEX_BASE_URL` on a `backend="local"` config produces a
`_CortexClientShim` with the cortex fields threaded. This is the load-
bearing case for driving verl/SkyRL against Cortex.
- `test_env_populates_all_recognized_fields` — every env key in
`_CORTEX_ENV_MAP` reaches its target field on the unified config.
- `test_env_does_not_clobber_explicit_config_fields` — a field the
adapter did populate on `config` wins over the matching env var.
- `test_env_ignored_when_toggle_value_is_not_cortex` — leftover
`ARCTIC_RL_BACKEND=onprem` in a shell doesn't accidentally trigger the
rewrite.
- `test_env_invalid_max_seq_len_is_warned_not_crashed` — bad
`CORTEX_MAX_SEQ_LEN` values log a warning and drop, rather than
hard-failing the launcher.
The `_clear_cortex_env` autouse fixture scrubs the env slate before each
test so ordering isn't load-bearing.
### `tests/e2e/fake_cortex_gs.py` (new, 380 LoC)
Self-contained plumbing-only stand-in for the real Cortex GS. Speaks every
route the client's `CortexTransport` actually hits:
- `POST {prefix}` (CreateJob) → `{job_id}`
- `GET {prefix}/{job_id}` → `{status: job_state_running, sub_jobs}`
- `POST {prefix}/{job_id}:cancel`
- `POST {prefix}/{job_id}/{forward-backward, forward-no-grad, generate}`
(octet-stream chunks, DSSST1-encoded); reassembles multi-chunk
requests via `wire.decode_byte_chunks`.
- `POST {prefix}/{job_id}/{step, save, log-probs, operation}` (JSON).
- `GET {prefix}/{job_id}/requests/{request_id}` → completed state +
result inline (JSON for loss-only responses; DSSST1 base64 for
shape-bearing responses like fwd_no_grad logprobs / generate results).
Response shaping reads the decoded request payload to synthesise shape-
correct fake responses: `fwd_no_grad` returns a `model_outputs.logprobs`
tensor of shape `(batch_size, seq_len)` read off `input_ids`; `generate`
returns one result per prompt in the request. Losses / grad_norms are
random — this validates plumbing, not convergence.
`serve_in_background(host, port)` spins uvicorn in a daemon thread and
polls `server.started` until the socket binds; used by the test fixture.
Also runnable standalone (`python -m tests.e2e.fake_cortex_gs --port 8080`)
for wiring verl / SkyRL recipes against.
### `tests/e2e/test_cortex_transport_smoke.py` (new, 18 tests)
Pins the entire client→wire→REST→response chain against the fake GS:
- `TestInitialize` (1) — CreateJob + GetJob polling completes, sub-jobs
captured under both `training` and `sampling` keys.
- `TestCoreOps` (8) — every op the client dispatches:
- `fwd-bwd`: verl-GRPO envelope `{batch, meta, processing}` → server →
response has `avg_loss` + `metrics.grad_norm` at the top level after
`_shape_train_response` (both verl's `.get("avg_loss")` and SkyRL's
`.get("grad_norm")` work uniformly).
- `fwd-no-grad`: shape-correct logprobs come back under `batch.logprobs`
thanks to `model_outputs -> batch` alias in `_shape_train_response`.
- `step`, `save-checkpoint`, `generate` (multi-prompt result count),
`log-probs` (JSON in, DSSST1 out), `sync-weights`, `reset-prefix-cache`.
- `TestColocationNoops` (9) — every colocation lifecycle op registered
under `_colo_noop` returns `{}` cleanly. SkyRL calls these
unconditionally under `colocate=True`; regression here breaks the
colocated recipe on the first weight-sync.
Total: 116 tests pass locally on a driver with only pydantic + requests
+ fastapi + uvicorn + safetensors + torch — no vllm, no ray, no
arctic_inference. That's the surface a Cortex-only driver actually needs.
## What this validates
Full client-side chain end-to-end, no mocks at any layer between the
driver and the REST endpoint:
launcher env → create_arctic_rl_client → _maybe_override_from_env
→ build_cortex_client → _CortexClientShim → ArcticRLClient
→ CortexTransport → DSSST1 chunked octet upload → fake GS
→ request polling → wire.decode → _shape_train_response
→ back to shim → back to adapter
## What this does NOT validate
- Training convergence — losses returned by the fake GS are random. A
real Cortex server is still needed for loss-curve validation.
- Verl / SkyRL end-to-end runs — those need `verl` / `skyrl` +
`arctic-inference[vllm]` installed on the driver. Follow-up.
Co-authored-by: Cursor <cursoragent@cursor.com>
Framework-level end-to-end driver — runs SkyRL's actual gsm8k GRPO recipe with ARCTIC_RL_BACKEND=cortex pointing at a locally-spawned fake_cortex_gs, so the full chain (SkyRL launcher → arctic_rl integration adapter → env-override → _CortexClientShim → ArcticRLClient → CortexTransport → localhost HTTP → mock GS) is exercised without touching either integration. Convergence is NOT validated (fake GS returns random losses); this is the plumbing gate that complements the 18-test transport smoke suite by proving the SkyRL recipe layer above it reaches the Cortex path with zero adapter change. Portable defaults: CACHE_ROOT falls back to $HOME/.cache/cortex-e2e; SKYRL_ROOT is required and fails fast with a clear message. Uses UV_FIND_LINKS + UV_PRERELEASE=allow to make the recipe's `--with arctic-platform` resolve to a locally-built wheel of this branch (no recipe edit). Co-authored-by: Cursor <cursoragent@cursor.com>
Collaborator
Author
End-to-end test on a Cortex QA endpoint (CPU-only driver)Ran the
What works out of the boxJob spin-up, Two CPU-only-driver frictions worth guarding
Convergence blocker (already called out in the "Follow-ups" section)Training gets through eval + step-0 sync, then dies on step 1 inside Worth noting for reviewers: this call is not a KL-loss thing (KL is fully off in the recipe, no reference model instantiated). It's the π_old snapshot for the PPO/GRPO ratio, invoked once per training step regardless of KL config. Once the endpoint lands (or old-log-probs are folded into |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on / stacked on top of #47. Do not merge before #47 — base branch is
mwyatt/unified-client-3so the diff is only the compat + dispatch + env-override delta.Tracked by #49.
Why
Both merged upstream integrations —
NovaSky-AI/SkyRL#1837 (
integrations/arctic_rl/) and verl-project/verl#6422 (ArcticRemoteBackendinarctic_platform/integrations/verl/adapter.py) — construct their client via one call:Both hardcode
backend="local"in that construction (SkyRL config.py line 570, verl adapter.py line 559), so settingbackend=cortexin yaml is a silent no-op today. This PR delivers Cortex serverless enablement without touching either adapter — the launcher exports env vars andcreate_arctic_rl_clientrewrites the config before dispatch.What
Compat surface on
arctic_platform/client/*Every call site both integrations reach for is now available on the unified client + Cortex transport.
arctic_platform/client/config.py—extra="ignore"(was"forbid") so legacy fields pass through._apply_legacy_aliasescollapsesbackend={local, dss-platform, neutrino}→{onprem, cortex}(withDeprecationWarning), aliasessample_gpus→sampling_gpus, dropslog_prob_engine/sampling_engine/reference_model/job_name/experiment_name. Adds on-prem parity fields (log_prob_ds_config,ds_worker_config,arctic_inference_config,full_determinism) threaded to the on-prem transports via_init_payload.arctic_platform/client/client.py—training_job_id/sampling_job_id/log_prob_job_id/get_server_state()properties.fwd_bwd/fwd_no_gradgrow**legacy_kwargsfolded into the transport body (SkyRL:post_processors=[...]; verl:reference_model=…,context=…,router_replay=…).sync_weights(cuda_ipc=False, low_memory=False). All colocation-lifecycle ops (wake_ / sleep_ × {training, inference, log_prob},empty_training_cache,weight_norm,save_weights)._flatten_metricsbubblesmetrics.grad_norm/.lossto the top level and mirrorsloss↔avg_loss.arctic_platform/client/transport.py— adds nine colocation-lifecycle ops to the canonicalOPSset.arctic_platform/client/transports/cortex.py—_fwd_no_gradand_log_probshandlers; colocation ops registered as_colo_noop;_normalize_train_bodytranslates the verl-GRPO{batch, meta, processing}envelope into Cortex's RPC style;_shape_train_responseshims loss-only Cortex responses into the on-prem envelope and aliasesmodel_outputs→batchsoresponse["batch"]["log_probs"]works uniformly.arctic_platform/client/transports/onprem_ray.py— accepts an optionalserver_state=parameter so verl's driver → Ray-worker reconnect flow can reattach across processes.arctic_platform.rl→ Cortex dispatcharctic_platform/rl/config.py—backend: Literal["local", "dss-platform"]widens to include"cortex". Cortex-only fields added:cortex_host/_database/_schema/_endpoint/_pat_env_var/_base_url,max_seq_len._derive_host_portshort-circuits on cortex, and now also when the caller passed bothhostandportexplicitly (avoids an eager ray_cluster → tensordict import chain on CPU-only test envs).arctic_platform/rl/client.py—create_arctic_rl_clientearly-branches whenconfig.backend == "cortex"and returns via_cortex_dispatch.build_cortex_client.ArcticRLHTTPClient/ArcticRLRayClientmoved to lazy imports inside the on-prem branch so the Cortex code path never triggers the on-prem server chain.arctic_platform/rl/_cortex_dispatch.py(new):_to_unified_configtranslates the legacy config to unified shape;_CortexClientShimwrapsarctic_platform.client.ArcticRLClientand re-exposes the legacy async surface both integrations reach for.Env-var Cortex override — bridges the adapter-hardcode gap
create_arctic_rl_clientcalls_maybe_override_from_env(config)before dispatch. WhenARCTIC_RL_BACKEND=cortexis set, the incoming (hardcoded-local) config gets rewritten to cortex with fields sourced from the env. This is what makes "zero integration change" actually work today, given the adapter hardcodes.Recognized env vars:
ARCTIC_RL_BACKEND=cortex— toggle; any other value is a no-op.CORTEX_BASE_URL,CORTEX_HOST— target.CORTEX_DATABASE,CORTEX_SCHEMA,CORTEX_ENDPOINT,CORTEX_PAT_ENV_VAR.CORTEX_MAX_SEQ_LEN— int; bad values logged and dropped.Explicit fields on the input config win over env — env is a fallback for adapters whose yaml doesn't yet thread cortex knobs.
Lazy
arctic_platform/rl/__init__.pyEager exports (pydantic only):
ArcticRLClientConfig,WeightSyncConfig. Everything else (create_arctic_rl_client,WeightSyncCoordinator, allprocessors.*re-exports) loads via PEP 562__getattr__on first attribute access. Result: a Cortex-only driver needsarctic-platform+pydantic+requests. Nothing else. Pinned bytest_cortex_path_never_imports_onprem_transports+test_onprem_transports_are_lazy_at_module_level.Tests
116 pass locally on a driver with only
pydantic+requests+fastapi+uvicorn+safetensors+torchinstalled. No vllm, no ray, no arctic_inference, no tensordict.Breakdown:
tests/client/test_skyrl_verl_compat.py— 26 tests pinning every call site SkyRL + verl make against the unified client (using a_RecordingTransportshaped like the on-prem server).tests/client/test_client_ops.py— updated for the expandedsync_weightsbody and the nine new lifecycle ops.tests/client/test_rl_cortex_dispatch.py— 29 tests total:TestConfigTranslation(5),TestShimAsyncSurface(10),TestShimSyncSurface(3),TestPropertySurface(2),TestFactoryDispatch(3).TestEnvOverride(6, new): env unset is no-op; env flips local → cortex; every recognized env field threaded; explicit config fields win; non-"cortex" toggle values ignored; badCORTEX_MAX_SEQ_LENwarned and dropped.tests/e2e/test_cortex_transport_smoke.py(new, 18 tests) — full plumbing E2E: driver → unified client → CortexTransport → real HTTP →fake_cortex_gs→ back:TestInitialize— CreateJob + WaitForJob polling completes; sub-jobs captured.TestCoreOps(8) — every op the transport dispatches (fwd-bwd,fwd-no-grad,step,save-checkpoint,generate,log-probs,sync-weights,reset-prefix-cache) round-trips through real DSSST1-chunked octet upload + response decode.fwd-no-gradreturns a real logprobs tensor atresponse["batch"]["logprobs"]of shape(batch, seq_len)aftermodel_outputs → batchalias.TestColocationNoops(9) — every colocation lifecycle op registered under_colo_noopreturns{}cleanly.The fake Cortex GS
tests/e2e/fake_cortex_gs.pyspeaks every REST route the transport actually hits (CreateJob/GetJob/forward-backward/forward-no-grad/generate/step/save/log-probs/operation/cancel/GET requests/{id}), decodes DSSST1 chunked uploads viawire.decode_byte_chunks, and returns shape-plausible canned responses. Fake for training values (random losses, random logprobs) but plumbing-honest: same wire codec, same octet-stream framing, same response envelopes. Doubles as an executable reference for what the real Cortex-training endpoint needs to accept.Runnable standalone:
python -m tests.e2e.fake_cortex_gs --port 8080. Test suite uses the in-threadserve_in_background()helper.User-facing UX (what a launcher does to go serverless)
Zero code change on either integration. The launcher exports env vars:
Then
run_gsm8k_grpo_arl.sh(verl) orrun_gsm8k_grpo_4gpu.sh(SkyRL) runs unchanged; both adapters still build abackend="local"config but the factory rewrites it before dispatch.What this validates
requests-mock, norespx. Wire codec, chunked upload, response polling, response shim, all exercised.mainand stays lazy;test_cortex_path_never_imports_onprem_transportsguards the invariant.What this does NOT validate
verl/skyrl+arctic-inference[vllm]+tensordictinstalled on the driver box. See Cortex-serverless enablement for verl + SkyRL via the unified ArcticRLClient #49 §3 for the runtime pieces.Follow-ups (see #49)
Server-side (Neutrino GS), blocking real Cortex training convergence:
POST /{job_id}/forward-no-gradaccepting{args, kwargs, meta, processing, reference_model?}(DSSST1 chunks in) →{model_outputs: {logprobs, entropy?}, metrics: {...}}(DSSST1 out).POST /{job_id}/log-probs(JSON in / DSSST1 out).metrics: {grad_norm, ppo_kl, pg_clipfrac_lower, pg_loss, kl_loss, kl_coef}block onfwd_bwd/stepresponses.kwargs={input_ids, attention_mask, position_ids, prompts, responses, response_mask, advantages, old_log_probs, ref_log_prob?}on a real cluster.The
fake_cortex_gs.pyshipped in this PR speaks the exact interface the real server needs to match — GS team can run their implementation againsttest_cortex_transport_smoke.pyfor a red/green.Test plan
pytest tests/client/ tests/e2e/locally — 116 pass on a Cortex-shaped dev env (no ray/vllm/arctic_inference/tensordict).arctic_platform.rl.clientno longer exposesArcticRLHTTPClient/ArcticRLRayClientat module scope; the cortex dispatch path never imports the on-prem transport modules.run_qwen3_0.6b_gsm8k_grpo_arl.sh— loss curves matchmainto within numerical noise. Runs unchanged withbackend: local(default). Requires the on-prem stack installed.ARCTIC_RL_BACKEND=cortex+ a real Cortex endpoint, once the server-side items in Cortex-serverless enablement for verl + SkyRL via the unified ArcticRLClient #49 §3 land.Superseded
#51— closed. The verl adapter swap it proposed is no longer needed; the env-override + dispatch shim in this PR handles both integrations from the Arctic-Platform side.