Ordered, per-workload deploys: deploy_order config + deploy-image --workload (renderer-before-Rails rolling deploys)

[justin808]

Summary

Add support for ordered, per-workload deploys so a deploy can bring one workload up and healthy before another. Two composable pieces:

1. A repeatable deploy-image --workload/-w <name> flag — deploy a subset of app_workloads.
2. A declarative deploy_order key in controlplane.yml — ordered groups that deploy-image (and therefore promote-app-from-upstream) deploy one group at a time, waiting for each to be ready before the next.

Motivation

React on Rails Pro's rolling-deploy adapters pre-seed the Node Renderer's bundle cache so that during a rolling deploy the renderer can serve the draining bundle that old Rails pods still request. For the separate-workloads topology (Rails and the Node Renderer as independent Control Plane workloads), this only works if the renderer rolls out and is healthy before Rails cuts over — otherwise Rails serves traffic while the new renderer is still cold (or, with a boot-time seed, the renderer reads the new hash from the live endpoint and misses the draining bundle).

Today cpflow promote-app-from-upstream / deploy-image update all app_workloads at once (see DeployImage#deploy_image_to_workloads), so there is no way to express "renderer first, then Rails." This issue adds that.

Context / related work:

• React on Rails docs + gem (multi-URL adapter, promotion/boot-seed/topology): Pro rolling deploy: multi-URL previous_urls + promotion boot-seed docs react_on_rails#4544
• App-side boot seed (Layer 2): shakacode/hichee#9663
• This feature is Layer 3 (deploy ordering) of that three-layer plan.

Current behavior (for reference)

• deploy-image (lib/command/deploy_image.rb): app_workload_data builds { workload => data } for every config[:app_workloads]; deploy_image_to_workloads sets the image ref on each. No subset/order.
• ps:wait (lib/command/ps_wait.rb): already waits per-workload — config.options[:workload] selects one, else all; readiness is cp.workload_deployments_ready?(workload, ...); suspended workloads are skipped. This is the readiness primitive to reuse.
• promote-app-from-upstream (lib/command/promote_app_from_upstream.rb): copy_image_from_upstream then deploy_image — so anything deploy-image honors is honored by promotion too.

Proposed design

1. deploy-image -w/--workload (repeatable) — the primitive

• Add a repeatable workload option to DeployImage::OPTIONS (the existing workload_option is single-value as used by ps:wait; this needs a multi/array variant, e.g. workloads_option, or make -w repeatable).
• In app_workload_data, when workloads are given, restrict to that set (validating each is in config[:app_workloads]); default (no flag) = all, unchanged.
• Everything else (deploy_image_to_workloads, digest/release handling) is untouched.

cpflow deploy-image -a $APP -w node-renderer            # one
cpflow deploy-image -a $APP -w node-renderer -w sidekiq # several
cpflow deploy-image -a $APP                             # all (unchanged)

2. deploy_order in controlplane.yml — the declarative default

hichee-production:
  app_workloads: [node-renderer, rails, sidekiq]
  deploy_order:
    - [node-renderer]   # deploy this group, wait until ready…
    - [rails, sidekiq]  # …then this group
    # any app_workloads not listed here deploy last, as an implicit final group

• When config[:deploy_order] is present and no explicit -w was passed, DeployImage#call deploys group by group: deploy the group (same path as -w), then wait for it via the extracted ps:wait readiness logic (cp.workload_deployments_ready? per workload, honoring --location, skipping suspended), then proceed to the next group.
• Workloads in app_workloads but absent from deploy_order deploy in a final implicit group (so partial lists are safe).
• The release phase (--run-release-phase) still runs once, before any workload group (unchanged ordering relative to workload updates).

3. promote-app-from-upstream — no change needed

It already delegates to deploy-image, so deploy_order flows through. Consumers using the promotion model (build on staging → promote to production) get ordering by only adding the config — no deploy-script changes.

Composition

deploy_order orchestration deploys each group using the same subset mechanism as -w, and calls the same readiness check as ps:wait. Users can still drive it manually for one-offs:

cpflow deploy-image -a $APP -w node-renderer && cpflow ps:wait -a $APP -w node-renderer
cpflow deploy-image -a $APP   # the rest

Backward compatibility

• No deploy_order and no -w → deploy all workloads at once. Identical to today.
• deploy_order is additive/optional; existing configs are unaffected.

Edge cases

• Suspended workload in a group → skip its readiness wait (mirror PsWait's cp.workload_suspended? skip) so a scaled-to-zero workload doesn't hang the deploy.
• Workload in deploy_order but not in app_workloads → validation error with a clear message.
• Empty / malformed deploy_order (not a list of lists, duplicate workload across groups) → validate at config load with an actionable error.
• Timeout: readiness waits should be wrappable the same way ps:wait documents (timeout 300 cpflow deploy-image ...); consider a per-group timeout option as a follow-up.
• Explicit -w + deploy_order both present → -w wins (imperative override); document this precedence.

Testing

• rspec, mocking the CPLN API as the existing deploy-image / ps:wait specs do.
• New cases: -w filters to the subset; unknown -w errors; deploy_order deploys groups in order; readiness is awaited between groups (assert deploy of group N+1 does not start until group N is ready); unlisted workloads deploy in the final group; suspended workload is skipped; -w overrides deploy_order; no-config path is byte-for-byte the current behavior.

Docs

• docs/commands.md: deploy-image (new -w) and a note under promote-app-from-upstream that it honors deploy_order.
• The controlplane.yml reference: document the deploy_order schema + semantics (ordered groups, implicit final group, -w precedence).

Acceptance criteria

• deploy-image -w deploys only the named workload(s); default unchanged.
• deploy_order deploys groups in order, awaiting readiness between groups, via promote-app-from-upstream with no call-site change.
• Unlisted workloads deploy last; suspended workloads don't block; clear errors on invalid config.
• Specs + docs updated; no change to existing deploy-all behavior.