feat: fit the 18B model on a 24 GB GPU + 64 GB RAM box (staged T5, eager FP8, FP8-only deepcopy)

[yangohuang]

Motivation

The README's RTX 4090/5090 command currently cannot run on a 24 GB GPU with a 64 GB-RAM host:

• At 416×720 the per-step KV caches alone need ~20 GB on GPU (3 denoise steps × 40 layers × 14·frame_seqlen tokens, fp8), so the GPU OOMs while the other modules load.
• Switching to --offload_cache moves the pressure to the host, where the bf16 DiT (18B ≈ 36 GB) cannot co-reside with umT5-xxl (~11 GB) — the kernel OOM killer terminates the process during model loading.

Related reports: #18, #12. Builds on the same test setup as #19 (independent, both merge cleanly).

Changes

Three commits, each self-contained; the datacenter (no-offload) path is untouched:

1. Staged T5 encoding — pre-encode all prompts (including edit_prompt) in a staging pass and release the T5 encoder before loading the DiT; also load the DiT with low_cpu_mem_usage=True. Peak host RAM drops by ~11 GB. Outputs are unchanged for every input json.
2. FP8-only __deepcopy__ for FP8Linear — WanBlockOffloadManager deep-copies blocks[0] for its staging buffers; if fp8 weights were materialized with fp16_weight_storage='discard' before enable_block_offload(), this currently raises RuntimeError: FP8Linear cannot be deep-copied without an FP16 weight source. Reconstruct from a placeholder and clone the fp8 buffers instead. This is a latent bug reachable by any API user today, and a prerequisite for change 3.
3. Eager per-block FP8 materialization under --fp8_gemm --block_offload — quantize block-by-block on GPU right after the DiT loads, discarding each bf16 copy immediately. Host steady-state drops from ~36 GB to ~18 GB before the VAE/CLIP/wav2vec loads, and the per-step H2D copy volume of the offload double-buffer is halved. Gated on --block_offload, so the H100/H200 path keeps its existing lazy behavior.

Measured (RTX 4090 24 GB, 62 GB RAM)

Command: --size 256*416 --fps 24 --fp8_gemm --fp8_kv_cache --block_offload --t5_cpu

metric	value
end-to-end	full 38 s clip generated successfully (3-step, 256×416 @ 24 fps)
steady-state (SDPA attention)	~37.7 s per 1.33 s chunk (RTF ≈ 28)
steady-state (SageAttention v2.2.0)	~31.9 s per 1.33 s chunk (RTF ≈ 24)
GPU peak	~13 GB / 24 GB
host peak	~30 GB (OOM-killed without these changes)

Not real-time, but it turns "cannot run at all" into a working offline-generation path on a common consumer box.

---

中文说明

动机：README 的 RTX 4090/5090 命令在 24G 显存 + 64G 内存的机器上实际无法运行——416×720 下仅 KV cache 就需约 20G 显存（3 个去噪步 × 40 层独立缓存）；改用 --offload_cache 后，主机又无法同时容纳 bf16 DiT（18B ≈ 36G）与 umT5-xxl（约 11G），进程在加载阶段就会被内核 OOM killer 杀掉。相关：#18、#12。与 #19 相互独立、可分别合并。

三项改动（均不影响 H100/H200 原有路径）：

1. T5 分阶段编码：加载 DiT 之前先把所有 prompt（含 edit_prompt）预编码并释放 T5，同时 DiT 改用 low_cpu_mem_usage=True 加载。主机内存峰值降低约 11G，任意输入 json 的输出完全不变；
2. FP8Linear.__deepcopy__ 支持纯 FP8 源：offload manager 初始化时会 deepcopy blocks[0] 作为暂存缓冲，bf16 已丢弃时现有代码会抛 RuntimeError——改为从占位结构重建并克隆 fp8 buffer。这是一个当前 API 即可触发的潜伏 bug，也是改动 3 的前置条件；
3. --fp8_gemm --block_offload 下的即时 FP8 量化：DiT 加载后立即逐 block 在 GPU 上量化并丢弃 bf16 原权重，主机常驻内存 36G → 约 18G，同时 offload 双缓冲的每步 H2D 拷贝量减半。已用 --block_offload 门控，数据中心路径保持原有惰性行为。

实测（RTX 4090 24G + 62G 内存）：256×416 @ 24fps、3 步去噪，38 秒成片完整生成；稳态 SDPA 约 37.7s / SageAttention 约 31.9s 每 1.33 秒视频（RTF≈28 / 24）；GPU 峰值约 13G，主机峰值约 30G。改动前同配置在加载阶段即被 OOM killer 终止。

虽非实时，但把消费级主机上的"完全跑不起来"变成了可用的离线出片路径。