Skip to content

cudart: stub Runtime exports required by conda libtorch_cuda - #638

Merged
BinSquare merged 4 commits into
smol-machines:mainfrom
NickyHeC:nickyhec/cudart-stub-device-set-limit
Jul 28, 2026
Merged

cudart: stub Runtime exports required by conda libtorch_cuda#638
BinSquare merged 4 commits into
smol-machines:mainfrom
NickyHeC:nickyhec/cudart-stub-device-set-limit

Conversation

@NickyHeC

@NickyHeC NickyHeC commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Unblocks #602 conda e2e.

Why

After #602 overmounts shim over libcudart.so.12*, libtorch_cuda.so (conda torch 2.4 / cu12) fails at dlopen on versioned undefs — first hit was cudaDeviceSetLimit@libcudart.so.12. Real cudart is gone, so LD_PRELOAD of the same incomplete shim no longer fallthrough-fixes missing symbols.

Method

objdump -T libtorch_cuda.so  # UND cuda*
nm -D libcudart-shim.so      # T exports
comm -23

→ 8 missing vs current shim. Stub only those.

Exports added

Symbol Behavior
cudaDeviceSetLimit success no-op
cudaDeviceGetLimit write 8 MiB (mirrors cuCtxGetLimit)
cudaGraphAdd{Kernel,Host,EventRecord,EventWait}Node NotSupported (801); stream-capture remains the forwarded path
cudaGraphRetainUserObject success no-op
cudaUserObjectCreate sentinel handle
cudaStreamUpdateCaptureDependencies success no-op

Proof — Lambda A10, #602 agent + this shim, no LD_PRELOAD

Image: pytorch/pytorch:2.4.0-cuda12.4-cudnn9-runtime

cuda: True
matmul_ok torch.Size([4, 4])
backward ok   # transformers tiny-random-LlamaForCausalLM

pip portal-cuda control unchanged (cuda: True) — that link set didn't require these symbols at import.

@ghost

ghost commented Jul 15, 2026

Copy link
Copy Markdown

Ready to review this PR? Stage has broken it down into 2 individual chapters for you:

Title
1 Stub device limit APIs for PyTorch compatibility
2 Add stubs for CUDA graph and stream capture
Open in Stage

Chapters generated by Stage for commit 8d7e0d6 on Jul 15, 2026 7:10am UTC.

@NickyHeC

Copy link
Copy Markdown
Contributor Author

Lambda retest — PASS

On A10 / Ubuntu 22.04 with #602 agent + this shim in packaged rootfs, image pytorch/pytorch:2.4.0-cuda12.4-cudnn9-runtime, no LD_PRELOAD:

cuda: True
matmul_ok torch.Size([4, 4])
backward ok   # HF tiny-random-LlamaForCausalLM + transformers

Audit method: objdump -T libtorch_cuda.so UND cuda* vs nm -D shim exports → 8 missing symbols stubbed across two commits on this branch (79d9ed52, 8d7e0d66).

@NickyHeC NickyHeC changed the title cudart: stub cudaDeviceSetLimit / GetLimit for conda torch import cudart: stub Runtime exports required by conda libtorch_cuda Jul 15, 2026
@NickyHeC

Copy link
Copy Markdown
Contributor Author

Evidence (see description).

Audit: 8 UND cuda* symbols in conda libtorch_cuda.so missing from shim → stubbed.

#602 agent + this shim, pytorch/pytorch:2.4.0-cuda12.4-cudnn9-runtime, no LD_PRELOAD:
cuda: True → matmul → tiny Llama backward ok.

@NickyHeC

NickyHeC commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

Companion to #602 (stubs the 8 conda libtorch_cuda.so Runtime exports missing from the shim). Combined #602 + #638 is the validated e2e config (cuda: True → matmul → tiny-Llama backward ok, no LD_PRELOAD). Marking ready for review; best landed together with #602.

@NickyHeC
NickyHeC marked this pull request as ready for review July 17, 2026 21:01
NickyHeC added 2 commits July 20, 2026 11:58
Conda libtorch_cuda.so version-requires cudaDeviceSetLimit@libcudart.so.12 at
import. After smol-machines#602 stages the shim over real cudart, that missing export breaks
dynamic linking. Mirror the driver shim's cuCtxSetLimit no-op / GetLimit default.
Audit of pytorch/pytorch:2.4.0 libtorch_cuda.so vs the shim left 7 UND
symbols after SetLimit: AddKernel/Host/EventRecord/EventWait Node,
GraphRetainUserObject, UserObjectCreate, StreamUpdateCaptureDependencies.
Export stubs so bind-mount staging can satisfy the dynamic linker; Add*Node
returns NotSupported (stream-capture remains the forwarded path).
@NickyHeC
NickyHeC force-pushed the nickyhec/cudart-stub-device-set-limit branch from 8d7e0d6 to 07b04d2 Compare July 20, 2026 19:01
@BinSquare
BinSquare merged commit 3637e8d into smol-machines:main Jul 28, 2026
11 checks passed
@github-project-automation github-project-automation Bot moved this from Backlog to Done in smol machines Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants