Skip to content

Port NInfer to RTX 4090 and Qwen3.6-27B - #1

Merged
jram4 merged 7 commits into
masterfrom
rtx4090-v1
Aug 2, 2026
Merged

Port NInfer to RTX 4090 and Qwen3.6-27B#1
jram4 merged 7 commits into
masterfrom
rtx4090-v1

Conversation

@jram4

@jram4 jram4 commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Summary

  • port the inherited NInfer runtime from RTX 3090 sm_86 to RTX 4090 sm_89 on Linux
  • specialize the compiled and registered product for Qwen3.6-27B
  • retune Gated-DeltaNet dispatch for Ada's 128 SMs
  • retain measured Q5 residual improvements and restore slower projection candidates to their inherited controls
  • add bit-exact validation, curated benchmark evidence, public documentation, and maintenance workflows

Attribution and contribution boundary

This is a maintained port of Don-Chad/ninfer-3090, derived from Neroued/ninfer. MTP, CUDA Graphs, the .ninfer artifact format, C++ Engine API, CLI, and OpenAI/Anthropic-compatible server are inherited capabilities.

The contribution in this branch is the sm_89 architecture port, RTX 4090 GDN dispatch, retained Q5 optimization, 27B-only product specialization, tests, benchmark evidence, documentation, and public maintenance setup.

Verification

  • Hardware qualification evidence: one RTX 4090, CUDA 13.0, GCC/G++ 13.4, Linux
  • Stored result: 51.716 tok/s ordinary decode; 124.574 tok/s MTP-3
  • Local portable suite: 20 passed (tests/artifact, tests/test_bench_matrix.py)
  • Public-tree, secret, local-link, YAML, JSON/CSV, and whitespace checks: passed
  • Provenance: seven commits based directly on upstream 464e2d6

Release note

This is a source-only public release. No model weights, binaries, builds, profiles, request logs, credentials, or raw Nsight traces are included. The stored GPU results are historical qualification evidence; no post-sanitization CUDA binary is claimed as validated.

@jram4
jram4 marked this pull request as ready for review August 2, 2026 06:40
@jram4
jram4 merged commit 222aa34 into master Aug 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant