feat(rust): Laguna batched prefill, 156 to 559 pp tok/s, plus weight cache and dtype-cache fix#70

TheTom merged 3 commits into

tom/feat/cuda-hip-vulkan-backendsthewafflehaus/butter:tom/feat/cuda-hip-vulkan-backendsfrom

tom/laguna-prefillthewafflehaus/butter:tom/laguna-prefillCopy head branch name to clipboard