Re: Ollama package ships no CUDA backend, NVIDIA users left with slower Vulkan path

On Wed, Jul 22, 2026 at 10:16 AM Priscila Gutierres via devel
<[email protected]> wrote:
>
> Hello!
>
>   I wanted to bring up something I noticed about the current Ollama package in Fedora (0.12.11 / 0.24.0 in rawhide). The package ships  with ROCm and Vulkan GPU backends, but there is no CUDA backend at  all. This means users with NVIDIA GPUs can only use the Vulkan path
>   for GPU acceleration, which is noticeably slower than CUDA for LLM
>   inference workloads.
>
> I'm running a laptop with an RTX 3060 (6 GB VRAM) and 64  GB of RAM. Many of the models I run don't fit entirely in VRAM, so  some layers get offloaded to the CPU.
> In this scenario, both the GPU  and CPU inference paths matter a lot. On the CPU side, the package  already does a great job by shipping multiple architecture-specific  builds (Alder Lake, Haswell, Ice Lake, etc.) with the right SIMD flags  for each, so the layers that run in RAM are well optimized.
> But on  the GPU side, the layers that should be running fast in VRAM are going  through Vulkan instead of CUDA. The result is that half of the  inference pipeline is properly optimized and the other half is not,  which becomes especially noticeable with partially offloaded models  where both paths are active at the same time.
>   I know CUDA is proprietary and cannot live in Fedora's official  repositories, which is why the package only ships ROCm and Vulkan  today.
>  But the package already splits GPU backends into separate  subpackages (ollama-rocm, ollama-vulkan), so the structure for an ollama-cudasubpackage is already in place. Since the NVIDIA drivers  and CUDA toolkit are already available through RPM Fusion, it seems  like a natural fit to offer ollama-cuda there as well, following the  same pattern. This would save NVIDIA users from having to compile from  source or fall back to the upstream install script (which bypasses  the package manager entirely and breaks on every kernel or driver  update, as discussed in the Getting ollama to use GPU thread (Fedora Discussion).
>
>  It is worth noting that a community COPR already solves this, the  fachep/ollama repository packages an ollama-cuda subpackage built  against CUDA 12.8, covering a wide range of GPU architectures from  Maxwell to Blackwell.
>  Given that this work already exists and is  functional, would it be possible to incorporate it into the official  Fedora package as an RPM Fusion dependency, or coordinate with the  COPR maintainer to move it into RPM Fusion proper?
> Thanks for all the work on the Ollama package. The CPU variant builds  and the overall packaging quality are really solid.
>

RPM Fusion is not part of the Fedora Project. You'll need to talk to
them if you want ollama-cuda. From the Fedora point of view, I would
hope that we'd push harder for everything to use Vulkan Compute or
OpenCL instead of these disparate proprietary APIs. And while ROCm is
open source, it is a vendor-specific API, and I would love for us to
not rely on that in the future.



-- 
真実はいつも一つ！/ Always, there's only one truth!
-- 
_______________________________________________
devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Do not reply to spam, report it: https://forge.fedoraproject.org/infra/tickets/issues/new