# ggml HIP backend built against AMD-provided ROCm ###

The ggml HIP backend in the official Debian/Ubuntu APT repositories, package
[libggml0-backend-hip](https://packages.debian.org/sid/libggml0-backend-hip),
is built using the AMD ROCm packages from the same official repositories.

In some cases, those ROCm packages (and possibly also the Linux kernel) might
be too old to support the most recent hardware.

To address this, the Debian/Ubuntu ggml maintainers provide versions of the
ggml HIP backend built against the ROCm packages in AMD's APT repository.
Currently, the following versions of this backend are provided:

  - Package `libggml0-backend-vendor-hip-7.14`, built against ROCm 7.14

The maintainers ship these packages from their private APT repository, for the
following releases:

  - `trixie-backports`, for Debian 13 (trixie)
  - `noble-backports`, for Ubuntu 24.04 (noble)
  - `resolute-backports`, for Ubuntu 26.04 (resolute)


## Prerequisites

### Enable official backports

To enable backports on Debian, follow the official instructions at
[Add Backports to sources.list](https://backports.debian.org/Instructions/#index2h2).

Ubuntu enables backports by default, so no action is required there.

### Enable AMD's repository

These backends require ROCm packages from AMD's APT repository. To enable AMD's
repository, follow the instructions at 
[Install AMD ROCm](https://rocm.docs.amd.com/en/latest/install/rocm.html).

Be sure to check the section "Install the kernel driver", in case your hardware
also requires a newer kernel.


## Enable the Debian/Ubuntu ggml maintainers' private repository

To make use of the maintainers' private APT repository, you'll first need to
install that repository's archive key:

```
$ sudo wget -O /usr/share/keyrings/ai-archive-keyring.gpg \
        https://apt.ai.debian.net/debian/ai-archive-keyring.gpg
```

Then, you need to create a `.sources` file for that repository. Pick the
version of this file applicable to your distribution from below: 

```shell
# Debian 13 trixie
$ sudo tee /etc/apt/sources.list.d/llama.cpp.sources <<EOF
Types: deb deb-src
URIs: https://apt.ai.debian.net/llama.cpp/debian
Suites: trixie-backports
Components: vendor-amd
Signed-By: /usr/share/keyrings/ai-archive-keyring.gpg
EOF

# Ubuntu 24.04 noble
$ sudo tee /etc/apt/sources.list.d/llama.cpp.sources <<EOF
Types: deb deb-src
URIs: https://apt.ai.debian.net/llama.cpp/ubuntu
Suites: noble-backports
Components: vendor-amd
Signed-By: /usr/share/keyrings/ai-archive-keyring.gpg
EOF

# Ubuntu 26.04 resolute
$ sudo tee /etc/apt/sources.list.d/llama.cpp.sources <<EOF
Types: deb deb-src
URIs: https://apt.ai.debian.net/llama.cpp/ubuntu
Suites: resolute-backports
Components: vendor-amd
Signed-By: /usr/share/keyrings/ai-archive-keyring.gpg
EOF
```

Finally, run `sudo apt update`.


## Installing the backend

With everything configured as per the instructions above, all you need to do is
install the backend:

```shell
# For ROCm version 7.14
$ sudo apt install libggml0-backend-vendor-hip-7.14
```

If you still have
[libggml0-backend-hip](https://packages.debian.org/sid/libggml0-backend-hip)
installed, the above command should automatically uninstall it. This is
expected, as both versions cannot be installed at the same time.
