Ubuntu Inference Snaps run AI models locally on Ubuntu 26.04 and 24.04 LTS. Start with Gemma 4 for terminal or browser chat and a local API. Add Inference Manager if you want one API address for several models.
Before you begin
Ubuntu and Snap: Standard Ubuntu 26.04 and 24.04 LTS installations ship with snapd. Run snap version and check that snapd is 2.68 or newer for Gemma 4. On a minimal image with no snap command, install snapd through APT first.
Memory and storage: You can start on a CPU. Plan for a multi-gigabyte download and enough RAM or GPU memory for the selected model. Run df -h /var/lib/snapd to check free disk space.
Drivers first: Install the drivers you need before installing the model snap so it can select the correct engine.
NVIDIA: Run nvidia-smi or apt list --installed 'nvidia-driver*'. The CUDA 12.9 engine requires NVIDIA driver 575 or newer. Ubuntu Desktop’s Additional Drivers tool shows the recommended driver for your GPU. nvidia-driver-580 is one example when it is recommended for your hardware. Reboot after changing the driver and check nvidia-smi again.
Intel NPU: Run sudo snap install intel-npu-driver before installing a model that lists an Intel NPU engine, such as DeepSeek R1. Gemma 4 lists Intel GPU acceleration on supported systems. Check Canonical’s hardware matrix for engines by model and architecture.
Gemma 4 is available on the default stable channel. Use --beta when you intentionally want to try a preview channel.
Ubuntu terminal
sudo snap install gemma4
After the snap and its model components finish downloading, check the engine, model name, services and URLs.
Ubuntu terminal
gemma4 status
gemma4 list-engines
status identifies the active engine. list-engines shows the engine and model combinations offered by the snap. The backends your machine can use depend on its hardware and drivers.
Step 2. Chat in the terminal or browser
Start the terminal chat and ask “How do I check free disk space on Ubuntu?” Press Ctrl+C when you’re done.
Ubuntu terminal
gemma4 chat
For browser chat on the Ubuntu machine running Gemma 4, open http://127.0.0.1:8337. If Gemma 4 runs on a server, start this tunnel from the computer where you’ll open the browser. Replace user@ubuntu-server with your SSH login and keep the session open.
Open http://127.0.0.1:8337 on your computer. Port 8337 serves the page and 8336 serves its API. Browser chat needs both forwards.
Step 3. Test the local OpenAI-compatible API
Use the openai URL from gemma4 status. This example uses the default API port and a /v1 base path. If your status output differs, use the full base URL it reports.
Ubuntu terminal
curl -fsS http://127.0.0.1:8336/v1/models
Use the id from the response. Replace gemma4-e4b if your ID differs. A successful response includes the reply under choices.
Ubuntu terminal
curl -fsS http://127.0.0.1:8336/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"gemma4-e4b","messages":[{"role":"user","content":"Suggest a name for a backyard herb garden."}],"max_tokens":96}'
Manage multiple models with Inference Manager
Step 4. Install the manager and discover model snaps
Canonical’s Inference Manager (version 1.0.1) gives compatible apps one local OpenAI-compatible API address. Use it to switch between separately installed models from the same client.
Ubuntu terminal
sudo snap install inference
inference find gemma
If you skipped the standalone install, install Gemma 4 through the manager. If it’s already installed, go straight to Step 5. To add another model, use inference find to search, then pass a name from the results to inference install. Each model can take several gigabytes.
Ubuntu terminal
inference install gemma4
Step 5. Check the proxy, upstream and supported engines
Ubuntu terminal
inference models
inference status
inference info gemma4
models lists the available models. status shows the proxy base URL. info gemma4 prints the proxy and upstream as a tree, followed by the full engine and model-size matrix. Look for rows such as amd-gpu, cpu, intel-gpu and nvidia-gpu. A check mark indicates a supported combination. Run gemma4 status to see which one your machine is using.
Ask the proxy for model IDs, then send a chat request using the exact ID it returns. The manager prefixes the model ID with the provider name. This example uses gemma4/gemma4-e4b.
Ubuntu terminal
curl -fsS http://127.0.0.1:8400/v1/models
Ubuntu terminal
curl -fsS http://127.0.0.1:8400/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"gemma4/gemma4-e4b","messages":[{"role":"user","content":"Suggest a name for a backyard herb garden."}],"max_tokens":96}'
In Open WebUI, OpenClaw or Continue.dev, choose an OpenAI-compatible connection and set the base URL to http://127.0.0.1:8400/v1. Enter the proxy’s full model ID if the client asks for one. If a client requires a non-empty API key field, enter a placeholder such as local or none because the default local endpoints do not require authentication.
Loopback refers to where the request originates. For a browser-based direct connection, that’s the computer running the browser. For a request from a Docker backend, that’s the container unless it shares the host network. For a desktop client on another machine, keep the proxy on loopback and forward port 8400 over SSH. If you use OpenClaw, pick a model that supports tool calling.
When to use Inference Snaps or Ollama
Inference Snaps package a model as a strictly confined snap. At install time, they detect the hardware and choose a supported runtime and pre-quantized weights. Depending on the snap, the engine can target a CPU, NVIDIA CUDA, AMD ROCm, Intel GPU or NPU, or a Vulkan-capable GPU. This route fits when the Snap Store has the model you want and you want the package to choose the initial engine and weights.
Choose Ollama on Ubuntu if you already use ollama pull or want to browse its model library. Open WebUI gives you a browser interface that can connect to an OpenAI-compatible endpoint or Ollama.
Troubleshoot engines, services and ports
Gemma 4 selected the CPU or a service stopped
Check the services and read the last 50 log lines. Use the second block if you installed Inference Manager.
If you installed a driver after the model snap, rerun engine detection. Switching can download new components and interrupt active requests.
Ubuntu terminal
sudo gemma4 use-engine --auto
gemma4 status
To choose an engine yourself, run gemma4 list-engines, then pass one of the listed names to sudo gemma4 use-engine. For example, sudo gemma4 use-engine cpu selects the CPU engine.
An API or browser port is already in use
Canonical’s model registry assigns ports 8324 through 8371. The manager uses 8400 by default. Check which ports are listening and pick unused replacements.
Ubuntu terminal
sudo ss -ltnp
For Gemma 4, use its CLI to inspect and change the API and Web UI ports. If the command prompts to restart the services, accept to apply the changes.
Ubuntu terminal
gemma4 get http
gemma4 get webui.http
sudo gemma4 set http.port=8436 webui.http.port=8437
gemma4 status
For Inference Manager, use Snap configuration. The configure hook restarts the proxy when the port changes.
Ubuntu terminal
sudo snap get inference http
sudo snap set inference http.port=8440
inference status
Update client URLs and SSH forwards to match. If you changed a model’s port, run inference info gemma4 to verify the upstream too.
Update or remove Inference Snaps
Snap updates automatically. For an immediate check, run sudo snap refresh gemma4 or sudo snap refresh inference for the snap you installed.
Removing Gemma 4 frees the space used by its installed model components. --purge removes snap data without making a new automatic snapshot. Save any data and settings you want to keep before running the relevant command. Existing snapshots are listed by snap saved.
Ubuntu terminal
sudo snap remove --purge gemma4
Removing the manager stops the unified endpoint. Remove any separately installed model snaps individually to reclaim their disk space.
Ubuntu terminal
sudo snap remove --purge inference
Frequently asked questions
Do I need Inference Manager to run Gemma 4?
No. The Gemma 4 snap provides terminal chat, browser chat and its own API. Add Inference Manager when you want one API address for multiple model providers.
Can I use an Inference Snap without a GPU?
Yes, if the model snap has a CPU engine for your architecture and you have enough RAM. Gemma 4 has CPU options for amd64 and arm64. The chosen model size and your hardware affect how fast it runs.
Why does the Gemma 4 page load over SSH but chat fails?
The browser page and the model API use separate ports. With the default Gemma 4 configuration, forward port 8337 for the page and port 8336 for the API in the same SSH session.