Key Takeaways
Canonical published a complete guide to fine-tuning and running your own language model entirely on Ubuntu.
Orchestrated with Charmed Kubeflow, feature management through Feast, and served via KServe.
The entire architecture runs on normal CPU cores without needing expensive or rented graphics hardware.
Roughly 1 hour to test with GPT-2, or approximately 24 hours to fine-tune a practical assistant like Qwen2.5.
What Canonical Announced
Canonical published an end-to-end tutorial showing how to prepare data, train, and host a custom language model locally on Ubuntu. The entire workflow runs inside a single-node Kubernetes cluster orchestrated through MicroK8s, Juju, and Terraform.
Instead of deploying a bloated enterprise setup, this project installs only three focused components: Kubeflow Trainer v2 to manage training jobs, Feast to store and serve clean training features, and KServe to host the finished model behind an OpenAI-compatible API endpoint.
Hardware and System Requirements
Because this pipeline runs entirely on system processor cores rather than graphics cards, your machine needs enough memory and threads to carry the computation.
How the Workflow Operates
Canonical provides automation scripts that manage the full progression step by step:
- Infrastructure Setup: MicroK8s starts a single-node cluster, while Terraform and Juju automatically configure the Charmed Kubeflow operators.
- Data Storage and Ingestion: The training dataset (
nampdn-ai/tiny-webtext) loads into a local PostgreSQL database, where Feast manages and serves the data features. - Model Training: Kubeflow Trainer v2 initiates the job, running the training cycles across your local CPU cores.
- API Deployment: KServe takes the resulting model weights and exposes them as a live service that matches the standard OpenAI API format.
Model Comparison: GPT-2 vs Qwen2.5
The tutorial starts with GPT-2 as a fast smoke test to prove your cluster works, but switching to Qwen2.5 produces far more practical results.
Why This Matters for Ubuntu Users
This setup shows that you do not need a commercial cloud subscription or multi-thousand-dollar GPUs to learn how modern AI workflows operate. With Ubuntu and open-source tooling, you have a private, self-contained environment where you maintain total custody over your proprietary data.
While CPU training takes longer for larger models, the architectural patterns are identical to full enterprise deployments. Once your local cluster runs smoothly, you can apply this exact design to dedicated GPU clusters whenever you want faster turnaround.
Have you tried fine-tuning smaller models on CPU hardware, or do you prefer using cloud GPUs? Share your hardware setup and run times in the comments below.
