Local Generative AI

Here we cover running generative AI on local hardware: new open-weight models, the runtimes and quantisations used to serve them, and the agent tooling built on top. We test on our own hardware and publish the measured numbers, including where a model or a quantisation falls short of its published claims. We also share articles about our own open-source projects, together with the setup guides and troubleshooting.