Local model operator

Paddock

Run local models as dependable infrastructure, not a pile of processes.

Paddock manages llama.cpp model lifecycle, OpenAI-compatible routing, downloads, telemetry, and a live operator dashboard on one machine.

Public alpha · Linux x86-64

Paddock

A real Linux and CUDA smoke run: model roles, durable aggregate telemetry, process identity, and GPU state.

Where it fits

The local inference layer

Operators who want a clear, local-first control plane for GGUF models and the applications that depend on them.

Capabilities

Apply deterministic capacity eviction and optional idle unloading across model roles.

Route supported chat, completion, Responses, embedding, and reranking requests through an OpenAI-compatible surface.

Download, validate, register, start, stop, restart, and inspect GGUF model sets.

Watch model, process, request, and GPU behavior without retaining prompts or responses.

Working sequence

01
Install the verified alpha
Use the public bootstrap and review the current compatibility evidence.
02
Build the managed runtime
Let Paddock detect the host and build its pinned llama.cpp baseline.
03
Register a model
Pull a GGUF artifact set and expose it through the local API when requested.

Boundary

The alpha is published for Linux x86-64; binary availability is not broad hardware certification.
Paddock binds to loopback by default and does not provide an application authentication boundary.
llama.cpp is the production backend. vLLM support is experimental and operator-supplied.

Go deeper