Kestrel
A multi-tenant GPU compute control plane with runway-aware scheduling.
Problem
Every GPU platform bills you after the fact. The scheduler places your job, the meter counts what it cost, and the two never speak, so a tenant who has burned through their budget keeps getting capacity until someone reads a report and turns them off. Billing is a receipt, not a control signal.
Approach
- Make the usage ledger the single source of truth, and let the scheduler read it live rather than only the billing job.
- Admit work against real quota with Kubernetes-native primitives instead of reimplementing a queue.
- Be honest about what is simulated. Fake GPUs, real control plane, so the numbers mean something.
Architecture
Control plane
A FastAPI service backed by Postgres and Redis. Tenants submit jobs (run and finish) or endpoints (stay up behind a URL); the plane admits, places, meters and autoscales them.
Data plane
A real Kubernetes cluster on kind, with a genuine API server, etcd and scheduler, and Kueue admitting workloads against real ClusterQueue quota. GPU nodes are KWOK fakes advertising nvidia.com/gpu, while CPU work runs on a real kubelet.
Metering
An append-only usage ledger records GPU-seconds per tenant, exported to Prometheus and a Next.js operator console. Every figure derives from a ledger row; nothing is seeded.
Decisions
Runway feeds back into scheduling
The same ledger that produces a tenant's bill is read by the scheduler and autoscaler, so remaining budget influences where and whether work runs, not just what it costs afterwards. That feedback loop is the whole point of the project.
Two pools, deliberately
CPU work runs on a real kubelet and takes real time; GPU work lands on KWOK fakes and returns instantly. Keeping both visible makes it obvious which measurements are real, rather than hiding the simulation behind a uniform dashboard.
Allocation, not utilisation
With no DCGM telemetry, GPU-seconds means seconds reserved, not compute performed. Real clouds bill on allocation too, but the distinction is stated rather than glossed over.
Outcome
- Exact metering: GPU-seconds recomputed from gpus x elapsed match the ledger to four decimal places.
- A real ONNX embedding model (all-MiniLM-L6-v2) is scheduled through the platform like any other endpoint.