Mohith

Writing

Notes on LLM systems, infrastructure, and things I’m building.

5 articlesRSS
Mean written reasoning gets shorter from one to three loop passes on fixed math, code and reasoning subsets.

Loop Dial: more depth, less written reasoning

I varied a research model’s loop count. Math and code improved by three passes, reasoning got shorter, and extra depth kept costing more.

8 min read · Language models, Inference, Evaluation
Finish rate and faithful share before and after supervised fine-tuning, with bootstrap intervals.

Teaching Inkling to stop overthinking

Post-training Inkling 2 Small on Tinker cut mean output tokens from 3,109 to 74 on a pixel-art task. What SFT improved, what it did not, and why the next stage stopped.

5 min read · Post-training, Tinker, SFT
Cover for Reward Hacking: When the Model Learns the Metric Instead of the Task

Reward Hacking: When the Model Learns the Metric Instead of the Task

Notes on reward hacking in reinforcement learning and RLHF: what it is, why RL is so prone to it, Goodhart's Law and its four flavors, how it shows up in LLMs (sycophancy, evaluator hacking, in-context reward hacking), and the mitigations that exist today.

8 min read · Reinforcement Learning, RLHF, AI Safety, Reward Hacking, LLMs
Cover for LLM Inference at Scale: Why Serving One User Is Easy and Serving a Hundred Is Not

LLM Inference at Scale: Why Serving One User Is Easy and Serving a Hundred Is Not

A ground-up walk through LLM inference: the KV cache, prefill vs decode, a capacity equation for Mistral 7B on an A100, and the model and serving optimizations (GQA, MLA, FlashAttention, PagedAttention, continuous batching, prefix caching, speculative decoding) that make LLMs servable.

10 min read · LLM, Inference, vLLM, SGLang, KV Cache, Systems