Databricks has built an agent-driven system that writes GPU kernels for specific inference shapes instead of asking one generic implementation to serve every request. The company details the approach in its September 4 Proteus report, where it says specialized kernels for parts of Qwen 3.5 122B ran 1.8 to 5.2 times faster than the best available vLLM implementations in its tests.
Proteus targets a basic weakness in general-purpose inference software. A model fixes some dimensions of an operation, but request-time conditions such as token count can change others. A kernel tuned for one shape may therefore leave performance on the table when the workload changes. Databricks’ system searches for code that matches the shape it actually sees, then keeps the result tied to that condition instead of presenting it as a universal replacement.
The unusual part is not that an AI model generates code. It is the checking system around that generation. Proteus compares candidates with a controlled reference implementation, runs correctness checks, and benchmarks only the candidates that pass. Databricks says it also clears compiled state, keeps setup and teardown comparable, uses multiple timers when needed, and holds back some input sizes so the agent cannot optimize only for a visible test set.
The useful win is narrower than the headline
The report’s case study follows a packed decode kernel on the Gated DeltaNet path in Qwen 3.5 122B, running on NVIDIA B200 GPUs with a Triton backend. The reference latency was 0.025 milliseconds. A candidate for a batch-one repair path reached a 1.5 times speedup, while a serving-decode candidate reached 0.018 milliseconds and a best shape speedup of 1.6 times.
That fastest serving fragment was specialized for a Batch=4, Key=128, Value=128 layout and processed the value dimension in 64-wide chunks. The result is useful precisely because its limits are explicit. It shows that a kernel can be improved for a known shape, not that every Qwen request or every GPU will receive the same gain. Databricks also records later C++ attempts that failed during generation or build, which keeps the successful candidate in context rather than turning it into a claim about effortless automation.
The same caution applies to the 1.8 to 5.2 times comparison. Those are company-reported results for individual kernels, and the report does not describe a whole serving system becoming five times faster. In practice, the benefit depends on how often a workload revisits a shape, how quickly kernels can be compiled and validated, and whether the dispatch overhead is smaller than the saved GPU time.
That focus on validation connects with our reporting on memory bandwidth and AI accelerators. A faster arithmetic path does not remove the cost of moving data or coordinating a serving stack. Shape-specific code can expose more of the hardware’s potential, but it can also create a larger catalog of variants that must be selected, cached, tested, and maintained.
Generation is cheap, trust is the bottleneck
Databricks found a second limit in the context supplied to the agent. A large memory of previous attempts can provide useful clues, but it can also consume tokens and mix relevant advice with stale notes. In one long run, the company says most of the model’s token budget went to retrieving and routing that knowledge rather than generating kernels. Its proposed answer is a smaller store of high-trust lessons that pair a specific situation with an action and stay scoped to related operations.
For developers, Proteus sketches a practical pattern for agentic systems that touch production infrastructure: give the agent room to explore, but keep measurement and acceptance outside its control. That is a different proposition from handing an AI model a benchmark and trusting the fastest number. The next question is how much of the specialized search can be reused across models, GPUs, and request distributions without rebuilding the validation harness each time.
Readers tracking the hardware behind local and cloud AI can also use our TPU versus GPU explainer for the broader accelerator picture. Proteus is a software layer, but its premise is hardware-specific: performance improves when the code respects the exact machine and workload instead of smoothing every case into one average path.
