Shopify CEO Uses AI Coding Agent to Make Liquid Template Engine 53 Percent Faster
Tobias Lutke ran 120 automated experiments in two days using a variant of Karpathy autoresearch system
The PR, which now contains 93 commits from roughly 120 automated experiments, was built using a variant of Andrej Karpathy's autoresearch system. This framework lets coding agents run hundreds of semi-autonomous experiments to discover performance optimizations.
Lutke's implementation started with an autoresearch.md prompt file and an autoresearch.sh script that let the AI agent execute test suites and report benchmark scores autonomously. Over two days, the system discovered dozens of micro-optimizations that a human developer would likely never find through manual profiling.
Among the successful techniques: replacing the StringScanner tokenizer with String#byteindex, eliminating unnecessary object allocations in hot paths, and restructuring internal data structures for better cache locality. The PR description carefully documents which experiments succeeded and which failed.
Simon Willison noted that Liquid was originally inspired by Django's template engine when Lutke first created it back in 2005. A CEO personally shipping performance improvements to a 21-year-old codebase using cutting-edge AI tooling speaks to both the maturity of AI coding agents and the hands-on engineering culture at Shopify.
Analysis
Why This Matters
This is one of the most concrete demonstrations yet of AI coding agents delivering real, measurable engineering value — not generating boilerplate, but discovering novel performance optimizations through systematic experimentation.
Background
Karpathy's autoresearch was originally designed for training optimization experiments on nanochat. Lutke adapted it for Ruby performance work, showing the pattern is generalizable beyond ML training.
Key Perspectives
The 53 percent speedup raises questions about what other mature codebases could benefit from this approach. If a CEO can point an AI agent at a template engine and get major performance wins in two days, the implications for large-scale software optimization are significant.
What to Watch
Whether other companies adopt similar automated experimentation pipelines for performance optimization, and whether this pattern moves from CEO side-projects to standard engineering practice.