Why I built Sleepy code-blind
Trevor Prater · sleepy's one engineer
There is an easier way to build an evolutionary code optimizer: upload source to a vendor cloud, let hosted models mutate it in a datastore, then download the best candidate. That can be a reasonable architecture. It is not one I could ask every team to trust with its hottest code.
Sleepy is built the other way around. The control plane coordinates the search but is structurally blind: it sees opaque digests and pass/fail, never source, never prompts, never raw measurements. Here's why I took the harder road.
1. The code worth optimizing is the code you can't upload
Hot paths are where companies keep their edge: the kernel that makes an inference stack viable, the parser on a latency-critical request path, or the scheduler that keeps scarce hardware busy. The teams with the most to gain from overnight optimization are often the teams whose security review will not approve shipping that code to someone else's datastore. An optimizer that requires upload excludes them before the first generation runs.
2. Self-reported scores are not evidence
When evaluation runs client-side, a hosted service has a choice: accept whatever number the runner reports, and publish a best-practices page politely asking customers to add timeouts and scan for eval() - or ship the integrity machinery as part of the product. I chose to ship it: sealed no-network containers, immutable test snapshots, poisoned-harness negative controls, preregistered paired measurement, and an evidence chain of eight digests per verdict. A search that writes its own code will eventually try to game its own test. "Please be careful" is not a gate.
3. No silent middleman between you and your search
When mutations are generated server-side, a vendor's safety filter can silently drop candidates - your queue just runs empty, and the official fix is to rephrase your problem description. With Sleepy, your prompts go from your machine to your providers with your keys. No relicensing your problem statement to get past someone else's filter, no model lock-in, no invisible hand in the loop. If a model family gets better next month, point your workers at it that night.
What it costs me
Honesty requires saying this part out loud: code-blind is worse for me. Sleepy is one engineer - me - and I can't learn from your code, can't build a training corpus from your candidates, can't upsell hosted compute against data I hold. I accepted that trade because the alternative - "trust me with your hot paths" - is a promise, and promises don't survive security review. Architecture does.
That's the whole position. The gate assumes every candidate is lying, and the business assumes you shouldn't have to trust us. The controls are public, the audit guide is written, and the proof ledger is open.