Ex-NASA dev reveals his Agentic Engineering Workflow (Dex Horthy)

The software factory, and where AI plugs in

Dex frames modern engineering as a "software factory" (the idea dates to 1968). The 2022 version: humans put work on a list (Jira/Linear), someone builds it, then a pull request, CI/CD, human review, prod, user complaints and feature requests loop back, and monitoring wakes engineers at 3am. Building takes hours-to-days; review also takes hours-to-days.

The first move with AI is to replace "person builds the thing" with "agent builds the thing," which collapses build time to minutes-or-hours and leaves review as the bottleneck. You then route work straight into the factory: an incident at 3am becomes a pull request waiting for you, not a page; a support/feature request gets triaged straight to an agent. David does a version of this on his product "Deep API," using a model to triage every uptime incident (provider outage vs missing migration) via cron jobs on Vercel plus a small agent loop on render.com.

Why "don't read the code" fails, and the "logic" reframe

The tempting shortcut is the "light software factory": invest in agent code review, live testing, better CI, more linters, so you never read code and just ask for features. Dex tried exactly this in July 2025, shipping a desktop app; then a bug appeared that three frontier models in a "council" kept mis-diagnosing, and the team spent weeks reading the slop code they had stopped reading to find it. His thesis: if you stop reading the code, the odds you hit an unsolvable bug are higher than the odds the models get smart enough in time to save you.

David offers a useful reframemaybe the word is not "code" but "logic." You need to understand the product's logic (what the first five minutes of onboarding does, what cancellation does, what an endpoint fires) even if you don't read every line. Both agree the human has to stay in the loop; a proposed HumanLayer feature even quizzed you on the current state of the codebase (with multiple-choice questions and mermaid diagrams) to check you hadn't lost the plot.

The four-stage program-design system

This is the heart of the video (Dex packages it as a free bundle linked from the video). It applies once you have product-market fit and a team or paying customers; for a pre-PMF startup, it is overkill and you should move fast.

1. Product. What user problem are you solving, and how do you measure it? Dex uses the Amazon "write the blog post (or PRD) before the feature" move and plain-HTML mockups of every relevant view, with no tech, database, or schema talk yet. Perfect clarity on what and why before any prompting.

Measurable output as back-pressure. If you can hand the agent a real number tied to the business (a website's conversion rate, "make this CUDA kernel faster until you hit a 20% resource reduction"), it will optimise hard toward it. David ties this to a /goal command.

2. System architecture. How the services fit together, the flow, new endpoints, new tables, outlines of the queries. Many people are already comfortable here.

3. Program design (the skipped step). One level below architecture: the actual call stack, where files go, and the type/method signatures (not the implementation). Do this in a context-light session so you get maximum model intelligence before you are deep in a long context window, because once thousands of lines exist, the model is biased toward its first choices and re-steering is costly. A related trick (Victor Tali): ask the model "which choices did you make that you're not confident of?", but do it before the run.

4. Vertical slices (tracer bullets). Models default to building horizontally (all of the database, then all of the services, then all of the API, then the frontend), so there is nothing to test until thousands of lines exist. Dex (crediting Matt Pocock's tracer bullets) recommends the opposite: build one thin slice end-to-end first, test it (manually, with curl, or with a browser agent), then add logic and business cases. It is how you learn a new language (Hello World, then push the boundaries), and it lets you re-steer cheaply and early.

Why benchmarks don't measure code quality

Dex dug into how reinforcement learning training works to explain the shared intuition that models can't be trusted to write maintainable code. In RL you give the model a problem, let it try many times at high temperature, score each trace (did it solve it, is it verified from outside), and reinforce the good traces. The problem: nothing penalises bad design.

So a model scoring 99% on SWE-bench proves it is good at problem-solving, not that you can stop reading its code. Maintainability has no fast "oracle": you can run tests in seconds, but the cost of bad architecture is measured in weeks and months, which a training run can't back-propagate. And every benchmark hands the model the whole problem up front; none feeds five features in a row to test whether the codebase stays changeable. Dex points to a newer Cognition benchmark that moves in the right direction: an LLM judge for functional equivalence to the human solution, an LLM judge for explicit quality rules, and a clever check, remove the model's patch and run its own tests against the pre-patch code; if they still pass, the tests test nothing. The takeaway: convert "taste" into numbers, tests, and rules wherever you can, because an LLM judging code quality only goes so far ("if the model knew what good code looked like, it would write it in the first place").

Context engineering, the dumb zone, and bottlenecks

Dex coined "context engineering" in the 12-Factor Agents paper (March/April 2025); the factor that "popped off" was own your context window. Everything is tokens in, tokens out; the only lever is putting the right tokens in (not more), with no wrong information, no missing information, and as small as possible (XML is more token-efficient than JSON). About two months later Andrej Karpathy and Tobi Lutke amplified the term.

Two practical corollaries. First, put context in the filesystem, because models are great at read/write/edit/grep/bash: document architecture decisions (/docs/adr) and external facts (/docs/external: which env vars exist, the payment-processor setup, testing emails) as plain markdown, and pull recent issues in with a deterministic hook at session start (that's free, it costs CPU not inference). Second, watch the "dumb zone": on Opus 4.1/4.5 results degraded past roughly 50% of the window (~100-120k tokens), so compact into a doc and start a fresh session; newer models (Fable, the 5.6 family) and better compaction push this to 200-300k. Models even get "context anxiety" near the limit, rushing and cutting corners like a student running out of exam time.

Finally, bottleneck thinking (Eli Goldratt's The Goal / Theory of Constraints): there are inefficiencies that are not bottlenecks. "Token-maxing" (exhausting your subscription every five hours) optimises a non-bottleneck; if review is the bottleneck, adding coding agents won't help. Work the biggest bottleneck and let non-critical fires burn. Or, as the HumanLayer office joke goes, "stop playing with your coding agents and get back to work."

Tools and resources

Claims to verify

• The title calls Dex an "ex-NASA dev"; that background is not discussed in the video, so treat the NASA framing as the channel's, unverified here.
• "Software factory of AI agents ran four months straight without human input" is stated in the intro about Dex; not substantiated on screen.
• The 12-Factor Agents publication date is given as March/April 2025 ("about 15 months ago" from the recording); confirm the exact date against the source if it matters.
• The transcript audio says "Dylan Mulroy from Cloudflare"; the on-screen tweet is "Dillon Mulroy" (@dillon_mulroy). Use the on-screen spelling.
• Product/benchmark specifics (Deep API on render.com + Vercel cron; the Cognition benchmark's exact mechanics; GLM 5.2 for incident review) are as described by the speakers, not independently checked.
• Model names in passing (Opus 5, Fable, "5.6 family", GLM 5.2) reflect the speakers' current tooling at recording time.

Glossary

• context engineering: the practice of filling an LLM's context window with just the right information for the next step; coined by Dex Horthy in the 12-Factor Agents paper.
• context window: the full set of tokens (system prompt, messages, tools, history) the model sees for one step; the only primitive you actually control.
• software factory: the end-to-end system that turns work items into shipped software (build, test, review, deploy, monitor, loop); here, increasingly run by agents.
• program design: the stage below system architecture, specifying the call stack, file placement, and type/method signatures before implementation.
• system architecture: the high-level design of how services, endpoints, tables, and flows fit together.
• vertical slice (tracer bullet): a thin feature built end-to-end (DB to UI) first so it can be tested early, as opposed to building each layer fully before moving on.
• horizontal build: the model's default of completing one layer (all the DB, then all the services) before the next, leaving nothing testable until the end.
• back-pressure: giving the agent a deterministic, outside-verified signal (a business number or target) to optimise toward.
• oracle (in RL): a fast, reliable check of whether a solution is correct; maintainability lacks one, which is why RL doesn't optimise it.
• SWE-bench: a coding benchmark of real GitHub issues where a model must produce a patch that makes held-out tests pass.
• golden patch: the human's reference solution for a benchmark problem, used to grade or compare the model's answer.
• slop: low-quality, hard-to-maintain code that technically works (unnecessary try/catches, weird type casts, tests that assert nothing).
• own your context window: the 12-Factor Agents principle that you should deliberately control exactly what tokens go into the model, not delegate it blindly.
• ADR (Architecture Decision Record): a short markdown file recording a decision and its rationale, kept in the repo (e.g. /docs/adr) so agents can read it.
• hook (agent): deterministic code that runs around an agent's work (e.g. at session start) to inject context or sync files, costing compute rather than inference.
• dumb zone: the region of a long context window where a model's results degrade (older models past roughly half the window); the cue to compact and start fresh.
• compaction: summarising the current session into a document and starting a new session from it to escape the dumb zone.
• Theory of Constraints / bottleneck: from Goldratt's The Goal; only the true bottleneck limits throughput, so optimising non-bottlenecks (e.g. token-maxing) doesn't deliver more value.
• 12-Factor Agents: Dex Horthy's 2025 paper of principles for building agents, where context engineering was first named.