Last newsletter I was unhappy with my Tasktron workflow harness: The explicit state machine and reified effect design worked, but it obscured the core workflow, the harness’s raison d'être.
I embarked on a rewrite, hoping to find a more concise and self-evident design based on durable effects (memoized steps) instead — something that would let the code express the intended behavior directly (“Do X, then do Y, then Z”) while being robust to crashes and transparently replayable.
However, multiple weeks of attempts with Astra 6, Opus 5.5, and the newly released Sol 6.1 brought me no closer to the improved design I felt certain could exist.
Despite my best efforts and instruction, I couldn’t get the models to emit tight, concise code; instead, they kept:
This got me thinking about the two very different things I want from programming.
The first thing is, well, just a program — something I run which does a straightforward task for me. Vibe-coding with LLMs has been great for these; I’m sure the models get up to all the nonsense I just mentioned, but it doesn’t matter to me — I’m not reading the code, I just want to dictate to my computer instead of typing, export from Photos.app, flash microcontrollers over the network, or a gazillion other things I haven’t bothered to publish because it’s hyper-specific software like mapping MIDI dials to my speaker volume and monitor brightness.
I imagine this is what most people want from LLMs (and computers more broadly): Just solve some problem without making me care about the details.
However, the second thing I want from programming is, just sometimes, to care about the details. I want a precise language and semantics to model something; to build up a theory.
Sure, that theory should be useful for solving the immediate problem — but after that it’s also useful for thinking new thoughts.
For example, after I gave up on coding agents and started writing/refactoring Tasktron myself, I realized that instead of thinking in terms of “Tasks” with “Subtasks” that each yield a commit, I could instead think of the system as managing units of “Work”, each having a set of instructions, possible dependencies on other Work, and a target branch that it should be committed to.
Formalizing this new concept of Work enables both simpler core machinery (no need for it to distinguish tasks/subtasks) and an entirely new workflow — in addition to the upfront-planning workflow where a Task is defined and then decomposed into subtasks, the harness could also support one-off bits of work. E.g., after reviewing a complex GUI, I could dispatch a few one-offs like “Adjust the background color here”, “Move the popup over here”, “Put this bit in a sidebar” and have the harness execute all of those in parallel on different worktrees and serialize the three resulting commits on a branch.
This concept and the additional functionality it unlocked isn’t something I discovered in my week of prompting agents; it only came once I started digging in more myself.
That theories are useful, but can be built only with deliberate mental effort isn’t exactly a new take; Peter Naur discussed Programming As Theory Building 40 years ago:
[P]rogramming properly should be regarded as an activity by which the programmers form or achieve a certain kind of insight, a theory, of the matters at hand. This suggestion is in contrast to what appears to be a more common notion, that programming should be regarded as a production of a program and certain other texts.
This isn’t specific to programming either, of course. I don’t know any music theory, but I can tell this guy Remaking Kids by MGMT to Learn Why It’s So Good understands that banger of a song way more than I do and can explore musical ideas far beyond anything my plain English prompting could elicit from Suno.
Since theories reside in our minds, it follows that we must be mentally engaged to build them.
But in my usage of coding agents so far, I’ve struggled to stay deeply engaged in the details. Prompting apps into existance is fun and exciting, sure, but my sense for how it all fits together is much duller.
So far, I’ve used coding agents in two broad approaches.
First, I’ve tried the “obvious” workflow: Ask the agent to generate code, read it myself, then ask for changes.
I find this isn’t mentally engaging or invigorating, for two reasons:
Reading piles of agent-generated code is exhausting; there are nonsensical names, references to past instructions / removed code, poor information-hiding, etc. It’s hard to decipher “what’s going on here?”. Furthermore, the latency when asking the agent to explain or revise can be on the order of 1–10 minutes per turn, which makes it hard to stay focused.
Worse, the code sometimes works; you asked for a button that does X, then see for yourself that the new button does, in fact, do X. At this point, it’s extremely tempting to start implementing the next feature and skip the mentally arduous theory building. (See: Once the Cheat Code Is There)
Second, I’ve tried “plan” workflows, both:
the ones in the Codex and Claude Code harnesses, where an agent asks you a few multiple choice questions, then generates a markdown plan of work to be done
my own Tasktron workflow, which supports a longer interactive planning conversation before generating a markdown plan
I found these to be a mixed bag: The markdown plan tends to be shorter than the code that’s later generated from it, but much harder to parse and understand.
When it comes to reinforcement-learned slop, I find it much easier to wade through unnecessary try/catch blocks compared to LLM word salad prose full of “it’s not this, it’s that” and “load-bearing gates”.
While planning workflows are helpful for keeping agents on track over longer sessions, after the agent(s) have implemented the plan, you’re arguably in a worse position compared to before: Now you have an even larger pile of code to read and a markdown document too!
(Aside: Should you keep that around? Should you edit it to keep up with later changes in the codebase? Have the LLM edit it? I find the agents get distracted by these files, either maintaining them as “documentation” or getting tunnel-vision on parts of the code where the implementation doesn’t, in their fresh-context opinion, fully satisfy the original requirements.)
In any event, when it comes to theory-building, I haven’t found planning workflows to be much better than simple prompting.
In both cases, the walls of text/code wear me down into a passive bystander who nudges with low-information prompts and accepts the agent’s recommendations (“propose a few options for us to discuss” quickly turns into “okay sounds good”).
So, if theory-building requires that we be mentally engaged but our mortal minds cannot withstand the unthrottled output of LLMs, where does that leave us?
Well, we could throttle the output and increase the signal-to-noise ratio.
Some folks are trying to shift to a “higher level of abstraction” with formal models in TLA+, Lean, Alloy, etc. I’ve only dabbled with these, so I doubt I could use them effectively myself, much less supervise LLMs generating them. (Not to mention the famously hard problem of getting the implementation to actually match the formal model.)
Instead, I’m curious about a more prosaic approach: iteratively sketching using regular programming languages.
E.g., writing code “top-down”, starting from main and introducing types and function signatures / interfaces one level down the call tree at a time.
This is how I designed programs before LLMs, and I’m a bit embarrassed to admit that it’s taken about a year of using coding agents for it to occur to me to try this proven design approach with LLMs.
Pre-LLM, I’d usually start with a few different “entry points” like example invocations in the README, integration test scenarios, etc., to help me figure out the appropriate concepts from the “outside-in”. This process of sketching out from different perspectives helps me get a handle on the problem domain and the trade-offs between alternative designs.
So how exactly can I interactively collaborate with LLMs in this way?
Can they be simply prompted into this style of working? (“We’re sketching out architecture, only write types and function signatures, no implementations”.)
Or is their desire to write code too strong, such that they’ll need to be constrained within a custom harness?
(I.e., take away bash in favor of add_function_signature(name, args, docstring); or perhaps just a post-tool-call helper that strips implementation code and adds a stern warning to context.)
For me to engage with the medium itself — the code — the work needs to happen in my editor, where I have the muscle memory of my navigation and editing commands.
I want to LSP-rename some function in 3 seconds myself, not type out instructions in some anemic <textarea> or awkward TUI and then wait 15 seconds for a GPU cluster in California to do it for me.
Thinking hard about the names is the point, the workflow should make it easy to do!
Given a tasteful, human-directed sketch of a design code, how well do coding agents fill in the remaining details? Do they largely respect the module boundaries and interfaces, acting essentially as super-charged “typed-hole fillers”? Or will they immediately refactor and rename things?
Should one agent do all the work? I find LLMs can’t really “forget what they know” and write for the reader’s perspective — they write functions that depend on the implementation details of their callers and comments that make oblique references to distant parts of the codebase or prompt.
So perhaps invoking an agent for every empty function would yield better code — by construction they wouldn’t be able to introduce such layering violations, as the only context they’d have is the target signature and list of other pre-established functions in that namespace.
If implementation uncovers deficiencies in the sketched plan/architecture, should the agent escalate back to you and request an architectural redesign? Or (depending on the cost of tokens, perhaps) should it just try to push forward itself and “ask forgiveness” later?
What about for brownfield work? Can agents effectively “sketch” after having read a fully implemented codebase, or would they perform better if their context is limited to the “table of contents” of existing namespaces, function signatures, etc.?
Anyway, I’m sure people are exploring these questions. However, I’ve had a heck of a time finding their papers / blog posts / screencasts / GitHub repos amongst the tidal waves of hype around new model releases and vibe-coded harnesses with the exact same chat-based interaction mechanics.
Do you know of any collaborative human/LLM interaction work? Have you tried these sorts of experiments? Let me know!
Bitter Lessons and Premature Optimizations. “[The Bitter Lesson] meme risks encouraging attachment to a different set of ideas: that generality is always desirable, that scale will fix present architectural problems, that specializing is a lack of ambition, and that engineering effort is wasted if computation could theoretically learn the solution.
Semantics for 2D Rasterization. The authors develop a formal semantics for Skia’s command language, use Lean to prove certain rewrites are equivalent, then build an optimizer. Very readable, fun lil’ paper.
The Accelerationist Case for Frontier Pacing "This is another reason pacing can be strategically productive. It creates room for orientation. In an environment saturated with FUD and “resistance is futile” rhetoric, organizations can lose before competing because they assume frontier capability automatically determines every downstream contest. It doesn’t. Superior orientation does.”
The Cartesian Hand: In-Hand Manipulation with All-Linear Fingers