A practical workflow for LLM-assisted development
When LLMs work, it can feel like magic, but when they fail, it feels like you are arguing with a confident bullshit artist. It took me many months of daily use to develop some intuition for where LLMs are likely to produce code that is useful and where they are likely to fail. It also took me a bit of time to figure out how to limit scope and provide enough scaffolding to ensure I get useful results reliably. Having invested the time to learn to use the tool effectively, I very much see the benefits, as I am able to build projects on a scale I would not have attempted before.
In a way, the process is the inverse of regular programming. We tend to build up programs step by step when writing code by hand as we add each function with intention. LLMs tend to produce a lot of code out of the gate and the focus shifts to whittling the code down to what you actually need.
A good way to look at the agentic loop is to view the process as a genetic algorithm. Agentic harnesses are effective because you have an evolutionary process happening. The model outputs something roughly correct before the code gets tested, and then the model gets feedback to iterate on the code. Through this process, it gradually converges on a solution that fits the parameters being tested. In that sense, it is not actually all that different from how humans write code either. You almost never solve a non-trivial problem in one shot. You write your first approximation and then iterate on it. The difference is that the LLM can do this process a lot faster.
What to Delegate
LLMs are trained on massive amounts of public code, which makes them excellent at completing typical tasks. These are things that have been done a million times before and constitute what largely amounts to boilerplate. Throwing a sample JSON response at an LLM and having it write a service endpoint or throwing a bunch of API endpoints at it and having it build a UI using them can be very effective. These are the kinds of common tasks the agent will have a lot of training on, and they can produce something reasonable in one shot. It will probably put more diligence into that task than you would by adding tests and handling all the obvious edge cases.
They are also great at doing explorative work. Identifying a particular call graph and tracing through the steps to figure out how a particular service endpoint is implemented or what parameters you have to pass it are all tasks an LLM can do easily. This can save an enormous amount of time tracing through a codebase and mapping out a particular workflow that you are interested in.
These tools are also great at handling language specific syntax. If you know conceptually what you want to do, like looping through a collection and filtering by a specific parameter, but you are working in a language you are rusty in, then LLMs are great for bridging the gap. They can easily express the logic you want using idiomatic syntax. You can describe the algorithm in pseudo code where you write out the steps and it will handle the rest.
For example, I recently had to work on a JavaScript project, and I have not touched the language in over a decade. I am not familiar with modern tooling or libraries or best practices, and I just did not have the time to get up to speed on all that.
Using DeepSeek allowed me to use JavaScript as effectively as I do Clojure, which I am well versed in. It completely removed the friction of figuring out all the incidental things like syntax or tooling. If you are an expert in a particular domain and you understand the problem you are trying to solve, then LLMs can be a huge amplifier for what you are able to do. They do not replace your skills, but they do allow you to move a lot faster and focus on the big picture of the problem you are trying to solve.
When to Take the Wheel
In my experience, the biggest place where agents trip up is dealing with context and creativity. You have to remember that the AI does not know the specific quirks of your project. For example, if you just tell it to use a Clojure dialect, it might reach for the JVM toolchain it learned Clojure on, such as clojure and lein, none of which exist in that context, or it might assume a tree walking interpreter and try to run the source directly. You need to give it the exact logic, like telling it explicitly that the runtime is pure Chez Scheme and that everything builds through make commands via a chez --script execution, while specifying that the authoritative sources are host/chez/*.ss and jolt-core/*.clj over anything JVM flavored.
Then there is also the trap of the naive implementation. Often, when you give an agent a vague goal, it will hand you something that looks correct on the surface but ends up being structurally wrong. For example, the agent might decide that string method calls should be routed through a generic dispatch table, which ends up re-deriving the receiver type on every single invocation. The proper fix here is to do a type inference pass to prove that those values are strings at compile time, which allows you to emit a direct native call and skip dispatch entirely. An agent told to make the string methods fast will almost certainly keep the generic path by reordering a few cond arms and never bother designing a proper solution. Similarly, if you ask it to implement count on a sequence, it will likely walk the whole thing allocating a fresh cell per element when the collection already knows its own length that can be called in constant time. Ask it to join strings and you will probably get repeated concatenation instead of a single walk. It is akin to an evil genie that will interpret your queries in the worst way possible, leading to the solution having a completely wrong shape. The trick is that you have to spell out the constraint, which incidentally forces you to think through the problem as well.
The key to using LLMs effectively is to make sure you already have a solid understanding of what you are aiming to build before you start. You always have to be explicit regarding what you want done at a structural level. The more scaffolding you provide up front the less room the agent has to go outside your design. A corollary to this observation is that you do have to understand the domain to make effective use of LLMs. If you are not equipped to evaluate whether the code it produced solves the problem in a correct way, then you basically end up at a casino pulling a lever on a slot machine and hoping for a decent solution to fall out. LLMs are good at filling in the gaps and doing boilerplate, but you still have to do design and architecture the same way you always did.
Here are some tricks that I found useful for keeping it on the rails.
Always start out by planning out the task. Make sure you have a clear picture of what you are aiming to do along with what algorithms you are intending to use and how the code should be structured to fit within the existing architecture. You must be able to answer these questions before you even think about delegating to the LLM.
Once you have a clear picture in your head, you can move on to the planning stage with the agent. Give it the requirements and spell out the goals before asking the model to write a phased plan in Markdown. Even better, ask it to generate a Mermaid.js diagram of the flow.
After it makes the diagram, you can visually inspect the logic. If a particular step looks wrong in the diagram, you tell it to change that specific step to do something else. Doing that is a lot easier than simply arguing with it using text prompts. Once there is a clear structure for the steps being performed, it is easy to identify parts that you do not like. Review the plan and get the model to break it up into independent tasks, each focusing on implementing a specific feature. Have the model create a branch and then make a pull request for the task. At that point you can review the code fairly easily because you know what the scope of the change is and what specific problem it solves.
It can be very helpful to have the model do research on prior work for steps where you are not sure which approach to take. It is rare that the problem being solved is entirely novel, and agents are great for looking up relevant papers you can review to get a better idea of what is more likely to work. Again, it is important to spend the time to familiarize yourself with the different paths you can take and to pick one consciously.
I would also argue that having a clean architecture with low coupling becomes extremely important when using LLMs. They tend to do best on smaller tasks that do not have dependencies because there is less context to consider. So if you can break up your project into small pieces that can be worked on in isolation, then you can give the agent a task with clear boundaries. That also makes it much easier to review its output as well.
I find that functional style maps particularly well here because it focuses on context isolation and passing state around explicitly. The same tricks that make large code bases manageable by humans also help LLMs for the same reasons. Aggressively controlling the context is a key tactic for using LLMs effectively.
It bears repeating that you never want to give the AI a blank canvas. Always do the work of laying out what the scaffolding should look like yourself. Make sure you intentionally set up the file structure and decide on the components before asking the agent to fill in the blanks.
But even with all these great functional tools, we still tend to tangle two rather different kinds of code together. We tend to mix code that cares what the data means and the code that decides how it travels from one component to another. Traditional software design structures embed the routing implicitly in the function call graph. Control logic often ends up being coupled with the internal implementation details in an ad hoc manner. Breaking things up into independent steps helps control the scope.
Routing logic should be elevated to first class citizenship in the design. State machines are the natural fit for this, since they force the separation of what to do from how to do it. The control flow logic can be largely declarative and expressed as a graph such as the Mermaid diagram I mentioned earlier, while the implementation details live at each step in the flow and become the tasks the agent works on.
Doing these steps forces the agent to work within your architecture rather than inventing its own structure, which largely avoids the problem of it going off the rails. Once you get it to build a diagram and you have reviewed it, you can create the initial project structure based on that.
Use Tests as a Contract
I find it is useful to think of tests as the ultimate requirement doc when working with LLMs. If you define your desired functionality as tests first, you can get the agent to work through them using test driven development until they pass. It will typically do a decent job running the tests and analyzing the failures and fixing its own code to meet the spec. The tests are the contract that the agent works against. Going back to the whole genetic algorithm analogy, these are the selection pressures that drive the evolution of the code.
Having tests up front gives you a solid guarantee that the code is doing what you intended functionally. It is also your best defense against regressions. Without tests, an agent adding a new feature is just as likely to silently break three old ones. Having a contract for the existing functionality avoids that problem.
The types of tests that tend to be most valuable are the ones that focus on the functionality of different components along with end to end integration tests. They do not need to be too granular because issues will get shaken out as the whole workflow gets exercised. I can also highly recommend making storybooks and creating automated testing using Playwright for web apps where the test goes through the entire workflow end to end driving the page as the user would.
Additionally, since tests do not capture performance characteristics, it is helpful to create a benchmarking suite to check performance metrics such as CPU and memory usage. Having one from the start has been very informative in guiding my development of Jolt.
Git is Your Safety Net
You can think of Git like having a quick save in a video game. Every single time the agent gets into a stable state where tests pass and the code looks like it is doing what you want, you should commit that code immediately. This gives you the freedom to let the agent try different experiments or complex refactors. If the agent makes a mess or the idea does not pan out, you do not have to untangle it manually. You just revert to the last good commit and move in a different direction.
I have noticed that if the agent does not get the solution mostly right on the first shot, it is unlikely to make it work properly later. The agent is not going to step back to understand the underlying problem when you point out a bug. Instead, it just adds kludges to fix your specific complaint and the problems tend to multiply as a result. If the original solution was not a good fit, then adding more kludges on top only makes a huge mess that will never work right. If it starts spiraling, then it is time to reframe your problem statement and start from scratch.
A related point is that LLMs make it very cheap to do exploration with your codebase. I mentioned earlier that you always want to understand the problem before you get the agent to start working on it and that is true for code you intend to keep. However, working through a problem is a great way to understand it better. So when you hit a point where you are not sure what to do or which approach might be best, that is when you can spike up different ideas and see how they pan out. Since you have version control, it is trivial to roll back to a known stable commit and try something new from there.
This sort of thing used to take a significant amount of effort, but the barrier to exploration is a lot lower. For example, when I started working on Jolt, I picked Janet as the runtime for it. My rationale was that Janet was superficially similar to Clojure and had a compact runtime while being embeddable. However, I quickly realized that the lack of generational garbage collection did not mesh well with the lots of short lived objects that persistent data structures generate. So I did a bit of research and landed on Chez Scheme instead. I was able to do the whole Janet spike in around a week, and that is something that could have easily been a months long project without LLM use. Similarly, proving out a solution on top of Chez only took a few days to get to the point where it was clear that it would work better.
The Harness Matters
There are a lot of agentic harnesses around, and they all optimize for different use cases. What I found to be important is that the harness meets the expectations of the model and provides the flexibility to customize the workflow to fit a specific project.
In the end, I ended up building my own harness, which I discussed in a previous post here. I spent some time observing how models like DeepSeek and GLM behave within the agentic loop and where they appear to get tripped up. Dirge also integrates proven tricks from existing tools like the official deepseek-harness to avoid reinventing the wheel here. Additionally, I used Janet to provide a plugin system similar to Pi. You can create a .dirge folder per project to place custom plugins there, allowing the harness to evolve alongside each project.
I also spent some time on addressing the common pitfalls that I kept seeing to make the workflow smoother. For example, one common problem is that the model will produce mismatched parens in code. If you simply send the code back to the model, then it is going to burn tokens trying to figure out where the missing paren is. Often, it ends up doing things like writing python scripts to count them. Doing the repair inside the harness solves the problem mechanically so that the model never has to be involved.
Another thing I found was that tracking things using Markdown files tends to be fragile. These files can get stale, which leads them to be misleading and the models do not do a good job keeping them up to date. My solution was to use sqlite as the datastore for the harness and to use it as project memory. I extended that to track tasks as well, modelled on the way beads works. The harness asks the model to create tasks before it starts work and then tracks the active tasks and injects the task being worked on at the top of the context. This helps keep the model focused and continue working on larger features. When the task is done, I use a separate critic role to review it by examining the diff and then provide feedback, which helps avoid cases where the model decides to ship a half baked solution.
I have also integrated some ideas from papers such as Behavior Trees Enable Structured Programming of Language Model Agents, which focus on having the language model act as a leaf node in a larger deterministic control structure instead of trusting it to make decisions end to end. The main idea is to move from treating the model as the whole agent to making it a primitive, which produces a behavior. The workflow is then composed with a small set of classical control structures. The finalization gates like the verifier and critic, along with the code reviewer, form a fixed sequence of deterministic checks the model must clear before a run is allowed to finish. The failure ladder rungs use retry fallback nodes to catch a stuck or failing model, and mechanisms like publish state guard enforce safety constraints structurally. Once you have the right structure around it, even a local model can solve fairly complex tasks competently.
Conclusion
You are still the engineer who is responsible for understanding the problem you are trying to solve and what the project is meant to be doing. Your job is to provide the high level thinking and the architecture while understanding what the correct solution should look like. The LLM is there to save you from the boring and repetitive work like typing out boilerplate and looking up syntax.
The key part to keep in mind is that an LLM is just another tool in your belt. It cannot help you solve problems that are outside your existing expertise effectively. These tools lets you work faster once you learn their sharp edges, but they do not do your thinking for you.
In fact, an LLM on its own is not able to do much of anything useful. It requires its user to have domain expertise to apply it effectively. My ability to build a Clojure compiler using these tools stems from nearly two decades of experience working with the language. I know how it works internally along with what the end state needs to look like and what pitfalls to avoid.
Trying to solve a problem I have no familiarity with would just be me throwing darts at the board. Maybe the LLM will produce the right solution and maybe it will not. I would not be equipped to evaluate that one way or the other.