I am borrowing Douglas Hofstadter’s tender and complex phrase to write down some thoughts that have been circling in my head over the last several months on, well, LLMs and AI agents.
The Learning Loop #
I derive a lot of self-worth from my profession as a software engineer. The most rewarding occurrences in my work are:
- appreciation and acknowledgment of my work by co-workers
- the joy I feel when understanding something new, be it while solving a problem or realizing I have understood or discovered (for myself) a new concept
- building something cool that works, and sharing that accomplishment with my co-workers
All of these types of events have come under a perceived threat recently: praise started to feel hollow. Do I deserve the praise? The chatbot did the work. The joy of understanding turned into frustration because of the LLM’s infuriating aptitude at solving problems and its irritating way of explaining what it did.
I am still working on what it means to get a thumbs-up for work an AI agent did in large part; maybe it was a somewhat over-pronounced reliance on validation in the first place.
But I found a satisfying method for the learning part. The LLM chatbot is an infinitely patient teacher. But that was not enough, at least for me. Having the solution or explanation given by the LLM made me feel left behind: it uses concepts I don’t follow in a language that reads as alien. And sometimes this made me stop trying to understand, in frustration and even anger. Like a misattuned mother struggling to teach her child something new, the LLM is unaware of how to explain something so I can grok it.
And then I read in an online forum about somebody’s experience with the Socratic method and LLMs.
I tell the chatbot I want to learn about X and to use the Socratic method. It starts asking me questions, and I have to look for answers. There are often several rounds, a loop that starts at a high level of basic understanding and goes into more and more detail. The LLM gets a sense of what I understand and can adjust its explanations and questions. It has helped me understand a lot, from complex firmware update procedures to alien RTL constructs. And often it brings the joy of understanding new things and the feeling of mastery back.
The Physical World Loop #
The Socratic method creates a loop of new information, feedback about what I understood, and updated or additional information.
Similar loops exist in software engineering: we take the intent, like fixing a bug, and our understanding of the code, update the code, test the code, and, if it fails, update our understanding, jump back to updating the code, and so on. This works really well; unit tests provide useful feedback.
And I have come to appreciate that coding agents are faster at running this loop. They ingest a lot of context and can generate code updates very quickly. And with the right feedback, they can iterate at superhuman speeds and reach impressive results fast.
This is somewhat harder in the physical world when building things, where getting feedback can mean taping out and fabricating a chip (several months), fabricating and assembling a PCB (several weeks), making a 3D print or running a simulation (several hours). Or, when optimizing things like power consumption, signal integrity, runtime, noise, cost, and RF performance, the agents need to sense and control the physical world.
I started spending a lot of time building and optimizing strange agent loops that can interact with the physical world. A USB hub with an API, relays the model can use, logic analyzers with MCP servers to observe data buses; source measurement units with APIs to measure power.
Building these loops has taught me a lot and helped me expand my agency from mostly software systems to the physical world. It feels rewarding, and it brings back the joy of understanding and discovering. One almost euphoric moment I experienced recently was writing code to describe objects in OpenSCAD and having them materialize in my 3D printer.
And I have been learning to make PCBs and have them manufactured, something that is quite far beyond my original set of skills. I believe the combination of the agents’ ability to design custom PCBs and their proficiency at building great interfaces by writing RTL for small FPGAs with lots of peripherals will make the physical world loops capable. This might be an important milestone along the way to models building tools to observe and act in the physical world.
In a way, the skills of the LLMs have given me the confidence and have reduced the activation energy required to go beyond writing software towards building physical things. And I look forward to learning and building more.
Imperfect Loop #
Sometimes, though, these loops are flawed, and the LLMs reveal their spikiness and are unmasked as the smartest idiots I have ever worked with. No, my real-time system cannot take a several-second-long “break” to accomplish a specific low-priority task. No, hundreds of milliseconds of latency are not acceptable when processing audio. The bug you found and spent 2 hours fixing is irrelevant because its precondition never occurs.
They still miss things a human would not and occasionally run full steam ahead with superhuman competency in the wrong direction or with the wrong assumptions. When this happens, I believe there is information or context missing or ignored, or the agents are overly cautious or focus on something irrelevant. A bit of a lack of common sense, maybe.
So the next logical step for me is to focus on the LLM harnesses or the models themselves. I look forward to building loops that improve their capavilities. And even if the next model generations will be more capable still, I am excited to learn more about these types of loops.
Strange Loop #
Interacting with LLMs has been a bit like a mirror that showed me what I value in my work. This is something I never had to think about before. Now that I know, I can think about it. And I can make more deliberate decisions about what I work on, how I work, and what I learn.