Agentic software is just software 04 Jun 2025

Friends of mine hear a bit of snark from me when it comes to the recent explosion of hype regarding AI, how it will solve all of the world's problems (while definitely not causing any new ones), and how it's revolutionizing software development. This post is an attempt at something other than snark; it's an attempt to get to the heart of the snark, to a thing that has been clear to me whenever anyone talks about the next wave of AI: agentic AI.

Agentic AI is not new. Agentic software is just a new term to describe an old concept that we used to call: software.

Generative AI models produce output with problems that go by a variety of names, typically either "hallucinations" or "bullshit," depending on the point of view of the speaker. Generative models are non-deterministic machine learning models that recursively predict the next most likely token, running over ginormous vector databases. Given the same inputs and configurations, the same inputs will return different outputs, sometimes correlating to facts, and sometimes not.

Companies hosting and selling AI are generally doing more to self correct for the most egregious of fictions produced by their models, with arguable success. In the race to make better AI, various strategies are being developed, generally by giving the models more data with which to produce output. Provide the models with more data, incorporate a retrieval capability into them, and force them to prioritize new data over old. Ask the model to do more, and hope that the outcome is more deterministic as well as more factual.

An alternative approach is to introduce the concept of agents... but what is an agent?

When searching for agentic AI on the internet of June 2025, most of what comes up are sales pitches, white papers, and other marketing materials. Wikipedia provides a relatively generic article with this line:

Agentic AI is a class of artificial intelligence that focuses on autonomous systems that can make decisions and perform tasks without human intervention.

But what is it really?

I posit the following:

Agentic systems are traditional software architectures: long-running event loops, persistent state, feedback loops, and the ability to interact with internal and external services in the form of APIs and system calls. The key difference to me is that some of those services take natural language inputs and produce natural language outputs. Agents thus require extra capabilities: the ability to call specific APIs for generative models; the ability to provide those APIs with a schema of available capabilities within the agent (basically a list of functions); a parser for natural language responses that recognizes tokens representing function calls; and the ability to retain state for RAG. For some agents a subset of these capabilities may get the job done, for example if business logic is clearly defined and easily parseable, inputs are structured and generative models produce output directly presented to users.

In my experience, generative models are best at summarizing simple language, and worst at generating facts. With the above definition, generative AI can be used for what it's best at: summarize natural language inputs provided by humans to match as closely as possible to structured data, using a schema defining a system's capabilities and its available data sources; use that structured data to initialize an ad-hoc data pipeline in which extra data is retrieved, passed through deterministic code where possible, and back into generative models for summarization where needed.

These types of systems can be reasoned about, tested, and run with both quantitative and qualitative measurements of outputs. For tests, mocks and fakes can be built to provide realistic recordings of the APIs interacting with the non-deterministic parts of the system. The non-deterministic parts may be tested independently, by using other models to summarize the outputs with prompts and configuration tuned to provide as deterministed an output as possible.

I personally think that for workflows requiring long-lived processes and the ability to instantiate ad-hoc data pipelines of linked processes, Elixir (and implicitly Erlang) is the best choice for building high-quality, resilient software. Replace "agent" with "actor," and you have Erlang.

Shameless plug: if you're interested in how Elixir, Erlang, or the primitives provided by those languages could be applied to other your problems, I do software development consulting. Check out my company and get in touch with us.