Start small before you sign a five-year, multi-agent deal | Ep. 8

Episode 8 - Start small before you sign a five-year, multi-agent deal - Transcript

Matt: Welcome to Data Versus Commerce, where we explore the messy middle between database and doorstep. I’m Matt Johnson.

Floyd: And I’m Floyd Blaikie. Let’s dig in. Okay, I am here with a very interesting character. His name is Anatolii Iakimets. He is the Director of Product Marketing at Kibo, and he tells me that his last name is harder to pronounce than his first one. So I’m curious, how did I do? Did I get it right?

Anatolii: It is pretty, pretty good — spot on.

Floyd: Okay, that’s great. I am really excited for today’s topic because we’re going to talk about AI agents, but in a way that probably not a lot of folks have talked about them before — we’re going to dissect them like it’s high school biology. But first, Anatolii, can you just tell us all who you are, what your background is, and what you’re doing these days at Kibo?

Anatolii: Yeah, I lead product marketing at Kibo with a focus on B2C commerce. I’ve been in this space for over 12 years, predominantly in commerce and telecommunications. My first adventure with artificial intelligence was around 2015, ’16, when it was still the age of machine learning and neural networks, and everybody was super hyped about it. So I have a history with AI, and it’s near and dear to my heart, so I’m glad to be here.

Floyd: Amazing, and that’s the hot topic. I don’t know if you’ve heard — everyone’s talking about AI, maybe too much — but if we dig into how that’s really playing out in the e-commerce world, everybody’s talking about, of course, agentic commerce. That’s been the subject of every press release from Google for months now. And what that means is just deploying agents into your commerce workflow, right? Does it get more complicated than that?

Anatolii: Yeah, I think everybody is trying to figure out what they can do with agents, and there are a couple of areas. One is pretty obvious: everybody’s trying to make their customer experience on the front end cheaper and more efficient in terms of customer service agents. You see commerce companies doing it, customer service companies doing it — basically trying to reduce time to resolution and resolution cost. At the same time, when you go more to the back end, back office, and operations side — how do you actually make your employees more productive? This is where everybody’s still trying to figure it out, because for commerce it’s not as obvious as it is for some other areas, like coding or software engineering.

Floyd: So what is an agent? I’m mostly not playing dumb — I think I’ve got a pretty good sense of it, but people talk about it in different ways. They might talk about Claude like he’s their best friend. They might just say “AI” to refer to LLMs and various tools. Some people might distinguish between AI as a concept and a large language model as a tool. But what is an agent? Can you break that down for us, super clearly?

Anatolii: Yeah, and feel free to stop me if I talk for too long. But in a nutshell — if you think about an agent, there are two distinct parts to it: the LLM, and what people call the harness, or scaffolding. The LLM is the straightforward part, whether it’s Anthropic or OpenAI or any other frontier lab, like Google, for example. Or it can be an open-weight model like Kimi K2, or any other model — there are a bunch of others, DeepSeek, for example. Then there’s this other part, the harness, which is all the code that makes the agent work — basically the software, in the same sense as any other application.

It has webhooks, it has events, it has what’s called a heartbeat, which allows the agent to run — for example, every 30 minutes — evaluating what the outstanding tasks are. It has scheduled jobs, it has the ability to pull data from sources, it has tools to do search. So, to give you an example of what the harness means for you as a user, in terms of experience: when you go to an agent and ask it to write a blog post, you’ll see it splits your request into a task list — “I need to go search for it, I need to pull data from the source, and then after I write the draft, I need to go back and check whether the draft actually matches the prompt.” And sometimes it’ll say, “Well, this doesn’t work well, it doesn’t answer the question, so I need to do one more iteration.” All those things that direct the LLM on what to do — that’s the harness, or the agent.

Floyd: Okay, so if I were to slice an agent in half, about half of that is the model — the thing we’re all really familiar with, right? We all have our own preferences, whether it’s Claude, ChatGPT — and then the harness is the actual stuff around it, the instructions that tell it what to do.

Anatolii: Yep.

Floyd: So for most agents, it’s probably true that the model is kind of interchangeable. If I’ve got an agent and I’ve decided that best friendship with Claude is over, now I’m best friends with ChatGPT — that’s simplifying it considerably, but you can drop different models into these agents, whatever they’re doing in your workflow. It might make a difference, but the harness — the code around it — stays the same.

Anatolii: Yeah, and basically, if you think about it, the models are becoming a commodity. It’s kind of a funny situation, because from one perspective you have all the brightest minds in the world — people with tons of PhDs — working on these models to make them better and smarter. But at the same time, the gap between models shrinks, and as a user you can pick whatever model, and they have similar costs — for you, it’s almost a commodity. The switching cost for models isn’t that high for you as a user. People have preferences, sure, but it’s become almost a commodity — even Sam Altman has talked about it, how models will become the intelligence on tap. Meanwhile, the harness becomes the differentiator. The agent becomes the key differentiator, because the model can be anything, and how well your agent runs will depend on the harness — on the software you write around the model.

Floyd: Okay, so as e-commerce leaders are thinking about introducing agents into their commerce workflows, you’re saying the harness around the model is the critical decision point in selecting that agent. So it sounds to me like this is just another software decision, like the ones we’ve been making for over a decade, right? We need to choose what set of instructions — whether a human wrote them or another AI wrote them — are going to tell the model what to do. So, thinking of it that way, where a lot of these agentic commerce decisions are kind of like platform decisions in a trench coat — give me an example of a type of agent an e-commerce leader might select, and what they might be thinking about when trying to make that selection in the best way possible.

Anatolii: Yep, so you’re absolutely right, and this is also an interesting situation, because if you think about it, how agents in the e-commerce space differ from agents in software development is actually very different. In software development, you’re not necessarily concerned with exactly how the code is written — as long as it works. Of course you have requirements, but as long as you meet them, you don’t care how the code is written. The same way, people who use agents to write text don’t necessarily care about the specificity of the text, as long as the idea is conveyed. And this is good, because it’s the inherent nature of the models — they’re not deterministic. Every time you ask a question, you get a slightly different answer, and that’s fine for coding — if it wrote the function slightly differently and it works, you still don’t care. If it wrote the text slightly differently and it still conveys the idea, you don’t care.

Commerce is a completely different world, because if you think about the commerce stack, basically everything is deterministic. Your order has a specific price — you cannot just—

Floyd: Hmm.

Anatolii: —whatever. Your taxes have to be specific. You can’t just say, “That’s good enough” — they have to be accurate to the point.

Floyd: They get mad about that.

Anatolii: Yeah, the IRS will not be happy if your taxes are just about right.

Floyd: Yeah — just, “It’s close enough.”

Anatolii: Well, yeah — same with products. The customer wants this specific product. If it’s a slightly different size of T-shirt — one size larger, they won’t be happy; one size smaller, same thing. The only thing around commerce that may not be deterministic is something like product descriptions, and even then it’s okay. But the vast majority of the data is pretty structured, and you want the agent to be very specific and very precise in what it can do.

This is why it boils down to: you should almost treat an agent in e-commerce like a platform choice, like an application, where you want it to perform certain functions very precisely and with a great degree of accuracy. And going back to the second part of your question, in terms of an example — say a customer calls a sales rep and asks about their order. The sales rep pulls up the window and asks the agent, “This customer asked me, ‘Where is my order?’ Here’s the order number.” The agent goes and pulls the data, and of course you want it to be accurate. You want to make sure that if the agent says, “The order will come tomorrow by 2:00 PM,” that it actually comes tomorrow by 2:00 PM — not two days from now, and that it doesn’t mess up the address or whatnot.

This is the important part, and the big question is: how do you make this happen? What’s the difference between making this happen and not making it happen? It goes back to having the harness be able to talk to different systems, because agents don’t just magically go and do stuff — they need to talk to a system via an MCP, a Model Context Protocol server, or via an API. But in the end, it’s the same integration question you’d have if it weren’t an agent at all, but another system — the same data question. You want to make sure your data is available and accessible to the agent, and that it knows how to read the data. For example, if you just tell an agent, “Well, the status of the order is five,” it doesn’t know what that means. It can go read documentation about it, but it’ll burn tokens and be inefficient. So the point is, it’s about the agent knowing about the systems, knowing the data, and having the proper instructions on how to interpret that data — which goes back to the harness.

So in this case, if somebody thinks, “Oh, we’ll just buy the commerce platform, it has this agent, and it’ll automatically talk to order management” — that’s not going to happen. You’ll have to do the integration work no matter how you approach it. This is the challenge, and everybody needs to get comfortable with it: you need to treat the agent as an application that needs to be properly integrated with all your systems, with full access to the data, and the ability to work with it properly.

Floyd: It sounds like some of this is almost like hard-coded context, right? Because we know that AI tools like Claude don’t work super well if you don’t give them the right context. So part of this is giving it context — but you said if you don’t do it properly, you can burn a massive amount of tokens. I think we all saw the story about Uber burning through its entire AI budget in four months, and having to put a hard limit on how much folks were using it and the ways they were using it. Is part of the agent-selection consideration set how much this is going to cost, and how you can increase your ROI — get the most bang for your AI buck?

Anatolii: Yeah, this is true, and it brings us to the next important part of agents: the model switch, and bring-your-own-model. The reality is every single company is moving toward a usage model. Maybe the consumer side will stay on subscriptions with hard limits, but everyone in enterprise and B2B is moving toward usage-based pricing. We saw Copilot from Microsoft, earlier this year, move from subscription tiers toward a usage-based model — and in general, that’s the only way this goes, and prices will probably go up.

In this environment, nobody’s going to pay $500,000 a month — a commerce vendor or customer service vendor isn’t going to eat that cost for you, this insane amount of money, because they need to make money off of you. The only way for you to have full control in this situation is to be able to switch models, because you never know who will raise prices first. Will it be Anthropic? OpenAI? Someone else? What will the terms be, how will it happen? You’ll want the ability to switch toward open-weight models, which are inherently cheaper. So you want this optionality — and that’s why we believe at Kibo that bring-your-own-model is extremely important. That’s the only reasonable way for anyone to proceed.

Floyd: Okay, so we’re thinking about the harness, or the instructions around the agent. We’re thinking about models being virtually interchangeable — I know there are significant differences between them, but functionally they’re interchangeable. We’re thinking about making the best decision for cost, and keeping things efficient and effective. So if you’re a commerce leader, whatever part of your digital supply chain you’re thinking about introducing agents to — what’s the number one thing you need to know to make a good choice, that you probably don’t know today?

Anatolii: I’d say there are three aspects a commerce leader needs to keep in mind when evaluating agents. First is composability, or flexibility — I’ll elaborate on each. The second is simplicity, and the third is the ability to start small and scale. The reason is that everything in AI is evolving rapidly — even what we’re discussing now might not be true in six to twelve months. We never know. But it is true at the current moment.

In terms of composability and flexibility, it’s about being able to change or choose your model. It’s about being able to switch to a different agent, theoretically — so the software needs to have MCP exposed for another agent to use. You want those kinds of options, which boil down to: have the ability to switch your model, make sure all the software has proper endpoints for agents to use, like MCP, APIs, and agent-to-agent protocols — because you might want to use your own agent in the future.

Simplicity goes back to the fact that everyone has a very different agentic strategy. If you look at the vendors, some of them position themselves as having 20 agents — and I’m not exaggerating, I went to one vendor’s webpage and they literally had 20 agents. It’s very hard to navigate this space, because one piece of software has 20 agents, another has 6, and yet another — like a CMS — has another 10. Suddenly you have 15 agents you need to understand — how they interact, how they work together, what use cases go where — and it can become overwhelming.

In terms of simplicity, that’s what we envision at Kibo: you have one single agentic experience, basically one entity to deal with, and of course there are multiple agents in the background. But you want to simplify how your organization deals with agents, unless you’re an extremely large company that already has internal development teams working on agents. You want to simplify so you can measure the ROI, know the use cases, and make it easy to use.

Then this “start small and grow” piece is kind of an extension of that. You want to start small — ideally one agentic experience for everyone to use, so there’s one agent people can go ask a question and deal with it. But then in the future, you want to make sure all use cases are covered. This is how we think about it at Kibo, and I think it’s a pretty good way to think about it. We don’t think about agentic in terms of splitting it into specific functional agents — like a supply chain agent or a returns agent — because sometimes the borders blend, depending on what you want to do. An agent might interact with multiple systems.

We think about it in terms of intent. There are five functions we have. Engage — that’s when the shopper interacts with an agent, think of a customer service agent. Then there’s the explain function, where if you ask, “What’s going on with this order, this product, this promotion — explain it to me,” no matter what, it gives you a descriptive answer. Then there’s a configure agent, where you ask it, “I want to change this” — basically, you want to make a change to a system. Then there’s an analyze agent, where you ask it for a data slice and it provides you with analytics information. And finally, there’s the optimize agent — the boss of all agents — which runs in a loop to optimize your operations. This is kind of an autonomous loop agent.

That’s the way we’re thinking about it, and we think it’s a pretty beneficial way, because as a commerce practitioner, it’s very easy for you to understand: “Okay, I want an agent that will deal with my customers” — that’s an engage agent, simple and easy to understand. Or, “I’m not sure I want my employees making changes in the system with an agent” — okay, you don’t need the configure agent, you can focus on just explain. Start small, see how many queries employees generate, do research on what questions employees want to ask, and expand later.

This is what we see as well — when we have some customers deploying it in beta mode, we already see the explain function is the most popular, because people are still kind of scared about using an agent that “will do whatever.” But when they have a function that just answers their questions the same as a Google search, they’re much more comfortable, much more likely to use it. So, to summarize my long-winded answer: make sure you start simple, start small; make sure the vendor has all the capabilities if you want to go larger and increase scope; and make sure you have the optionality to go composable — to switch agents, to switch models. Because the last thing you want is to end up in a situation where you go to a vendor, sign a three- or five-year deal with a lot of commitment across multiple agents, get locked into a specific vendor with a specific LLM, and then realize half your employees don’t use it, it’s burning more tokens than you wanted, and you don’t have an ROI. That’s the biggest risk you might end up with in agentic.

Floyd: Okay — BYOA, coming to a platform decision near you. All right, this is super illuminating. I love that, as cool and futuristic and high-tech as this all is, it really boils down to the same kinds of decisions we’ve all been making for a long time, and if we use those skills, I think we’ll be in a good spot. Anatolii, thank you so much. If people want to learn more about you, what you do, find you online — do you want to tell us where to find you, or would you prefer to stay anonymous and hidden?

Anatolii: Yeah, I’m on LinkedIn, although it won’t be easy to find, because my LinkedIn tag is my name — last name — which won’t be easy to type in.

Floyd: It’s easier than saying it, maybe — I’m not sure. We’ll link it, we’ll link it on the episode.

Anatolii: All right, thank you very much, Floyd, for your time. I’m so glad to be here.

Floyd: Thank you so much.

Matt: Thanks for tuning in to this episode of Data Versus Commerce. New episodes drop weekly. So if you’re responsible for any part of how products get from a database to a doorstep, subscribe now on Apple, Spotify, or wherever you listen.