Augmenting experts instead of replacing them with AI | Ep. 3

Episode 3 - Augmenting experts instead of replacing them with AI- Transcript

Matt: Welcome to Data Versus Commerce, where we explore the messy middle between database and doorstep. I’m Matt Johnson.

Floyd: And I’m Floyd Blaikie. Let’s dig in. Most of what you’re hearing about AI right now boils down to one promise: fewer humans and the same output. That is, it seems, the pitch on every analyst call this quarter. I don’t think we’re buying it necessarily — not because we’re overly sentimental about jobs, but because the math doesn’t work. The companies that are actually winning with AI are not the ones who fired their experts. They’re the ones who handed those experts something a bit sharper. That’s the bet that our CEO, Bill Di Nardo, and our chief architect, Joel Farquhar, are making. They’re calling it real intelligence plus artificial intelligence, which I like, because naming a thing is a power marketing move, but the order matters. Real intelligence first, built over years, built over decades in the heads of people who’ve actually shipped the work. So welcome, Bill. Welcome, Joel. I’ll open it up by asking you to tell us in your own words: what is the Pivotree stance on AI, and why are we taking it?

Bill: Let me start by saying we didn’t coin the term “real intelligence.” We’ve been talking about that topic for years around our industry expertise, our domain knowledge. I think increasingly there’s evidence and a growing sentiment that AI without judgment, AI without real intelligence, is not going to be nearly as effective. We’ve observed this. We’ve seen what happens when you give a domain expert a tool like AI — what else they can do, how far they can take it — and it’s been quite remarkable, how much more folks can work on higher-level thinking and take that domain expertise to another level. The AI tools are allowing them to spend less time on commodity-like work. So yeah, we’re huge proponents of it. What we’ve been spending the last five years assembling is experts. The last thing we’d want to do is have those experts exit the building before they actually get a chance to apply that knowledge, leveraging some of the tools that Joel and the team have been constructing.

Joel: I think it really comes down to things changing for sure. We can all feel it. But for us, it’s not about needing fewer people. A lot of it is: how fast can we accelerate value delivery, either for the things we do internally, but especially for our customers? Our customers are starting to expect — not that they need fewer people, or that they’d necessarily pay less for things — but that we’d be able to do more in less time. That’s really the perspective we’re taking. We’re trying to find ways, in the things we do, to accelerate so that our customers can deliver on their roadmaps. They often have three-year roadmaps, but now it’s like, okay, can we really look at that, break it down, and instead of a three-year roadmap, try to deliver some of that value in two years or less?

Floyd: Was there a point in time, or an interaction, or a conversation in particular, where it really crystallized that this was the direction we needed to take — augmentation, not replacement?

Bill: Look, I think that was our entry point, to be honest. We’ve believed that we have some of the best experts in the fields they work in. We saw AI as a tool — it is another tool, a very powerful one — and the question was how we leverage it. But some of the realization, especially in these early days, was that AI doesn’t do exactly what you need it to do all by itself. You need intelligent operators prompting it, directing it, guiding it. And when it fails to do what you need, you need those experts to get it back on track. In some cases, where AI isn’t quite ready, those experts know how to get the job done. So there’s going to be this natural evolution — AI is going to continue to get more powerful, and it’s going to be able to do more autonomously. It’s not there today, so it can’t work all by itself; you need those experts. What’s going to be interesting to observe over the next couple of years is how those experts elevate their game as they push more and more into AI capabilities. What else are they going to be able to do with that extra capacity they have?

Joel: For me, there was a turning point when I first started, because I have a traditional software development background. That’s where I came from, and now I’ve gone into architecture and supported our customers’ architectures, and our internal ones too. But when I started to first play with some of the second generation of code-assistant tools, like Anthropic’s Claude, I started to see the power these tools were going to create, and what we’re starting to call “how do you habituate?” How do you start to think about using AI to solve some of your bespoke problems very quickly? What we’re finding is that scale is growing. We have people within our organization with very specific domain knowledge, and we don’t think that domain knowledge is going away. We specialize in enterprise e-commerce, across the full delivery chain, and those domain experts can now leverage AI to build or design solutions significantly quicker. That’s really where we’re starting to see the power — when we put the right tools in the hands of some of these domain experts, we see real creativity in the solutions. Stuff that would have taken months previously, now we can stand up very quickly. Some of these things end up being throwaway too — we solve a problem quickly using these tools, then move on to the next challenge. So how do you build the mindset of thinking, “How can AI solve this problem faster for me so I can move on to the next challenge?” It starts to become about how Pivotree as a whole does more and scales faster than we’ve been able to before, because we’re not a big company, and scaling has always been a challenge for us. Now we can scale without having to go hire a hundred more people. We can really start to deliver a lot more with a smaller team. But that doesn’t mean reducing the size of the team — it means delivering more with what we have.

Bill: I think that’s been the most interesting observation with some of our clients — when we walk in the front door to perform a particular solution or service outcome for them that we’ve done for years, and we’re already known as experts in some of these areas — systems integration is a great example — when you introduce AI-assisted systems integration, you still have experts asking questions, still guiding the process. But what gets our customers excited is if they can get it twice as fast at a reduced cost. They’re not actually looking to pocket the money — they’re saying, “Well, what else can I do? I have a budget for this year, I’ve got a roadmap — are you telling me I can get more of the roadmap done? I’m happy to spend the same amount of money and actually get more for it.” That’s also been a really interesting observation: our customers want to take advantage of this capability if it means they can get more inside the same budget.

Joel: One of the things you said that we’re really hearing customers talk about is outcomes. Everything we’re doing now is more about outcomes — much less about the billable hour. It’s not about hiring twenty people and charging X amount an hour to deliver a service. It’s much more about: what is the outcome we’re trying to achieve? If you can show me you can get that outcome at a certain time, at a certain cost, with more certainty by leveraging AI, the advantage for us is we can deliver a lot more quickly. In a business like ours, that means we can grow our customer base, but also be more profitable.

Floyd: So there’s clearly a really strong business case for this approach of augmentation versus replacement. When you look at the other side of it — when you hear from folks in organizations who are worried about AI — it goes from a business framing to a very human framing. What is it about that human framing, about the impact on people, that made you believe in this approach versus any other?

Bill: If you look at a lot of the companies reducing headcount and blaming AI, I think to some degree they’re talking about labor they’ve already commoditized, and AI is just accentuating that for them. We’ve never talked about our people as commoditized labor. There’s a difference between valuing knowledge and judgment, and — as Joel said — we’re not a ten-thousand-person organization. We’re a very lean, efficient organization, and we’re looking at how we bring in tools to make our people more powerful, more capable. From the purely human side, a big part of what we’re trying to do right now is reflect on this: if we enable these folks with tools, they become more valuable themselves. If they decide it’s time to move on and go somewhere else, they’re going to have a set of skills that are highly valued. So for us, there’s an element of growing and learning together — let’s upskill folks, let’s give them tools and capabilities that’ll be valued outside of here, let’s try to get the best projects, let’s try to make this the place they want to stay to exercise those skills. But the world is changing, and what we’d really like to do is make sure our people are best equipped for that change — make sure we as a company are in demand, and the individuals in the organization with those skills are in demand.

Joel: Yeah, I suspect a lot of the large companies making big announcements about laying off huge headcounts are trying to get ahead of what they think is coming — betting that if they get rid of all these people, they’ll be able to leverage AI to replace them. I think a lot of them are going to find there are efficiencies to be gained, but we’re still in early stages. We had a plan this year where Q1 was heavily focused on experimentation, and for every experiment where we’ve seen efficiency gains, there are others where we go, “Whoa, that didn’t actually save us anything.” In some cases it took more time. So it’s tough, going through these experimentation phases, to really nail down: okay, yes, that’s a winner, that’s an area where we’re going to become more efficient. But I think it’s going to be a long time before we truly start to see people’s roles being replaced by AI, if at all. I think it’s similar to the rise of the internet — not the dot-com boom, because we know how that ended — but the rise of the internet did create a lot of new jobs, and I think AI’s going to do the same thing. We’re still in that early stage of figuring out what some of those new jobs are.

Bill: Yeah, I think you’re absolutely right, Joel. One of my observations coming out of this is that AI is going to drive inefficiencies out of a lot of these processes. There’s just a lot of waste in technology today — deployments that take longer, cost too much. At the end of the day, as Joel said, people want outcomes. The faster we can get them to an outcome — and the outcome isn’t just “on time and on budget,” it’s “how much did it cost to get that problem solved, how do we get the waste out of it, how do we get the rework out of it.” One of the best examples we’ve been honing in on is how do we get requirements gathered more crisply, faster, more accurately. If we get that right, and move into the development phase, we can spend a lot less time on rework. One of the biggest advantages we’ve seen recently is prototyping — if you can stand something up and show people, “This is what I’m thinking you’re saying — is this what you’re saying?” — we can reduce a lot of inefficiency and get to outcomes quicker. My sense is CFOs are fed up paying for things that don’t work. If they have higher confidence that what they’re spending on is going to work, they’re going to spend more money. But they don’t want to be the ones running the experiments, and they don’t want to take a wait-and-see approach. The more confidence we can give them going into an initiative that they’re going to get the outcome they’re looking for, the more likely business is to spend. I think AI is really going to help our hyper-intelligent, contextual individuals drive to those outcomes quicker and reduce the friction and inefficiency in getting there.

Joel: For sure. Bill hates the word “cool,” so we shouldn’t call it cool, but —

Bill: I don’t think customers buy cool.

Joel: Yeah, I agree — we can’t all be doing cool things, but some of it’s pretty cool. I think one of the most amazing things we see, and we’re seeing customers do this back to us too, is around one of the most challenging things in our business: getting the right requirements. We invented the agile process entirely because waterfall was terrible — you’d gather requirements over a long period, spend seven or eight months developing something, then show the customer and say, “Here it is, it’s all done,” and the customer would go, “That’s not what I meant at all.”

Floyd: Right.

Joel: So instead, we show a little bit over a period of time and correct course whenever we go off. But now, with AI prototyping, customers are coming to us and saying, “This is what we mean, this is how we want it to look, this is how we want it to behave.” They’re using tools to quickly show us what they mean, instead of us relying entirely on words. As we start to harness that capability, we’re even seeing people in our organization — who aren’t necessarily software engineers — stand up something very quickly that solves their particular problem. But then it really needs to come into a process where people with software engineering and architecture capabilities can look at what they’ve prototyped or POC’d and start to productionize it, so it scales, so it’s secure, so it can be monitored, so it can be run as a service — and ideally, in our world, maybe we could monetize it in some way. That’s some of the big transformation we’re seeing happen quickly.

Floyd: We’ve talked a lot about what happens down the road — things get delivered faster, we can prototype more, we can get tightly aligned earlier in the process. But what does it look like within the organization to upskill someone, to augment their existing real intelligence? What are we doing in a concrete way to give people those skills today?

Bill: I think the biggest thing we’ve done is say everybody needs to get on an educational path — basic training and understanding of how to leverage these tools correctly, which ones to choose for which workload. There’s an education process, and then there’s a training process. Training is actually hands-on — having mentors, having individuals who are a little further advanced, helping people work through and solve challenges. Because, as I said, right now AI isn’t at the point where you drop in your query and get a perfect answer every time. The more complex the challenge, the more help, guidance, and judgment it needs. So we’ve really tried to advance a collection of individuals in the organization to be our real leaders in practical application, and now we’re trying to get the rest of the organization exposed to: where do I apply this, how do I use it, and when I run into a roadblock, who can I go to for help? It’s a company-wide education process going on right now.

Joel: I think one thing Bill maybe doesn’t speak to — and he’s probably the biggest person responsible for it — is a culture shift. We’ve allowed, in some respects, maybe more leniency than a lot of organizations are giving. My wife works for a very strict organization where it’s “no AI allowed anywhere.” We’re more like — we’re really encouraging our employees to experiment, to see where they can find efficiencies, because Bill and I were both very early adopters of testing and experimenting as we learned. The only way you’re going to figure this stuff out is by trying different things, different ways. You’ve got to start learning the capabilities of what it can do. If you’re trying to analyze a large data set, it’s a lot faster to ask Claude to analyze it for you. I don’t tend to use it to write emails and that kind of stuff, because I don’t find it any more efficient, but if somebody sends me a long doc, it’s a lot easier for me to ask AI to summarize it first. Or if I’m trying to recruit and hire, it’s very efficient for me to write a small skill that analyzes a resume against a job description and gives me good insight. You start to learn those tips and tricks where you’re using it in almost everything you do over the course of a day. The culture we’ve created is that we really want people trying to work it into everything they do, every single day.

Bill: Yeah, I think that’s a really important point. A lot of folks started off treating it like a simple search query — they treated Claude like Google, for lack of a better analogy. What we’re really trying to do is get folks to go beyond simple question-and-answer, to understand context prompting and getting all of your thinking correct — putting the guardrails up and really driving to the outcomes you’re trying to achieve in more complex situations, rather than just, “Can you tell me the best way to build this new thing I bought, because the instructions are terrible?” It can do some really neat things at the most basic level, and most people know how to use it that way. But as we try to layer things up to solve more complex problems, there’s guidance and help people need in order to get more from the various LLMs. As Joel said, there’s a culture change going on — not just saying, “Hey, we accept it, we encourage it, we require it.” We want every one of you to be a strong, thoughtful, engaged user of AI, and that means constant education, training, and access to people — demonstrations, just showing people the art of the possible. That often triggers, “How did you do that? Can you show me how to do that?” We’re really trying to drive a culture of curiosity, which ties to one of our core values: adapt relentlessly. I think we’re seeing that in spades right now, really pushing folks to adapt to this remarkable capability that AI brings.

Floyd: Now that you’ve introduced this cultural shift and gotten folks on that educational path, is there one thing — a use case, a reaction — that’s been really surprising to you?

Bill: Are you asking about our employees or our customers?

Floyd: From employees, folks internal to Pivotree — anything that’s really surprised you? I won’t say “cool,” I know you don’t like that, but has anything been surprising?

Bill: I’ll tell you the first thing that surprised me — something Joel highlighted a little while ago — is how quickly the most senior levels of our organization became power users. We had folks at the highest levels of the organization working on solving big, complex problems and doing things that previously they might have needed IT or someone else to help them with. They were driving aggressively into deploying these new skills and capabilities, and that gave me a lot of confidence, because if the senior folks are becoming power users, that’s going to cascade down through the organization. There was always that worry — that risk — that it would be the young folks who really embrace this, and maybe our senior, older people would find it difficult to learn. It was the exact opposite. We’ve had virtually no resistance from our senior people, and we’ve got to keep working on showing folks this path forward and how it’s going to benefit them. That was probably the biggest, most interesting takeaway — buy-in from the top.

Joel: For me, it’s probably this: we have some power users, like Bill said, who have very strong business sense or a very strong understanding of a challenge they need to solve, but they don’t necessarily have the technical capabilities. We’re starting to see that wall — the “no technical capabilities” wall — start to crumble, to the point where I’ve created a matrix that tells us who’s going to benefit the most from AI. I put two things on the axes: one is business or domain knowledge, the other is technical capability. If somebody has very high domain knowledge and very high technical capability, we’re really seeing those people get, like, a tenfold multiplier on how much they can produce — I call them multipliers. But we’re also seeing people who have that strong domain knowledge and not as much technical knowledge — I call those people visionaries. They have ideas, but they never knew how to bring them to life, and sometimes technical people can be gatekeepers — we say, “No, you can’t do it that way,” and sometimes we’re horrible about it. AI is starting to knock down that wall. For me, it’s amazing — I think, “That’s cool, that’s amazing that somebody was able to bring that to life. Now how do I help them get it to where it needs to be so we can reap the full value of it?”

Bill: Yeah.

Joel: On the flip side, we’re seeing some very technical people — high technical knowledge but without a strong understanding of the business domain — struggling. So one of the shifts we’re going to have to make in our organization is that our technical people, if they came from a purely technical role, are going to have to start learning more about focusing on outcomes, understanding the business domain, thinking about what the customer’s trying to achieve — because that’s what’s going to move them to becoming a multiplier, truly getting value out of AI. If your intention is to stay purely technical, you’re probably going to have some challenges over the years. So we have to train those technical people to understand the business objectives, and then help our technical people move the domain-knowledge people up in their technical knowledge too.

Floyd: That’s —

Bill: — a really important point you’re making, which is: it doesn’t matter whether you’re a senior technical person or a senior business person, AI is going to give you increased capabilities. That’s what we’re seeing — our most senior people, senior architects, senior business leaders — AI is making them more productive, allowing them to get further without short-term technical assistance. It’s freeing, it’s democratizing in some respects, in order to get to outcomes. As Joel said, a senior technical person and a senior business person generally have a really good idea of the outcomes they’re trying to get to, which is why AI can really amplify their capabilities. I think the biggest concern I have — and it’s something Joel hinted at, something we’ve got to figure out and take responsibility for — is: if those senior people are going to be in high demand well into the future, how do we make sure the young folks coming out of school, who don’t have this judgment yet, don’t have domain expertise, get that training? Some of the stuff they used to do coming out of school was lower-level activity — coding, QA, some of the more commodity-like processes — that’s where they cut their teeth and started to build judgment. If we had one concern about what the next ten years are going to bring, it’s not about our experts, because they’ve got domain expertise — it’s what the next generation is going to do to break in, because there’s not going to be a lot of work at the lower end of that hierarchy anymore. That’s something we spend a lot of time talking about now — what’s our ethical and moral obligation to make sure there’s space and room for young people to get into the industry? I don’t think we have a good answer yet.

Floyd: And that’s where they get a lot of that business context too, kind of by osmosis, as they’re doing that — I don’t want to call it lower-level work, but the type of work that’s more replaceable by AI. So yeah, it’s a good conversation to be having, and a lot of people have a lot of anxiety around it. I want to dig more into Joel’s matrix, and I want to talk about some of the real AI projects getting off the ground — that’ll have to be a next-time thing. Thank you so much for joining us today. Bill, Joel, any parting thoughts?

Bill: I’d just go back to the original question about our point of view. Our point of view is: we’re not automating our people out. We’re building them up so they can do only what they can do, at a scale that wasn’t possible before. This is really about enabling real intelligence to hyperscale.

Joel: My parting thought is: we’re still in early days, and it’s probably the most chaotic time I’ve experienced in my career — just how fast things are moving. I think this creates so much opportunity for everybody. As we try to get our arms around where the world is going, I’d also be cautious — there are a lot of experts claiming to know where the world is going. Bill and I are both hesitant to make predictions too far into the future. It’s very difficult to do, and I’d caution people against trying to predict where this is all going, because it’s changing too fast.

Floyd: Nobody knows, which is the scary part, but also the most exciting part.

Bill: It is. It is the most exciting part.

Floyd: Thank you so much. We’ll catch you next time.

Joel: Right.

Matt: Thanks for tuning in to this episode of Data Versus Commerce. New episodes drop weekly. So if you’re responsible for any part of how products get from a database to a doorstep, subscribe now on Apple, Spotify, or wherever you listen.