The biggest trap for leaders right now is letting “AI futurists” influence their tech roadmap. A lot of the AI hype being published on LinkedIn, Substack, etc. makes fully-functioning agents and AGI seem imminent, which creates wildly false expectations.

The truth is, I don’t see many companies building viable agents. Sure, OpenAI’s Operators are impressive … but they’re also a multi-billion dollar company. Any other organization who says they’re building agents successfully is probably lying – or just has a modicum of logic in their workflows (e.g. AI can tag support tickets by priority level and route them to the right person but not resolve the problem for the user or respond to them), which is not the hype that these articles are portraying.

Based on AI’s actual current capabilities, leaders have a choice to make. Do they shell out big for custom solutions that work, try to adapt general AI tools to their specific needs, or neither? Here’s how you should be thinking about it.

The problem: an AI maturity gap

Right now, off-the-shelf LLMs are affordable solutions for broad business use cases, like writing ad copy or analyzing spreadsheets. Because these tools are built to be widely applicable, they work best for common use cases​. But core company processes tend to be very bespoke. There’s a lot of nuance based on each company’s needs, history, and processes.

Serving that level of specificity and complexity currently requires custom AI solutions – which are not only prohibitively expensive for most businesses, they require solid internal infrastructure (more on that here).

There’s a middle piece here that doesn’t exist yet: Customization at an accessible price point.

And in place of that non-existent middle piece is a huge trough of disappointment. Leaders hear  about the incredible things AI is 'supposed' to do, spend a ton of money on an LLM, and then realize they would need to spend a whole lot more to get the value they thought they would.

Because of this AI maturity gap, leaders have 3 realistic options – though none of them are likely ideal:

1. Build an expensive custom solution
2. Buy a generic tool that only gets you 80% there
3. Wait for something better to come along

Build, buy, or wait

When to buy an AI solution

There are two questions to ask before you buy an off-the-shelf LLM:

1. How niche are the tasks you want to use AI for?
2. How vital are these tasks to your bottom line?

You should buy an off-shelf solution when an AI tool already exists that can solve your problem well enough, and building a more tailored version in-house won’t give you enough of an advantage​ to justify the cost.

AI vendors are spending millions of dollars improving their tools. If they’re solving your problem better and more cheaply than you could in-house, just pay for access. Don’t spend hundreds of thousands of dollars developing a chatbot from scratch if a third-party tool can already meet 80% of your needs​.

You should also consider whether you need to own the AI solution in order to benefit from it. If the process is a differentiator for your business, you may want to own the tech behind it.

When to build a custom AI solution

The two questions to ask yourself here are:

1. Will this solution give you a lasting competitive advantage because no one else is doing it?
2. Are you unable to find or buy another solution to the problem this is solving?

When AI directly impacts how you deliver value, and off-the-shelf solutions can’t fully support your needs, that’s when you build​. Think about where you make your money, because that’s where AI is going to make the biggest impact​. If owning or selling the solution is going to be your differentiator, build. If nothing exists today to help you pull ahead of your competition, build.

To give an example, Machine & Partners is working with Section to develop an AI learning coach – ProfAI. The custom AI solution is worth it to Section because it meets our two parameters above:

1. ProfAI solves the problem of students wanting personalized learning content and feedback – which other solutions (staffing human TAs or buying a solution off-the-shelf) couldn’t do.
2. Because few other tools like ProfAI exist (and it’s difficult to build), the solution gives Section a lasting competitive advantage until the market catches up.

Choosing to build is obviously the highest risk, because you’re making a much larger investment and a better off-the-sheld solution might emerge just as you launch yours.

So to be safe, you also need to consider the third option.

When to wait on AI implementation

Here’s the really unsatisfying – and probably controversial – take: If an AI solution doesn’t exist yet and it would cost too much to develop without a meaningful benefit, you’re better off waiting a couple of years.

We’re currently stuck in between inflexible mass-market AI solutions and expensive custom ones – so for some companies, the most strategic move will be waiting for the market to catch up to avoid unnecessary spend and risk.

Here’s an example from a real Machine & Partners client:

They wanted to be able to plug in client details and have AI automatically generate high-quality, personalized sales decks. But if a salesperson only spends 1 hour on a deck 15 times a year, and the cost of building a custom solution for it is $250,000 minimum, it doesn’t make economic sense to build this today.

In two years, this kind of AI-powered deck generation will probably exist off-the-shelf as part of a marketing or sales enablement platform​. ChatGPT won’t do it for you now, and there’s not enough ROI in building a custom solution, so the best option is to keep doing it by hand and wait for AI solutions to advance.

Take off the rose colored glasses

Following AI hype will cause you to make costly mistakes – whether you adopt something generic before realizing it doesn’t do what you expected, or you build something before thinking about the real value it adds.

Agents that can really act autonomously are in progress but still pretty rudimentary, and AI tools that can solve your hyperspecific pain points are probably being mulled over somewhere in Silicon Valley. But don’t let the futurists sell you ahead of the roadmap.

If what you need isn’t possible right now, chances are it will be soon. So if you can’t pin the value on the AI solution, start getting comfortable with waiting as a strategy.