What an investment in AI infrastructure really costs

Essay #004 · 2026 · Reading time 5 min · Field: Investment · Market: Mittelstand

An offer sits on the table: €340,000 in the first year, €180,000 per year after that. AI infrastructure for a Mittelstand company: vector database, model hosting, orchestration, monitoring, two developers for the integration. The numbers are cleanly calculated. The business case is plausible. The investment is supposed to pay back after 14 months.

The managing director reads the paper — and does not sign. His gut tells him: this calculation is not complete. The visible costs are in there. But the costs that only arrive after the first year are missing.

He is right. This essay describes what those costs are and how you make them visible before you sign.

Four cost categories: one visible, three hidden

Every investment in AI infrastructure has four cost categories. The first one is in the business case. The other three are usually missing, not by intent, but because they are hard to put a number on.

Category one: visible costs. License fees, cloud hosting, model costs, development, integration, external advice. A defined amount per year, easy to calculate. The companies I know have this part well under control.

Category two: rebuild costs inside the organization. The new infrastructure changes existing processes. That costs money, and not once, but over years. Teams need training. Workflows need adjusting. And someone has to make sure internally that the new ways of working are actually adopted.

From my projects: for every €100,000 of visible infrastructure cost, roughly €40,000 to €80,000 of rebuild cost arises in the first year. That is a figure from experience, not a study. It does not appear in the business case because it spreads across many departments and gets booked there as normal operating cost.

Category three: switching costs (lock-in). What does it cost to leave the system again in three or five years? The deeper the integration, the more expensive the switch. That was already true for classical enterprise software. With agentic infrastructure it is more pressing, because architectures evolve faster and switches come up more often.

Again a figure from experience: if you tie yourself deeply to one vendor today, plan for a switch in three years to cost 30 to 70 percent of the original investment. That sum has appeared in no business case I have seen so far. It only becomes visible when the switch is due, and is then often so high that the switch gets postponed.

Category four: the cost of waiting. Not investing costs money too. Competitors build a lead. Employees use public AI tools with company data because an official solution is missing (so-called shadow AI). That creates data-protection and security risks that can later become more expensive than the investment itself.

These costs are the hardest to quantify. They still belong in the calculation.

The opening example, calculated in full

Back to the offer: €340,000 in the first year, €180,000 after that.

Add to this:

• Rebuild costs in year 1: around €150,000 (a factor of 0.45 on the visible costs)
• Rebuild costs in years 2 and 3: around €60,000 per year, decreasing
• Switching-cost risk in year 4: €150,000 to €250,000, to be reserved as a contingency
• Cost of waiting if you do nothing: roughly €50,000 to €200,000 per year

Over three years, the 340/180 calculation becomes a 490/240 calculation, plus the switching-cost risk you have priced in.

That is no reason to reject the offer. Many of these investments still pay off in the complete calculation. But they pay off differently. The payback comes after 24 months, not 14. And the alternative is no longer “do nothing” but “do it differently.”

Three questions for your IT lead

Before you sign a business case for AI infrastructure, check whether it answers these three questions:

How much rebuild work does this system create in the departments? Not once, but over two years. Whoever has no answer to this is calculating only the technology, not the organization.

What does a switch cost in three years? In my conversations, this question is often seen as out of place. It is not. It shows how dependent you are making yourself on one vendor.

What would the alternative be? Not “do nothing,” but concretely: a different investment, a different architecture, a different timing. If no convincing answer comes back, the business case was developed without a real comparison.

What follows from this

Investments in AI infrastructure are not to be avoided, but they must be calculated differently from earlier IT projects. On top of the visible costs come rebuild costs, switching costs, and the cost of waiting. Whoever calculates only the visible costs decides on a foundation that is too friendly.

That does not mean: wait. It means: know what you are signing. The decision of whether, how, with whom, and when to invest in agentic infrastructure is one of the most important strategic decisions of the coming years for your company. Do not make it solely on the basis of a business case that leaves out three of four cost categories.

The concrete next step: hand the business case back and have the three questions answered in writing. Only then do you sign — or you deliberately do not.

— Axel Roth