The visible costs of an AI infrastructure investment are the tip of an iceberg. Organizational rebuild, lock-in risk, and the opportunity cost of waiting dominate the real bill. Three questions expose overly optimistic business cases.

What an investment in AI infrastructure really costs

Essay #004 · 2026 · Reading time 6 min · Field: Investment · Market: Mittelstand

An offer sits on the table. €340,000 in the first year, €180,000 annually thereafter. AI infrastructure for a Mittelstand company — vector database, model hosting, orchestration, monitoring, a team of two developers for integration. The numbers are neatly calculated. The business case is plausible. With a conservative assumption, the payback is estimated at 14 months.

The managing director reads the paper. He mentally nods. And he does not sign.

He does not sign because something inside him says: this calculation is not complete. The visible costs are listed. But the hidden ones — those that do not appear in the first year, but dominate the investment when viewed over three years — are missing.

He is right. This essay tries to describe what sits in his stomach before it sits in a spreadsheet.

The four categories of hidden costs

Every AI-infrastructure investment has four cost categories. The first is visible. The other three are hidden — not secret, but structurally under-lit in business cases.

Category one: visible costs. These are the ones on every slide. License fees, cloud hosting, model costs, development effort, integration, external advice. A figured sum per year. Easy to compute. Most companies are well prepared here.

Category two: organizational rebuild costs. Costs that arise when existing processes must adapt to the new infrastructure. Not one-off — ongoing. It starts with training the affected teams. It continues with adjusting workflows. It ends with the political work inside the company to ensure new ways of working are actually adopted.

In practice: for every €100,000 of visible infrastructure cost, roughly €40,000–80,000 of rebuild cost arises in year one. This number is not scientifically derived — it is a rule of thumb from experience. It appears in no business case because it is distributed across many departments and booked as normal operating cost.

Category three: lock-in costs. Costs that arise when the company wants to switch the chosen system in three or five years. The deeper the integration, the higher the switching effort. With classical enterprise software this was a familiar topic — with agentic infrastructure it is more acute, because architecture evolves faster and switches come more often.

In practice: anyone who today enters a deep integration with a specific vendor must assume that a switch in three years costs between 30 and 70 percent of the original investment. That sum appears in no business case. It only appears when the switch is due — and is then usually so high that the switch is pushed off.

Category four: opportunity cost of waiting. Paradoxical but real: not investing has costs. While you wait, competitors move ahead. While you wait, employees get used to shadow AI — they use public tools with company data because official infrastructure is missing. While you wait, data-security risks arise that will later be more expensive than the investment you shied from today.

These costs are the hardest to quantify. But they are real — and those who do not include them in the calculation calculate incompletely.

A realistic full picture

Take the opening example: €340k in year one, €180k thereafter.

Add:

• Rebuild cost year 1: about €150,000 (factor 0.45 on visible costs)
• Rebuild cost year 2–3: about €60,000 per year (tapering)
• Lock-in risk at switch in year 4: €150,000–250,000 (reserved as contingency)
• Opportunity cost of inaction: estimated €50,000–200,000 per year (hard to pin down)

The picture changes. The original 340/180 calculation becomes a 490/240 calculation over three years, plus a priced-in lock-in risk.

That is not tragic. Many of these investments are still worthwhile in the more realistic calculation. But they are worthwhile differently — and the decision logic shifts. The payback is not 14 months but 24. The comparison alternative is no longer “do nothing” but “do differently.”

Three questions to ask your IT lead

Before you sign a business case for AI infrastructure, check whether the following three questions are answered:

How much rebuild work does this system generate in the affected departments? Not one-off — across two years. If there is no answer to this question, only the technology is being calculated, not the organization.

What does switching this system in three years cost? This question is almost always felt to be inappropriate. It is not. It is the only question that makes the difference between an investment and a hostage-taking visible.

What is the alternative of “not doing this”? Not doing nothing — but concretely: which alternative investment, which alternative architecture, which alternative timing. If your IT lead has no convincing answer here, your business case was not developed in open competition.

The actual point

AI-infrastructure investments are not to be avoided per se. But they must be calculated differently from earlier investments. The visible costs are the tip of an iceberg whose base consists of organizational rebuild, lock-in risk, and opportunity cost. Those who compute only the tip make decisions on too friendly a foundation.

That does not mean: wait. It means: know what you are signing. The decision of whether, how, with whom, and when to invest in agentic infrastructure is one of the most important strategic decisions of the coming decade for many Mittelstand companies. It is too important to be made out of a business case that is structurally too optimistic.

It is too important to be fully delegated.

— Axel Roth