Why AI is re-designing data center architecture

Data center design has been shaped by a familiar set of priorities for years: keep systems available, resilient and predictable in any condition. Just like the electrical grid that powers these sites, they have been engineered to provide a highly consistent service regardless of what happens, even when individual components fail.

This has meant operators build layers of redundancy into power, cooling and network infrastructure.

However, artificial intelligence has changed the story. While organizations are racing to roll out AI tools at scale, the data center industry is discovering that not all workloads have the same infrastructure requirements.

For instance, training a large language model, running real-time inference, supporting enterprise applications and processing business-critical transactions each place very different demands on the underlying infrastructure.

Today, one data center doesn’t need to serve every purpose equally and we’re increasingly seeing that facilities can be both flexible and tailored to specific workload requirements.

The end of the traditional model

Historically, 99.999% uptime was non-negotiable. Data centers have traditionally powered systems like banks, emergency networks and customer-facing digital services, requiring continuous availability.

In these types of environments, where outages could have an extreme impact (from high financial losses to putting real lives at risk) this approach makes sense. Since operators couldn’t always predict which applications would be truly mission-critical, many facilities were built to the highest resilience standards by default.

But AI has changed this. “One-size-fits-all” redundancy isn’t necessary anymore. Different models, training and inference processes each require totally different service levels. For example, facilities for AI training workloads are being designed without backup generators, complex redundancy systems or high-tier architecture.

The good news is that there is a growing understanding of the distinction between environments needed for AI training and AI inference. Training facilities are increasingly being located wherever power is available.

The primary constraints are energy supply, cooling capacity and speed of deployment. In many cases, maximizing compute density and accelerating delivery timelines are more important than achieving the highest possible redundancy levels.

Inference infrastructure presents a different set of priorities. These workloads are often deployed closer to users and support services that people interact with daily. In these scenarios, latency, availability and customer experience become notably more important, creating a stronger case for resilient infrastructure and geographically distributed architectures.

Precision resilience to support an industry under pressure

It’s clear, therefore, that reliability still matters. However, infrastructure requirements vary significantly depending on the service being supported. In today’s age of AI, ‘precision resilience’ should be the focus, e.g., redundancy matching how workloads actually behave, rather than relying on legacy design assumptions.

The key challenge here for operators is determining where resilience delivers genuine business value and where it simply adds cost and complexity.

In a time when developers are facing a huge amount of pressure amid labor shortages, with demand outpacing supply, defaulting to ultra-resilient, high-tier designs for every AI deployment only intensifies challenges.

The industry is also expected to deliver capacity faster than ever before, while battling an ongoing power gap, meaning large-scale developments are increasingly difficult to execute. In this landscape, overengineering infrastructure can have unintended consequences.

Every additional layer of redundancy consumes capital and increases complexity. This is triggering an increased focus on efficiency, not just in terms of energy consumption, but in how capital is allocated throughout a project. Operators are looking to design infrastructure that maximizes the value generated by every watt of available power.

The role of upgradability

As operators move away from this one-size-fits-all redundancy to optimize their bottom line, it’s crucial that their facilities can adapt as workload requirements change.

While inference is expected to account for a growing share of AI demand, the landscape continues to evolve and it’s difficult to predict which workloads, densities and cooling requirements will dominate in the future. Infrastructure that can accommodate changes in compute technologies will be better positioned to support the next generation of AI applications.

Flexibility and fungibility are therefore the new non-negotiables in data center design. How is this made possible? Increasingly, developers are using ‘building blocks’ constructed off-site in factory environments, and then later assembling them on site to create an adaptable facility that can forever evolve, grow and shift.

This approach reduces the need to make every resilience decision upfront and builds with tomorrow’s changes in mind. In the coming years, we will see a shift towards multiple types of facilities, each developed for a different purpose.

These will range from energy-optimized training campuses built close to power sources, to distributed inference sites where uptime and latency directly affect user experience, alongside hybrid environments supporting both AI and traditional workloads. Yet they should all be built with flexibility front of mind to ensure they can evolve as requirements change.

We've featured the best web hosting service.

This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit