Enterprises bringing their first serious AI cluster on premises usually spend months evaluating GPUs, racks, and vendors, and comparatively little time on the network tying it all together, even though that network ends up shaping nearly everything else in the build. The fabric determines how much resilience is achievable, resilience shapes how much room there is to grow, and growth runs straight into whatever the facility and its cooling can physically support. Underneath all of it sits a harder question about why the cluster is being built in the first place, since that answer changes what the whole architecture needs to deliver. Here is where the biggest shifts are showing up, and how enterprises can get ahead of each one as the next generation of AI clusters comes online.

The Fabric Decision Comes First

The first of these decisions, and often the most contentious, is the fabric itself. Ask the major networking vendors whether a new AI cluster should run on InfiniBand or Ethernet, and most will answer with Ethernet, which says more about their product line than about the cluster.

NVIDIA’s reference architecture splits the job between the two, pairing InfiniBand for GPU-to-GPU traffic with Ethernet for everything else, including storage, and most manufacturers build to that standard. InfiniBand remains low-touch to deploy and cost-competitive, requiring far less tuning than Ethernet needs to reach comparable performance. Ethernet’s advantage shows up in daily operations, where its skill set is widely available across most IT organizations in a way InfiniBand’s specialized protocol still isn’t.

Ultra Ethernet, announced about a year ago, is closing that gap further. The standard applies more advanced congestion control to bring InfiniBand-level performance into a multi-vendor Ethernet environment, and vendors are beginning to commit to release dates for supporting code. Within two years, Ethernet is likely to carry the majority of new AI cluster deployments, driven as much by the maturity of its management and observability tooling as the underlying technology itself.

Building Resilience for Production Workloads

As more enterprises move from internal, proof-of-concept clusters toward customer-facing production deployments, how well the fabric handles real workloads under failure becomes the next test, and resilience turns into a design requirement rather than something layered in after the fact.

NVIDIA is pushing rail-optimized leaf-and-spine designs to build that resilience directly into the fabric. This is a cabling and logical layout that differs from the topologies most network engineers have used in traditional environments. The metric worth tracking is end-to-end job completion time rather than uptime on any single switch or link, and how to get there depends heavily on the workload. An eight-GPU cluster connected to a pair of switches carries a different risk profile than an NVL72 system, where seventy-two GPUs sit in a single rack with resilience largely built into the platform itself. Enterprises building their own infrastructure outside that kind of pre-integrated system also need to weave power, cooling, compute, and job scheduling into the same resilience plan.

Architecture Should Account for Growth from the Start

The biggest difference between a cluster that scales smoothly and one that needs a redesign two years in usually comes down to how the initial architecture accounted for growth (which is really the same resilience question playing out over a longer timeline).

Enterprises often approach AI networking the way they would with a standard compute cluster, counting GPUs, counting ports, and sizing switches to match. That approach overlooks what the network is actually doing. A cluster’s GPU connectivity, often assumed not to need 800-gigabit bandwidth because the workload itself isn’t moving that much data in and out, is largely handling memory copies between GPUs – a function closer to a backplane than typical network traffic. Building that bandwidth via a Scalable Unit methodology from the start costs more upfront, but pays for itself the moment the cluster needs to expand without the complexity of a full re-cabling effort. A complete architecture also accounts for storage, out-of-band management, and front-end networks, all of which rarely factor into a simple GPU port count yet still meaningfully shape what gets built.

AHEAD spends significant time with clients understanding where their AI initiative is actually headed, then builds architecture around a six-, twelve-, and twenty-four-month horizon so that today’s investment keeps making sense as the cluster grows.

Validating the Facility Before the Gear Ships

Forward planning at this level routinely extends past the rack and into the facility itself. AHEAD’s Foundry team validates details that customers rarely think to check themselves, from whether a data center can support a given rack’s power and cooling load to whether its fiber optics are the right type for the job. For example, many existing deployments still run older UPC fiber, where newer builds call for APC fiber to avoid light diffraction issues – a detail worth catching before it becomes expensive to fix.

That same diligence carries into direct conversations with data center managers about power delivery and cooling capacity, often across a cage holding multiple 30kW+ racks. It is core to how AHEAD approaches every AI infrastructure deployment, ensuring what ships from Foundry performs the way it should once it is on site.

Where Cooling Becomes a Network Decision

Cooling has become as much a network problem as a facilities one. Air can’t dissipate heat at the density modern AI racks demand, so switch vendors are building platforms meant to run fully liquid cooled. Today, most equipment lands around 85% liquid cooled and 15% air; a practical middle ground while the technology matures.

Decisions that used to live entirely within networking now factor into facilities planning as well. Where switches sit relative to racks, which optic form factors a design can support, and how airflow gets sealed and directed through cold and hot aisles (when air cooling is still part of the mix) all belong in the same conversation now. Liquid-cooled switches bring their own engineering challenges, since a pluggable transceiver is harder to cool than a component that can simply take a heat sink. Matching a customer’s specific data center or colocation capabilities to the right combination of these technologies has become as much a part of network design as choosing a switch vendor.

The Underlying Question of ‘Why’

Fabric, resilience, scale, and facility readiness all eventually circle back to why the enterprise is building the cluster in the first place. Enterprises bringing GPUs on premises often start that conversation around reducing cloud or neo-cloud spend. The bigger opportunity usually extends well past that math, though, once the cluster actually goes live.

In the cloud, GPU access is typically limited to a narrow set of developers, and the economics push toward using those GPUs as little as possible since someone is paying by the hour. Bring that same capacity on premises and the incentive flips. Idle GPUs represent wasted capital, so enterprises want to keep them busy, which means use cases multiply well beyond the initial developer-focused scope. Internal agentic tools, employee-facing applications, and customer-facing products built on the same cluster tend to show up faster than most leaders expect, and each persona can pull the hardware and software stack in a different direction.

Getting a clear read on organizational appetite, anticipated use cases, and the range of teams that will eventually touch the cluster before locking in an architecture gives the fabric, resilience, and growth decisions that came before it a real chance of continuing to make sense two years from now.

Why AHEAD Is the Right Partner for This Shift

AHEAD combines deep networking expertise with hands-on validation in AHEAD Foundry to help enterprises design AI-ready fabrics that hold up as their initiatives grow well beyond the first use case. From the InfiniBand-versus-Ethernet decision through power, cooling, and facility readiness, AHEAD helps clients build network architecture for where their AI roadmap is actually headed.

If your AI plans are starting to outgrow what your current network can support, connect with AHEAD to assess where you stand and build a plan that scales.