Role of Networking in the data center buildout

We continue our AI data center component series with networking. Networking is the less flashy but equally critical infrastructure that holds an AI cluster together. It connects thousands of GPUs and allows them to work as one system. As AI models grow larger and more complex, the speed and efficiency of these connections increasingly determine how much computing power a data center can actually deliver.

What is networking and why does AI need it?

Networking is the infrastructure that allows computers and processors to exchange data with one another. In traditional data centers, networks connect servers to storage systems, other servers and the outside world. In AI data centers, networking takes on an additional, increasingly critical role. Training a large AI model can involve thousands of GPUs working on the same model simultaneously. These GPUs must constantly exchange intermediate calculations and coordinate their work. If the network cannot transfer information quickly enough, GPUs spend time waiting rather than calculating, leaving expensive computing capacity underutilized.

A simple analogy

Picture a factory with thousands of workers, each responsible for one part of a product. The speed at which the factory produces finished goods depends not just on how quickly individual workers operate, but also on how efficiently materials move between them. GPUs are the workers, while networking is the infrastructure moving materials between workstations. Adding more workers without expanding the transportation infrastructure eventually creates congestion. Similarly, for AI workloads, adding more GPUs does not necessarily translate into proportionately more computing power unless the network can keep up.

What makes up a network?

There are four key physical components. Network switches receive data and forward it towards its destination. Network interface cards (NICs) connect individual servers to the network. Copper cables and optical fiber physically carry information as electrical signals or pulses of light. Finally, optical transceivers convert electrical signals into light and back, allowing information to travel across fiber connections quickly without loss of data. Together, these components allow thousands of servers to exchange information at enormous speeds.

AI Networking is overtaking traditional data center networking:

Traditionally, most data center networking expenditure went towards connecting servers to storage, applications and users, known as front-end networking. AI has created a rapidly growing market for back-end networking, which connects GPUs so they can work together. According to Dell'Oro Group, in Q2 2026, sales of switches used in AI back-end networks exceeded those used in traditional front-end networks for the first time. Within just three years, AI has effectively created a new networking market that is already larger than the established market for front-end switches.

How do thousands of GPUs communicate?

When training an AI model, different GPUs perform different calculations and repeatedly need to share their results with one another. Within a tightly connected system, GPUs communicate through dedicated high-speed connections, known as scale-up networking. But when information needs to reach GPUs in other servers or racks, it travels through network interface cards, switches and copper or optical-fibre cables. This is called scale-out networking, where Ethernet and InfiniBand are the two major technologies. The challenge is not simply connecting thousands of GPUs but ensuring that information moves quickly enough between them so that expensive processors aren't left waiting for one another. Scale-across takes this a step further, connecting AI clusters in different data centers or even geographical regions through high-capacity optical networks, allowing geographically separated GPUs to work on the same AI model.

One rack, 72 GPUs and thousands of connections

In March 2024, Nvidia introduced GB200 NVL72 system whose single rack houses 72 GPUs connected through more than 5,000 copper cables and Nvidia's NVLink switching technology, providing 130 terabytes per second of aggregate communication bandwidth. In January 2026, Nvidia introduced the newer Vera Rubin NVL72 system which doubles that bandwidth to 260 terabytes per second while connecting the same number of GPUs. These high-speed connections allow dozens of GPUs to operate as one tightly integrated computing system rather than independent processors. As AI systems become more powerful, the networking infrastructure connecting them is advancing just as rapidly.

Multiple layers of networking gear bind together an AI cluster:

• Network Switches: Directing traffic between GPUs. Switches are physical boxes installed in data center racks, with dozens of ports for connecting cables. When a GPU sends information to another server, the network switch examines the destination information and forwards the data through the appropriate port. These networks primarily use two technologies: InfiniBand, dominated by Nvidia, and Ethernet, supported by a broader group of suppliers. While InfiniBand initially dominated AI networking, Ethernet captured more than two-thirds of AI back-end switch sales in 2025, according to Dell'Oro. This shift is opening the market to more equipment suppliers, including Broadcom, Arista, Cisco and Celestica, while Nvidia also competes through its own Ethernet products.

• Network Interface Card (NIC): connecting servers to the network. Before information reaches a switch, it leaves the server through a NIC, an electronic component installed inside the server with ports connecting to external cables. The technology is advancing rapidly. Nvidia's 2026 Rubin architecture supports up to 1.6 terabits per second (Tbps) of aggregate networking bandwidth per GPU (that’s roughly 200 GB per second) and AMD's Helios architecture goes further, supporting up to three 800G Vulcano NICs per GPU, providing a combined 2.4 Tbps. Increasingly powerful GPUs require more and faster networking hardware, expanding the opportunity for NIC suppliers.

• Cables and connectors: The physical connection. Thousands of copper and optical-fiber connections link servers to switches and switches to one another. Copper is attractive for short distances because of its low cost and power requirements, while optical fiber becomes more useful as distances and speeds increase. Amphenol and TE Connectivity supply the cables and connectors used across this infrastructure. As networks move towards 800G and 1.6T connections, equivalent to theoretical data-transfer capacities of 100 GB and 200 GB per second, respectively, the components carrying these signals become increasingly sophisticated.

• Optical transceivers: As AI clusters expand across racks and increasingly across different data centers, optical fiber becomes essential for moving enormous amounts of data over longer distances. Unlike copper, where electrical signals weaken significantly over long distances, optical fiber transmits light with much lower signal loss, allowing data to travel farther at high speeds. Optical transceivers convert electrical signals into light and back, with companies like Coherent and Lumentum supplying lasers, transceivers and related components. For scale-across networking, Ciena and Nokia provide optical transport systems that connect geographically separated data centers through high-capacity fiber networks.

Current key debates

We highlight below the key debates around AI networking

• Can networking grow faster than GPUs? Every new generation of GPUs requires faster connections, potentially increasing networking expenditure per GPU. But improvements in network efficiency and AI software could reduce the amount of networking hardware needed for a given workload.

• Copper vs optical. As connection speeds increase, optical networking becomes increasingly attractive. How quickly will optical connections replace copper, and will technologies like co-packaged optics change which suppliers capture the most value?

• Will scale-across become the next growth driver? Power and space constraints are encouraging data center operators to distribute AI infrastructure across multiple locations. But latency and the cost of long-distance connections could limit how effectively geographically separated GPUs can work together, particularly during training.4

Missed our previous deep dives? Catch up here: