The Reconfigurable Fabric: Optical Circuit Switching vs Electrical Packet in AI Clusters

Inside a modern AI cluster, thousands of accelerators talk to each other constantly. The network that connects them has quietly become one of the largest power consumers in the building, and the switches that route the traffic are a big reason why.

For two decades the answer was electrical packet switching: fast, flexible, and power-hungry. A newer approach, optical circuit switching, offers a different trade: it reconfigures slowly but moves bits at a fraction of the energy. The choice between them is the central design decision of the next generation of AI infrastructure.

An electrical packet switch takes a signal in, reads its header, decides where it goes, and drives it out. At high line rates, every one of those steps costs power, and the biggest cost is the conversion: the signal must be received, retimed, and re-transmitted at every hop. Each conversion burns energy, and in a large cluster the number of hops is large.

As line rates climb from 100 to 400 to 800 gigabits per second, the power per port climbs with them. The result is that the network's share of total cluster power has grown from a rounding error to a double-digit percentage. At some point, the switch becomes a reason you cannot build the cluster you wanted.

An optical circuit switch does not read packets. It sets up a physical path — a circuit — between two endpoints using mirrors or similar mechanisms, and then light flows through that path without conversion. Because the light is never turned back into electrons and out again, the energy per bit drops dramatically.

The catch is reconfiguration time. Setting up a circuit takes microseconds to milliseconds, far slower than the nanoseconds an electrical switch needs to route a packet. That is fine for workloads with predictable, long-lived flows — a training job that shuffles large tensors between a fixed set of nodes — and awkward for bursty, many-to-many traffic where the destination changes every packet.

Optical circuit switching moves a bit for a fraction of the energy of electrical packet switching, but it pays for that saving in flexibility, and the bill comes due whenever the traffic pattern changes faster than the mirrors can move.

In practice, the winning design is not one or the other but both. A cluster uses an electrical packet network for the general, bursty traffic and an optical circuit network for the heavy, predictable flows — the all-reduce and all-to-all exchanges that dominate training. The optical layer handles the bulk data; the electrical layer handles the coordination.

This hybrid matches the physics of the workload. Training traffic is dominated by a few very large, structured transfers, which is exactly what a circuit handles well. Inference traffic is more diverse, which is what packets handle well. Splitting the job lets each technology do what it is good at.

An optical circuit saves energy by never converting light back into electrons, but its microsecond reconfiguration is far too slow for the packet-level bursts that dominate general traffic, which is why the electrical layer never disappears.

A market estimate grounds the shift. If AI clusters continue to grow and the network's share of power keeps rising, the optical-switching segment could grow from a niche into a multi-billion-dollar market over five years, driven by the largest hyperscalers first and then by enterprises building private training clusters. The total is not enormous compared with the accelerators, but the power savings compound across every rack.

The social effect is indirect but real. Cheaper bits between accelerators mean cheaper training, which means the models behind everyday tools — translation, search, coding help — get better at the same price. A household does not see a switch, but it sees the assistant get faster and the subscription stay flat.

There is an infrastructure consequence too. A cluster that spends less power on networking has more power for compute, and in a world where power is the binding constraint on data-centre growth, that headroom is worth more than the switch itself. Optical fabrics are, in effect, a way to buy back megawatts.

The lesson is not "replace your switches with optics." It is that every network is a trade between energy and flexibility, and the right choice depends on the traffic pattern. A circuit is cheap per bit but slow to change; a packet is flexible but expensive per bit. The mature design uses each where its trade is favourable.

For architects, the practical rule is to characterise the traffic before choosing the fabric. If the workload is dominated by large, structured, predictable flows, an optical layer will pay for itself in power. If it is bursty and diverse, packets remain the safer default. Measuring the flow is worth more than any vendor benchmark.

The network of the next AI cluster will not be all-optical or all-electrical; it will be the cheapest mix of both for the traffic it actually carries.

Light is cheap to move and expensive to switch. The art is knowing when to stop switching and just let it flow.

SOL: HCqPmPJM3NVtS6YrmVaTZYnuP8XZ2PFgQZKT94VK1LkS

If this saved you time, a tip is appreciated.