This chapter is from the book
This chapter is from the book
This chapter is from the book
Transmission Modes
Beyond the DSP and mux/demux, we move more toward optics. Two decades ago, most network gear, including switches and routers, relied on copper cables for data transmission. But these cables are limited in terms of how fast data can move. As data speeds picked up, these cables couldn’t keep up, as they only reached about 3 meters for 100 Gbps Ethernet links. They were good for connections in the same rack, though, such as connecting servers to the top of rack (TOR) switch.
Today, we have high-performance switches, routers, and SmartNICs that can handle speeds of 200 Gbps, 400 Gbps, and even 800 Gbps. To handle these speeds, fiber-optic cables have taken over from copper ones. Instead of using electrical signals, optical fiber uses light to send data.
The advantage in fiber-optic cables is that they can go much farther—up to 80 km or even 120 km—without losing signal strength. They’re also much more reliable than copper because they use total internal reflection to carry light and therefore aren’t affected by electromagnetic interference. In addition, fiber-optic cables handle changes in temperature and pressure well.
The following sections detail the different cable options available for connectivity in data center fabrics.
Multi-Mode Fiber (MMF)
MMF is optical fiber that is designed for the transmission of multiple rays of light at a time (see Figure 4-7). Normally, the core diameter of MMF is 50 μm (micrometers) and 62.5 μm. It propagates 850 nm (nanometers) and 1300 nm wavelengths from low-cost light sources like LEDs or VCSELs and has more attenuation. MMF is used for short-to medium-range connections, as well as for connecting devices within the same rack or in nearby racks.
)
There are five different grades of MMF:
OM1: OM1 supports 1 Gbps speeds up to a distance of 300 m and 10 Gbps speeds up to 33 m.
OM2: OM2 supports 1 Gbps speeds up to a distance of 550 m and 10 Gbps speeds up to 82 m.
OM3: OM3 supports 40 Gbps speeds up to a distance of 240 m and 100 Gbps to 400 Gbps speeds up to a distance of 100 m.
OM4: OM4 supports speeds of 100 Gbps to 400 Gbps up to a distance of 150 m.
OM5: OM5 supports speeds of 100 Gbps to 400 Gbps up to a distance of 150 m. It can accomplish 400 Gbps with just four transmit and receive cables and wavelength division multiplexing (WDM) technology. OM5 uses the wavelength 953 nm along with the wavelength of 850 (OM4).
OM1 has a glass core diameter of 62.5 μm. The other grades of MMF have a glass core diameter of 50 μm.
Single-Mode Fiber (SMF)
SMF is an optical fiber that is designed for the transmission of a single ray of light at a time, eliminating distortion from overlapping light pulses (see Figure 4-8). Normally, SMF has a core diameter of 8 μm to 10 μm, which can propagate higher wavelengths of 1310 nm and 1550 nm from the laser. It is an expensive type of cable that is used for long-distance data transmission between different buildings or data centers.
)
Dense Wavelength Division Multiplexing (DWDM)
DWDM enables dense wavelength multiplexing by multiplexing multiple electrical signals to a single optical lane, as illustrated in Figure 4-9. DWDM technology helps in transmitting data over long distances, as described earlier in this chapter. This is specially required in case of data center interconnects.
)
AI Server Connectivity Options
On the switch side, the connection could be either single-mode or multi-mode fiber and optics, depending on the distance. Since the server ports an OSFP transceiver with 800 Gbps, the switch side must be a 800 Gbps port or two 400 Gbps ports with the use of breakout cables. Figure 4-10 illustrates both of these connectivity options. For rail-optimized design (ROD), another option is able to connect 8 GPUs to 8 different switches.
)
Figure 4-11 illustrates the connectivity option with the A100 server. There are 8 ports on the server that can connect to 8 different switches in ROD.
)
