Home > Articles > Networking > Storage

This chapter is from the book

3.5 Gigabit Ethernet

Gigabit Ethernet owes its existence to the technical innovations of Fibre Channel transport and the historical momentum of Ethernet and IP networking. From Fibre Channel, Gigabit Ethernet has taken the breakthrough technology of gigabit physical specifications, fiber optics, CDR, 8b/10b data encoding, and ordered sets for link commands and delimiters. From Ethernet, it has inherited mainstream status and seamless integration to a vast installed base of operating systems and network infrastructures. Although Fibre Channel has had to struggle for credibility as an emergent technology, Gigabit Ethernet's credibility was established before it was even implemented. Today, 10 Gigabit Ethernet and higher speeds are assumed to be the logical evolution of the technology and of future enterprise networks. In the process, Ethernet is shedding some of its characteristic attributes such as collision detection and shared topology, but is retaining its name.

3.5.1 Gigabit Ethernet Layers

The reference model for Gigabit Ethernet is defined in the Institute of Electrical and Electronics Engineers (IEEE) 802.3z standard. Like 10/100 Ethernet, Gigabit Ethernet is a physical and data link technology, corresponding to the lower two OSI layers, as shown in Table 3–4.

Table 3–4 Gigabit Ethernet physical and data link layers

OSI Reference Layer

Gigabit Ethernet Layer

Data link layer

Media access control (MAC) client sublayer

MAC control (optional)

MAC

Physical Layer

Reconciliation

Gigabit media independent interface

Media-dependent PHY group

Medium-dependent interface

Medium


The Gigabit Ethernet physical layer contains both media-dependent and media-independent components. This allows the gigabit media-independent interface to be implemented in silicon and still interface with a variety of network cabling, including long- and shortwave optical fiber and shielded copper. The reconciliation sublayer passes signaling primitives between upper and lower layers, including transmit and receive status as well as carrier sense and collision detection. In practice, Gigabit Ethernet switching relies on dedicated, full duplex links and does not need a collision detection method. Carrier sense multiple access with collision detection (CSMA/CD) is incorporated into the standard to provide backward compatibility with standard and Fast Ethernet.

Unlike Fibre Channel, Gigabit Ethernet's 8b/10b encoding occurs at the physical layer via sublayers in the media-dependent physical (PHY) group. As shown in Figure 3–14, Fibre Channel layers FC-0 and FC-1 are brought into the lower layer physical interface, whereas traditional 802.3 Ethernet provides MAC and logical link control (LLC; or its offspring, MAC client) to support the upper layer protocols.

Figure 3–14 Gigabit Ethernet/Ethernet and Fibre Channel.

To facilitate its integration into conventional Ethernet networks and wide area transports, Gigabit Ethernet uses standard Ethernet framing as shown in Figure 3–15. The preamble and SOF delimiter are followed by the destination (DA) and source (SA) MAC addresses of the communicating devices. Creative use of bytes within the length/type field enable enhanced functionality such as VLAN tagging, as discussed later. The data field may contain as much as 1,500 bytes of user data, with pad bytes if required. The CRC is part of the frame check sequence. Optional frame padding is provided by the extension field, although this is only required for gigabit half-duplex transmissions.

Figure 3–15 Standard Ethernet frame format.

IP over Ethernet is inserted into the data field and provides the network layer routing information to move user data from one network segment to another. TCP/IP provides higher level session control for traffic pacing and the ability to recover from packet loss. Although IP can be carried in other frame formats, link-layer enchancements for Ethernet offer additional reliability and performance capability unmatched by other transports, including Fibre Channel. These include VLANs, QoS, link-layer flow control, and trunking. Collectively, these functions provide a set of powerful tools for constructing storage networks based on IP and Gigabit Ethernet.

3.5.2 802.1Q VLAN Tagging

LANs in a switched Ethernet infrastructure enable the sharing of network resources such as large Gigabit Ethernet switches while segregating traffic from designated groups of devices. Members of a VLAN can communicate among themselves but lack visibility to the rest of the network. Sensitive information (for example, financial or human resource) can thus be isolated from other users, although the traffic is running through a common infrastructure. VLAN tagging was standardized in 1998 through the IEEE 802.1Q committee. An analogous capability is provided in Fibre Channel through a technique called zoning. Standards for zoning are still under construction, but only relate to the exchange of zone information by Fibre Channel switches. How zones are actually implemented within a switch is still proprietary. Consequently, there is no direct equivalent to 802.1Q's more open and flexible format.

VLAN tagging is accomplished by manipulating the length/type field in the Ethernet frame. To indicate that the frame is tagged, a unique 2-byte descriptor of hex "81–00" is inserted into the field. This tag type field is followed by a 2-byte tag control information field, as shown in Table 3–5, which carries the VLAN identifier and user priority bits as described later. The 12-bit VLAN identifier allows for as many as 4,096 VLANs to be assigned on a single switched infrastructure—far more than the number of zones typically offered by Fibre Channel switch vendors.

Table 3–5 IEEE 802.1Q VLAN tag fields

802.1Q Tag Type Field

Tag Control Information Field

81-00

User Priority

Canonical format indicator bit (CFI)

VLAN Identifier

16 bits

3 bits

1 bit

12 bits


From a performance standpoint, VLAN tagging is a highly efficient means of segregating network participants into communicating groups without incurring the overhead of MAC address filtering. Intervening switches use the logical VLAN identifier, rather than the MAC address, to route traffic properly from switch to switch, and this in turn simplifies the switch decision process. As long as the appropriate switch port is associated with the proper VLAN identifier, no examination of the MAC address is required. Final filtering against the MAC address occurs at the end point.

All major Gigabit Ethernet switch vendors support the 802.1Q standard. This makes it a very useful feature not only for data paths that must cross switch boundaries, but for heterogeneous switched networks as well. For IP storage network applications, 802.1Q facilitates separation of storage traffic from user messaging traffic as well as segregation of different types of storage traffic (for example, on-line transaction processing) from tape backup. Compared with Fibre Channel zoning, 802.1Q VLANs offer more flexibility and lack the complexity of vendor-specific implementations.

3.5.3 802.1p/Q Frame Prioritization

The 802.1Q VLAN tag control information field allocates 3 bits for user priority. The definition for these User Priority bits is provided by IEEE 802.1p/Q, and enables individual frames to be marked for priority delivery. The QoS supported by 802.1p/Q allows for eight levels of priority assignment. This ensures that mission-critical traffic will receive preferential treatment in potentially congested conditions across multiswitch networks, and thus minimizes frame loss resulting from transient bottlenecks.

For storage network applications, the ability to prioritize transactions in an IP-based SAN is a tremendous asset. Storage networks normally support a wide variety of applications, not all of which require high priority. Updating an on-line customer order or a financial transaction between banks, for example, rates a much higher priority for business operations than a tape backup stream. The class of service provided by 802.1p/Q allows storage administrators to select the applications that should receive higher priority transport and assign them to one of the eight available priority levels. In a multiswitch network, class of service ensures that prioritized frames will have preference across interswitch links. Except for a few proprietary port-based implementations, Fibre Channel currently does not support frame prioritization and thus cannot distinguish between mission-critical and less essential storage applications.

3.5.4 802.3x Flow Control

Flow control at the data link level helps to minimize frame loss and avoids latency resulting from error recovery at the higher layer protocols. In Fibre Channel, flow control for class 3 service is provided by a buffer credit scheme. As buffers are available to receive more frames, the target device issues receiver readys (R_RDYs) to the initiator, one per available buffer. In Gigabit Ethernet, link-layer flow control is provided by the IEEE 802.3x standard. The 802.3x implementation uses a MAC control PAUSE frame to hold off the sending party if congestion is detected. If, for example, receive buffers on a switch port are approaching saturation, the switch can issue a PAUSE frame to the transmitting device so that the receive buffers have time to empty. Typically, the PAUSE frame is issued when a certain high-water mark is reached, but before the switch buffers are completely full.

Because the PAUSE frame is a type of MAC control frame, the frame structure is slightly different from the conventional data frame. Like VLAN tagging, the length/type field is used to indicate the special nature of the frame, in this case hex "88–08" to indicate a MAC control frame. As shown in Table 3–6, this indicator is followed by an opcode of hex "00–01" to define further the MAC control frame as a PAUSE frame. The amount of time that a transmitting device should cease issuing frames is specified by the opcode parameter field. pause_time cannot be specified in fixed units such as microseconds, because this would prove too inflexible for backward compatibility and future Ethernet transmission rates. Instead, pause_time is specified in pause_quanta, with one pause_quanta equal to 512 bit_times for the link speed being used. The timer value can be between 0 and 65,535 pause_quanta, or a maximum of approximately 33 msec at Gigabit Ethernet's 1.25-Gbps transmission rate. If the device that issued the PAUSE frame empties its buffers before the stated pause_time has elapsed, it can issue another PAUSE frame with pause_time set to zero. This signals the transmitting device that frame transmission can resume.

Table 3–6 IEEE 802.3X PAUSE frame format

Length/Type

MAC Control Opcode

Parameters

88-08

00-01

Pause_time

16 bits

16 bits

16 bits


Because PAUSE frames may be used between any devices and the switch ports to which they are attached, and because Gigabit Ethernet only allows one device per port, there is no need to personalize the PAUSE frame with the recipient's MAC address. Instead, a universal, well-known address of 04-80-C2-00-00-01 is used in the destination address field. When a switch port receives the PAUSE frame with this address, it processes the frame but does not forward it to the network.

The 802.3x flow control provided by Gigabit Ethernet switches creates new opportunities for high-performance storage traffic over IP. Fibre Channel class 3 service has already demonstrated the viability of a connectionless, unacknowledged class of service, providing there is a flow control mechanism to pace frame transmission. In Fibre Channel fabrics using class 3, as with 802.3x in Ethernet, the flow control conversation occurs between the switch port and its attached device. As the switch port buffers fill, it stops sending R_RDYs until additional buffers are freed. In Gigabit Ethernet, this function is performed with PAUSE frames, with the same practical result. In either case, buffer overruns and the consequent loss of frames are avoided, and this is accomplished with minimal impact on performance.

The reliability provided by the gigabit infrastructure through data link flow control enables streamlined protocols to be run at the upper layer. For IP storage, the equivalent to Fibre Channel class 3 is UDP. UDP is connectionless and unacknowledged, and thus is unsuited to very congested environments such as the Internet. For contained data center storage applications, however, 802.3x flow control and storage over UDP/IP can offer a reliable and extremely high-performance solution without incurring the protocol overhead of TCP/IP.

3.5.5 802.3ad Link Aggregation

Link aggregation, or trunking, provides higher bandwidth for switched networks by provisioning multiple connections between switches or between a switch and an end device such as a server. Link aggregation also facilitates scaling the network over time, because additional links to a trunked group can be added incrementally as bandwidth requirements increase. In Figure 3–16, two Gigabit Ethernet switches share three aggregated links for a total available bandwidth of 7.5 Gbps full duplex.

Figure 3–16 Link aggregation between two Gigabit Ethernet switches.

Originally, the 802.3ad standards initiative was promoted as a means to provide higher bandwidth for standard 10/100-Mbps Ethernet networks. Link aggregation was a means of satisfying higher bandwidth requirements while Gigabit Ethernet was still being developed. As with memory, CPUs, and storage, however, whatever performance or capacity is reached at any given point in time is never sufficient for the increasing demands of users and applications. Consequently, bundled Ethernet links have been replaced with bundled Gigabit Ethernet links, which at some point will be superseded by bundled 10 Gigabit and higher Ethernet links. Replicators, for example, will no doubt require bundled 100 Gigabit Ethernet links.

Link aggregation must resolve several issues to avoid creating more problems than it fixes. In normal bridge environments, the spanning tree algorithm would, on encountering multiple links between two devices, simply disable the redundant links and only allow a single data path. This would prevent duplication of frames and potential out-of-order delivery. Link aggregation must therefore make multiple links between two devices appear as a single path, while simultaneously providing a mechanism to avoid frame duplication and ensure in-order frame delivery. This could be accomplished by manipulating MAC addresses (for example, assigning the same MAC address to every trunked link) or by inserting a link aggregation intelligence between the MAC client and MAC layers. The status of link availability, current load, and conversations through the trunk require monitoring to ensure that frames are not lost or inadvertantly reordered.

In-order delivery of frames is guaranteed if a conversation between two end devices is maintained across a single link in the trunk. Although this is not as efficient link utilization as simply shipping each frame over any available connection, it avoids the extra logic required to monitor frame ordering and to reassemble them before delivery to the recipient. At the same time, additional transactions by other devices benefit from the availability of the aggregated interswitch links, and switch-to-switch bottlenecks are avoided.

Link aggregation as specified in 802.3ad is almost mandatory for IP-based storage networks, particularly when multiple Gigabit Ethernet switches are used to build the SAN backbone. Along with 802.1p/Q prioritization, link aggregation can ensure that mission-critical storage traffic has an available path through the network and that multiple instances of mission-critical transactions can occur simultaneously. This requirement will be satisfied temporarily by the arrival of 10Gb uplinks between switches, but these will inevitably be "trunked" to provide even higher bandwidth over time.

3.5.6 Gigabit Ethernet Physical Layer Considerations

Gigabit Ethernet has borrowed so heavily from the Fibre Channel physical layer that there are relatively few differences between them. Gigabit Ethernet has a slightly higher transmission rate of 1.25 Gbps, compared with Fibre Channel's 1.0625 Gbps. For storage applications, Gigabit Ethernet's faster clock can drive approximately 15 MBps more bandwidth over interswitch links.

Transceivers for Gigabit Ethernet applications may also vary, although some GBICs can interoperate with both Fibre Channel and Gigabit Ethernet transmission speeds. Gigabit Ethernet has introduced support for new cable types for gigabit transport, including category 5 unshielded twisted pair. As shown in Table 3–7, cable distances are comparable with Fibre Channel, with the exception of long-wave, single-mode cabling (10 km for Fibre Channel).

Table 3–7: Gigabit Ethernet cable specifications

Cable Type

Diameter

Laser Type

Maximum Distance (m)

1000BASE-T

CAT-5 UTP

N/A

100

1000BASE-CX

STP

N/A

25

1000BASE-LX

10 mm

Long wave

5,000

1000BASE-SX

50 mm

Short wave

500

1000BASE-LX

50 mm

Long wave

550


InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020