Cisco Nexus FEX and vPC Configuration Guide Part 5
This is Part 5 of the Cisco Nexus Data Center series. In this guide, you will configure:
- Cisco Nexus Fabric Extenders (FEX)
- Nexus 2000 integration
- Single-homed FEX
- Dual-homed FEX
- vPC integration with FEX
- LACP peer-links
- Fabric interfaces
This is one of the most important concepts inside Cisco Nexus architecture because FEX technology fundamentally changed data center access-layer design.
Table of Contents
- Introduction to FEX
- FEX Architecture
- FEX Mathematics and Bandwidth Logic
- Task 1 - Configure Single-Homed FEX
- Task 2 - Verify FEX
- Task 3 - Configure Second FEX
- Task 4 - Remove Existing FEX
- Task 5 - Configure vPC Peer-Link
- Task 6 - Configure vPC Domain
- Task 7 - Configure Dual-Homed FEX 101
- Task 8 - Configure Dual-Homed FEX 102
- Verification Commands
- Modern Data Center Evolution
- Troubleshooting
- Related Articles
Introduction to Fabric Extenders (FEX)
A Cisco Nexus Fabric Extender (FEX) is not a traditional standalone switch.
Instead, a FEX behaves like a remote line card for a parent Nexus switch.
Important Concept
The Nexus 2000 FEX has:
- No independent control plane
- No separate management plane
- No standalone configuration
- No local spanning-tree processing
All intelligence comes from the parent Nexus switch.
FEX Analogy
Think of a FEX like a USB expansion hub connected to a computer.
The hub itself is not intelligent. The computer controls everything.
Understanding FEX Architecture
Traditional Access Layer
Traditional Design:
$$ Access\\ Switches = Independent\\ Devices $$
FEX Design
FEX Design:
$$ FEX = Remote\\ Line\\ Card $$
Benefits of FEX
- Simplified management
- Reduced operational overhead
- Lower power consumption
- Reduced STP complexity
- Centralized control
- Scalable architecture
Parent Switch Responsibilities
- MAC learning
- Forwarding decisions
- QoS policies
- Security policies
- Configuration management
- Spanning-tree participation
Bandwidth and FEX Mathematics
Fabric Bandwidth Formula
Total Fabric Bandwidth:
$$ BW_{Total} = BW_1 + BW_2 + BW_3 + ... $$
Example:
$$ 10Gbps + 10Gbps = 20Gbps $$
Oversubscription Formula
Oversubscription Ratio:
$$ Oversubscription = \\frac{HostFacingBandwidth}{FabricBandwidth} $$
Example:
$$ 48 \\times 1Gbps = 48Gbps $$
Fabric Links:
$$ 2 \\times 10Gbps = 20Gbps $$
Oversubscription:
$$ \\frac{48}{20} = 2.4:1 $$
Task 1 - Configure Single-Homed FEX on NX-5K1
NX-5K1 Configuration
feature fex
interface ethernet1/11-12
channel-group 101 mode on
no shutdown
interface port-channel101
switchport mode fex-fabric
fex associate 101
What Happens Here?
- FEX feature is enabled
- Interfaces are bundled statically
- Port-channel 101 becomes a fabric interface
- FEX ID 101 is assigned
Why "mode on" Instead of LACP?
FEX fabric interfaces typically use static port-channels because the parent Nexus directly controls the FEX.
Task 2 - Verify FEX Online Status
show fex
Sample Output
NX-5K1# show fex
FEX FEX
Number Description State
--------------------------------
101 FEX0101 Online
Why FEX Takes Time to Come Online
The parent Nexus switch performs:
- Discovery
- Firmware checks
- Image synchronization
- Control plane initialization
- Port provisioning
Task 3 - Configure Second Single-Homed FEX
NX-5K2 Configuration
feature fex
interface ethernet1/11-12
channel-group 102 mode on
no shutdown
interface port-channel102
switchport mode fex-fabric
fex associate 102
Task 4 - Remove Existing FEX Configuration
Before implementing dual-homed FEX, existing configurations must be removed.
NX-5K1
no interface port-channel101
default interface ethernet1/11-12
NX-5K2
no interface port-channel101
default interface ethernet1/11-12
Why Default Interfaces?
Defaulting interfaces removes all inherited configurations, ensuring a clean deployment for dual-homed FEX.
Task 5 - Configure vPC Peer-Link
NX-5K1
feature lacp
interface ethernet1/1-2
channel-group 12 mode active
no shutdown
interface port-channel12
switchport
switchport mode trunk
no shutdown
NX-5K2
feature lacp
interface ethernet1/1-2
channel-group 12 mode active
no shutdown
interface port-channel12
switchport
switchport mode trunk
no shutdown
Bandwidth Mathematics
Peer-Link Bandwidth:
$$ 10Gbps + 10Gbps = 20Gbps $$
Task 6 - Configure vPC Domain
NX-5K1
interface mgmt0
ip address 192.168.1.1/24
no shutdown
feature vpc
vpc domain 12
peer-keepalive destination 192.168.1.2 source 192.168.1.1
interface port-channel12
vpc peer-link
NX-5K2
interface mgmt0
ip address 192.168.1.2/24
no shutdown
feature vpc
vpc domain 12
peer-keepalive destination 192.168.1.1 source 192.168.1.2
interface port-channel12
vpc peer-link
Task 7 - Configure Dual-Homed FEX 101
NX-5K1 Configuration
feature fex
interface ethernet1/11-12
channel-group 101 mode on
no shutdown
interface port-channel101
switchport mode fex-fabric
fex associate 101
vpc 101
NX-5K2 Configuration
feature fex
interface ethernet1/13-14
channel-group 101 mode on
no shutdown
interface port-channel101
switchport mode fex-fabric
fex associate 101
vpc 101
What is Dual-Homed FEX?
A dual-homed FEX connects simultaneously to two parent switches using vPC.
Redundancy Formula:
$$ Availability = 1 - (Failure_1 \\times Failure_2) $$
Benefits
- Redundancy
- Load balancing
- Active-active forwarding
- No STP blocking
- Higher bandwidth
Task 8 - Configure Dual-Homed FEX 102
NX-5K1 Configuration
feature fex
interface ethernet1/13-14
channel-group 102 mode on
no shutdown
interface port-channel102
switchport mode fex-fabric
fex associate 102
vpc 102
NX-5K2 Configuration
feature fex
interface ethernet1/11-12
channel-group 102 mode on
no shutdown
interface port-channel102
switchport mode fex-fabric
fex associate 102
vpc 102
Enhanced vPC Configuration Using FEX Host Interfaces
In this section, we will configure an advanced Cisco Nexus feature known as Enhanced vPC.
Enhanced vPC allows a server to connect simultaneously through multiple Fabric Extenders (FEX) while still participating in the same logical Port-Channel.
What Makes Enhanced vPC Special?
Traditional vPC normally connects servers directly to parent Nexus switches.
Enhanced vPC extends this capability through FEX switches, allowing servers connected to different FEXes to appear as a single logical Port-Channel.
Enhanced vPC Architecture
The server in this topology supports NIC Teaming.
NIC Teaming combines multiple physical NICs into one logical interface to improve:
- Redundancy
- Bandwidth aggregation
- High availability
- Load balancing
NIC Teaming Bandwidth Formula
If:
$$ NIC_1 = 10Gbps $$
and
$$ NIC_2 = 10Gbps $$
Then:
$$ Total\\ Logical\\ Bandwidth = 20Gbps $$
NX-5K1 Enhanced vPC Configuration
interface ethernet101/1/1
channel-group 111
no shutdown
interface ethernet102/1/1
channel-group 111
no shutdown
interface port-channel111
switchport
switchport mode trunk
no shutdown
NX-5K2 Enhanced vPC Configuration
interface ethernet101/1/1
channel-group 111
no shutdown
interface ethernet102/1/1
channel-group 111
no shutdown
interface port-channel111
switchport
switchport mode trunk
no shutdown
Understanding FEX Host Interface Numbering
The interface format:
$$ Ethernet\\ FEX-ID / SLOT / PORT $$
Example:
$$ Ethernet101/1/1 $$
- 101 = FEX ID
- 1 = Module/slot
- 1 = Physical port
Why Trunk Mode is Used
The Port-Channel is configured as a trunk because the server may carry multiple VLANs simultaneously.
This is common in:
- VMware ESXi environments
- Hyper-V deployments
- KVM virtualization
- Container infrastructures
- Private cloud environments
Enhanced vPC Advantages
- Server redundancy
- Higher bandwidth
- Active-active forwarding
- No spanning-tree blocking
- Simplified virtualization connectivity
- Better east-west traffic performance
Verification Commands
show port-channel summary
show vpc
show interface trunk
show lacp neighbor
show mac address-table
Sample Verification Output
NX-5K1# show port-channel summary
Group Port-Channel Type Protocol Member Ports
----- ------------ ---- -------- ----------------------------
111 Po111(SU) Eth LACP Eth101/1/1(P)
Eth102/1/1(P)
Enhanced vPC Availability Formula
If:
$$ P_1 = Parent\\ Switch\\ 1 $$
$$ P_2 = Parent\\ Switch\\ 2 $$
Then:
$$ Availability = P_1 + P_2 $$
Meaning traffic survives even if one parent switch fails.
Common Enhanced vPC Issues
- NIC Teaming mismatch
- LACP disabled on server
- Native VLAN mismatch
- vPC consistency failures
- MTU mismatch
- Incorrect trunk VLANs
Key Takeaway
Enhanced vPC is one of the most powerful features in Cisco Nexus data center architecture because it combines:
- FEX scalability
- vPC redundancy
- NIC teaming
- High-performance virtualization networking
Verification Commands
show fex
show fex detail
show vpc
show port-channel summary
show interface brief
show lacp neighbor
show interface trunk
Sample FEX Verification
NX-5K1# show fex
FEX FEX
No State Model
--------------------------
101 Online N2K-C2232PP
102 Online N2K-C2232PP
Sample vPC Verification
NX-5K1# show vpc
vPC domain id : 12
Peer status : peer adjacency formed ok
Peer keep-alive status : peer is alive
vPC peer-link status : up
Configuration consistency status : success
Modern Data Center Evolution
Before FEX
- Many standalone access switches
- Complex management
- Large spanning-tree domains
- High operational cost
After FEX
- Centralized management
- Simplified architecture
- Reduced STP complexity
- Lower power usage
- Scalable deployments
| Traditional Access | FEX Architecture |
|---|---|
| Independent Switches | Remote Line Cards |
| Multiple Configurations | Centralized Configuration |
| Complex STP | Reduced STP Complexity |
| Distributed Intelligence | Centralized Intelligence |
Modern Alternatives
Modern data centers increasingly use:
- VXLAN EVPN
- ACI Fabrics
- Leaf-Spine architectures
- 100G uplinks
- Cloud-scale fabrics
However, FEX remains extremely common in enterprise environments.
Advanced Troubleshooting
FEX Not Coming Online
- Image mismatch
- Unsupported NX-OS version
- Incorrect FEX ID
- Fabric interface down
- Port-channel mismatch
vPC Consistency Failure
- VLAN mismatch
- STP mismatch
- Port-channel inconsistency
- FEX ID mismatch
Fabric Ports Down
- Incorrect interface mode
- Fiber issues
- Speed mismatch
- Hardware incompatibility
Useful Debug Commands
show system internal fex info
show system internal vpc brief
show logging logfile
show interface counters errors
show hardware internal fex
Advanced Data Center Mathematics
Dual-Homed Redundancy
If one parent switch fails:
$$ Traffic \\rightarrow Remaining\\ Parent $$
Thus:
$$ Downtime \\approx 0 $$
Scalability Formula
If:
$$ n = Number\\ of\\ FEXes $$
Then:
$$ Scalability \\propto n $$
Educational Summary
What You Learned
- How Cisco FEX architecture works
- Difference between standalone switches and FEX
- How to configure single-homed FEX
- How to configure dual-homed FEX
- How vPC integrates with FEX
- How fabric interfaces operate
- How centralized switching architecture improves scalability
Related Articles
- Cisco Nexus vPC and LACP Configuration Guide Part 1
- Cisco Nexus EIGRP and HSRP Configuration Guide Part 2
- Cisco Nexus VRRP Configuration Guide Part 3
- Cisco Nexus vPC Peer Gateway and VDC Configuration Guide Part 4
Final Conclusion
You successfully configured:
- Cisco Nexus Fabric Extenders
- Single-homed FEX architecture
- Dual-homed FEX architecture
- vPC peer-links
- Fabric interfaces
- High-availability FEX connectivity
These technologies form the foundation of scalable Cisco Nexus enterprise and data center deployments.
Understanding FEX architecture is critical for engineers working with:
- Nexus 5000
- Nexus 7000
- Enterprise data centers
- Private cloud environments
- Virtualized infrastructures
No comments:
Post a Comment