Entering the Nexus

If you have a data center or if you have any services running in a data center then at some point you may have considered Cisco’s Nexus switching platform.  Rightly so, Cisco’s Nexus platform promises and delivers huge flexibility and scalability gains within the data center.

FabricPath

To provide this next leap in data center  switching Cisco has created FabricPath.  A proprietary switching protocol that in some ways replaces spanning tree (no STP inside).  You’ll still need STP for backward compatibility, but FabricPath essentially turns your network into… well… a Fabric.

Groups of switches that are connected using FabricPath are now seen as one  single switch. An internal protocol adds fabric-wide “intelligence” (cisco speak for virtualization).  This protocol provides:

• Optimized Conectivity
• Low Latency
• Any to Any
• High Bandwidth, high resiliency
• Open Management and troublehsooting
• FabricPath also provide additonal L3 integration

Enabling FabricPath on a switch or set of switches is easy.  Almost too easy.  Two config lines enable FabricPath on an interface.

NK(config)# interface ethernet 1/1
N7K(config-if)# switchport mode fabricpath


In our lab tests we ran into a few gotcha’s that seemed a bit funky that we wanted to share with you.

Licensing:  When enabling fiber-channels on our switch, while it’s not documented anywhere that we could find, only a single fiber-channel 8-port license could be installed on a given 5k-series switch.   This was convenient in a way, because it unlocked all ports on the switch; However, to be compliant a license must be purchased for every FC port that will be used….you just won’t be able to actually install subsequent licenses on the switch itself.  Weird.

FC ports: Again, while it doesn’t seem to be documented anywhere “official”…there seems to exist a limitation on what ports can be configured to be in native FC mode on a Nexus 5k.

Basically, within a given module, the range of FC ports must extend to the last physical port *of that module*.  In a 5596, the first 48 ports that are included by default are all considered a single module, thus the FC ports must start from whatever port you choose and then must extend to port 48 with no ethernet ports in the middle. If you have expansion modules (the 5596 can take 3 modules), then that restriction exists within the ports of that module only. And since Cisco didn’t think it was important enough to document *HOW* to put a given nexus switch port in FC mode, here is a support forum thread which details the procedure: https://supportforums.cisco.com/message/3352978

Cisco Nexus isn’t the only new “Fabric” technology in town- check out Brocade!

Brocade has their own fabric technology called Virtual Cluster Switching (VCS).  Included in the Brocade VDX line of data center switches, VCS may also change how data center networks are architected, deployed, and managed.

One advantage Brocade seems to have over Cisco is it’s more graceful entry into the data center.  Starting with two small Top-of-rack switches you can then slowly grow to a larger virtualized architecture as it suits your need. (with Cisco purchasing the access layer mean also pruchasing a 5K aggregation switch which will cost you)

Brocade VDX 6720 devices support the following:

• Classic 10Gig active-active access/aggregation.
• Scalable fabrics that enables large scale server vitualization deployments
• LAN/SAN convergence. Converges storage and IP data segments onto a single data center network. End to end FCoE, iSCSI, NAS storage traffic IP LAN.
• Allows the entire switch fabric to be managed as one switch

We haven’t tested the new Brocade hardware quite as much as the Cisco Nexus but so far from what we have seen, the Brocade VDX devices work as promised and look to be a serious player in the new battle for the data center switching fabric.

One thing is for sure, whether you choose Cisco Nexus, Brocade VDX or any other new fabric switching technology, switching the old school way will never be the same. Long live STP!

Are you implementing Nexus or Brocade in your environment?  If so, leave a comment and let us know your experience with it.