This article explains why I think using Cisco’s Enhanced VPC may not be the best design when using dual-attached servers for two prime reasons:
- It does not add any additional redundancy to the design
- It does add complexity to the design.
I’ll assume you know what Port-Channel is and Cisco’s proprietary extension, VPC (Virtual Port Channel). Also assumed is that you are familiar with the Nexus 5000 and Nexus 2000 architecture. If not, check out the links and come back.
The particular scenario I want to begin the discussion with is the following example of Virtual Port Channel.
Firstly you have to realise that the Nexus 2000 FEXs are simply considered “line cards” in the Nexus 5000. There is no way to configure the Nexus 2000s except via the Nexus 5000. So stop thinking about them as stand alone switches – they are “Fabric Extenders”. Got it? The Nexus 2000s are so closely related to the Nexus 5000 that the only way to do a firmware upgrade on the Nexus 2000 is to upgrade the Nexus 5000.
Note: Hold that thought. We will visit it again later.
The 4x10G links connecting each Nexus 2000 to its respective Nexus 5000 is the backplane. The umbilical cord. It is configured as a regular Port Channel. Nothing virtual about it.
The Virtual Port Channel in this diagram is between the dual attached server and the Nexus 2000s – although since the Nexus 2000s are simply an extension of the Nexus 5000s, you probably should consider the VPC as being between the dual attached server and the Nexus 5000.
The resulting topology is a thing of beauty. The dual attached server has full connectivity even if any one of the Nexus 2000s or Nexus 5000s fail. The design is simple and life is good.
Of course, the single attached server does have a problem if either the Nexus 2000 or the Nexus 5000 fails, but clearly this is immaterial, because if high availability was important for that server, it would have been dual attached like its neighbour.
But still, it is for this special use-case (the single attached server) that Cisco invented Enhanced Virtual Port Channel.
Enhanced Virtual Port Channel hit the scene in about January 2012, with version 5.1(3)N1(1) NX-OS on the Nexus 55xx boxes , to allow a pair of Nexus 2000 Fabric Extender boxes (FEXs) to support a VPC to a dual attached server while at the same time supporting a second VPC upstream to the parent Nexus 50xxs. (Note, enhanced VPC is still not supported on the Nexus 5010, 5020 or Nexus 7000)
In other words, the umbilical cords have been split and spliced. And the hybrid twin child it bears is not simple.
True, this new topology has the additional high-availability feature that now allows the single attached host to still have connectivity if either of the Nexus 5000s fails. And network designers are so brainwashed and addicted to this double-crossing dual attached design between switches, that they look at it and think that it this is how it must be.
But there is danger lurking close by. And I will reveal it soon. But first, note that this scenario does nothing to help the dual-attached server. In both design 1 and design 2, any one of the Nexus devices could fail, and the dual attached server would be just fine. And in either scenario, should the upstream Nexus 2000 die that was connecting the single attached server, the sigle attached server is in trouble.
Remember, the only reason you would change from the first design to the second is to give the unimportant single attached server full connectivity if one of the Nexus 5000s failed.
|Key Point #1Enhanced VPCs do NOT give any additional redundancy to dual-attached servers.|
Remember that in the EVPC design, each Nexus 2000 is a child of two upstream Nexus 5000s – one is primary and one is secondary, but it is an Active-Active relationship, not an Active-Standby relationship that line cards normally have with supervisor modules. But in certain cases, the upstream Necus 5000s need to act as Active-Standby.
For instance, imagine that you are about to upgrade the firmware on the Nexus 5000s. Which of course also means that you are upgrading the Nexus 2000s, via the umbilical cord.
Keep your eye on design 2. You upgrade Nexus 5000 #1 which is the Primary switch of the VPC, and it will in fact upgrade first one Nexus 2000 then the other, without disruption in accordance with Cisco’s ISSU (In Service Software Upgrade). Once this is complete, you can upgrade the second Nexus 5000. And of course, there will not be a problem. There is never a problem upgrading software is there?
What happens if the software upgrade does go belly-up, and you find yourself with two Nexus 2000 switches in an unstable state? Or for some reason, the change in version caused a problem with the EVPC itself? Or…
Now it all comes down to risk. The EVPC design (design 2) has risk. IF, for some unexplained reason, the ISSU fails, you could potentially have two Nexus 2000 out of service. Not a big risk, ISSU has been around for over a year now. But the risk is higher in design 2 than it is in design 1.
In design 1, each Nexus 5000 has but one Nexus 2000 attached by umbilical cord. Should something fail during the upgrade, then one side of the path is lost, but the other is operational. No problem.
And my suspicion is that there is far less chance of error in the simple design 1 than in the twisted umbilical cord topology of design 2.
And that is why I wouldn’t bother with Enhanced VPC!
|Key Point #2Enhanced VPCs add complexity to the design without any benefit for dual attached servers.|
I haven’t even started on the price of this added complexity, but for starters, consider:
- Enhanced VPC means more configuration
- EVPC is more difficult to configure
- EVPC is much harder to troubleshoot
I’ll accept that Enhanced VPC is a valid design for single attached servers, and I’m happy for someone to explain to me why you would use this design if all servers are dual-attached.