50% off GNS3 Network Simulation Guide

PACKT are having a promotion on the GNS3 Network Simulation Guide until March 6 2016

Use the code JKTKU50y when ordering to get the discount or use the link http://bit.ly/1PaN7fS to get 50% off the normal price!

My original post obout the publishing of the book will give you more details!

 

 

 

 

Posted in GNS3 WorkBench | Leave a comment

I Bought a MS Surface Book, so now I’m really P…d off with Apple

Why am I annoyed with Apple?

I’ll get back to that, later. But first, let me tell you about my “Surface Book” experience.

The Microsoft Surface Book was a compromise to begin with.  I wanted, in order of importance:

  1. Touch screen technology – with precision (ie with a fine-point stylus). I do powerpoint presentations every day for a living. And I want to be able to draw over the slides better than I can with an iPad.
  2. Lightweight.  The one thing I didn’t like about my old 17″ MacBook Pro was the 3Kg load. But I’d be OK with 2Kg.
  3. 1 TB SSD. Preferably 2TB
  4. Intel i7 processor. I run lots of VMs
  5. As large a screen as possible. 17″ would be good, but I know that that is impossible with a touch screen and still under 2Kg

The Surface Pro 4 and the Surface Book were by far the leading contenders on #1 & #2 above, but both fell short on the SSD – although I believe in the US you can buy a Surface Book with 1TB SSD.

So it was the slightly larger screen size that made me decide on the Surface Book over the Surface Pro4, even though both are smaller than I want.

And what a horrible story that has been so far.

I have to admit that I thought it was strange that the sales rep in the shop told me “Don’t leave it for long periods on standby”

Hello? This started ringing alarm bells. I’ve had a Mac Book Pro since 2009, and I only ever put it on standby.

And now you might begin to see where I am coming from. I have got used to beautiful precision and elegant notebooks. MacBooks to be sure.

So when I took my Surface Book out of the box, I could see that some effort had been gone to to try and emulate Apple packaging, but the device held within is not nearly as elegant.

But the real clanger was when I turned it on.  It booted up, but didn’t find any wireless networks. I couldn’t log onto the device. In fact, attempting to enter my email as a login-id was a challenge, because every time I hit the @ key I got quotes instead.

And the Date/Time was stuck on 21:39 Dec  12 2015!

SadSurface

So I got onto Microsoft Support (using a Mac). To their credit, I opened a chat session and had someone attend to me within 5 minutes, and that person even called my mobile from Seattle to avoid the tedium of typing.  But after “Holding down the Power Button and the Volume Up and Volume Down buttons simultaneously” a couple of times (which brought up a “Surface EUFI” screen – whatever that is) I was old that he best option was to “take it back to the store where you bought it“.

So I have bought a dud. And I feel so frustrated and angry – not so much with Microsoft but with Tim Cook and Apple.

Why Apple?

Because if Apple had kept up its history of innovation in Personal Computers, Apple Laptops would now be shipping with touch-screens and stylus pencils. 

And then the choice would have been easy.  Instead, Apple have abandoned PC Innovation and concentrated on the more glamorous phones and tablets, and I believe stifling PC development so they would not take sales from the iPads.

So now I’m stuck with MS quality, rather than Apple quality, and it is not a pretty place.

RedNectar

Postscript: 2015-01-15

I returned the Surface Book to the Microsoft Store today, and they gave me a refund without any hassle.  Turns out that they were much more familiar with the problem (they say it is more of a Windows 10 thing, rather than a Surface Book problem) at the MS store than they were on the support line, and they offered to fix it on the sport (which they did anyway) or give the refund I’d already asked for.

Throughout this exercise I must say that the support “effort” from Microsoft has been outstanding, even if the advise from Seattle was a little off the mark.

When they bring out the 1TB version in Aus, I might re-visit the store.

Posted in Mac OS X, Macintosh, Microsoft, opinion, rant, Surface | Tagged , , , , , , , | Leave a comment

Cisco ACI Tutorial 4 – All about Access Policies – the new “interface range” command

All about Access Policies – the new “interface range” command

Cisco ACI Tutorial – Part 4

Since you no longer have to configure ports on individual switches, but rather configure multiple ports on multiple switches from a central controller, some concepts that were reasonably straight forward, are a little more complex in ACI.  In particular, defining a range of interfaces on a switch ready to add common configuration could be achieved in Cisco IOS CLI using the interface range command.  But in ACI, those interfaces may exist on multiple switches, and are configured using the ACCESS POLICIES submenu under FABRIC POLICIES.

In this part of the tutorial I’ll explain how the APIC uses the concepts of ProfilesPolicies and Policy Groups to achieve something like what we might have once tried to configure (but not succeeded, because there was no way to include multiple switches in the interface range command) as:

configure terminal
 interface range switch1/eth/1/1-10, switch2/eth/1/1-10
  cdp enable
  speed 1000

I will then expand this concept to include Attachable Access Entity Profiles, Domains and VLAN Pools to expand the conceptual configuration above to also include:

  switchport trunk allowed vlans 1000-1010

And of course, continue the sample configuration example.

TIP:RedPoint2 If you have not read Cisco ACI Tutorial – Part #1, now would be a good time. It contains a diagram of the topology used in this tutorial and some important notes about conventions used. Similarly, you will get more familiar with the context of this tutorial if you also take a look at Parts #2 and #3 of this Cisco ACI Tutorial.

Recall that in Part #1, I presented a table that showed the planned usage of a set of ports on my test ACI Lab.  Here is is again.

Port range Use
1/1-2 External L3 Routers
1/3-4 External L2 Switches
1/5-10
1/5-7 10Gb/s
1/8-10 1Gb/s
 Single Attached Bare Metal Hosts
1/11-14 VPCs to ESXi  Hosts
1/15-20
1/15-17 10Gb/s
1/18-20 1Gb/s
Single Attached ESXi  Hosts
1/21-24 VPCs to UCS B Series
1/25-28 VPCs to Bare Metal Hosts
1/29-32 Port Channels to Bare Metal Hosts
1/33-40 FEX connections
1/41 SPAN Port
1/42-44 APIC attachments

For my example, I’ll assume that I want to prepare ports 1/5-10 on both leaf switches ready to connect to bare metal hosts (BMHs).  I’ll also assume that ports 5-7 will be connected to hosts with 10Gb/s NICs, and that ports 1/8-10 will go to hosts with 1Gb/s NICs.  In reality, I’d probably be connecting the hosts using a Virtual Port Channel (VPC) so to complete the example I’ll allocate ports 1/25-28 for VPCs to BMHs.

Note:RedPoint Keeping the configuration of ports the same on pairs of switches is going to make life easier, especially if VPCs are involved.

Switch Profiles, Switch Selectors, Interface Profiles and Access Port Selectors

As usual, I will begin with a picture.

SwProf-IntProf

First of all, keep in mind that the interface range concept now needs to include switch identifiers as well as slot/interface numbers, so you will see in the diagram above that there is an object known as a Switch Profile(*) that has at least one child entity called a Switch Selector.  These two combine to define a named range of switches which are rather useless unless they are linked to at least one Interface Profile(*) with its associated Access Port Selector entities.

Switch Profile can define a single (leaf) switch, or a number of switches, and this profile can be linked to a number of Interface Profiles defining a range of interfaces, and so it is a combination of the Interface Profile and the Switch Profile which is the more powerful replacement for the command:

interface range

But there is a couple of important differences between the ACI concept and the old IOS interface range

Firstly, in the ACI world, everything (including Switch/Interface ranges) is done through named objects.  And when it comes to defining interfaces, each physical port can only belong to one named Access Port Selector.  Any attempt to include the same port in a different Switch Profile+Selector plus Interface Profile+Access Port Selector combination will result in an error.

Which leads me to the other difference – with the IOS interface range command, the effect is transient.  Once I have completed the configuration steps I need, there is no evidence in my configuration that I’d used the range construct rather than configuring each interface individually.  And there would be no problem in using a command like interface range 1/1-10, do some configuration, then use the interface range 1/1-5 to modify just some of those ports.

This is not the case in the ACI world.  In the ACI world, it is possible to define a named Switch Profile+Selector combination that includes all your switches, and link it to an Interface Profile that includes an Access Port Selector that defines the range of ports 1/1-10.

And once you had done this, those first 10 ports on all of your switches are stuck with congruent configuration forever (meaning if you change any one, you will change them all).  In other words, the Switch Profile+Selector plus Interface Profile+Access Port Selector is NOT transient – it stays forever.

So the ACI construct is both more powerful (because you can group ports across multiple switches) and more inflexible (because these ports can be stuck in these groups, and hard to re-purpose).

To maximise flexibility, I recommend the following:

  • Define one Switch Profile for each individual leaf switch in your data centre.  It will have one Switch Selector that specifies that single switch.
  • Define one Switch Profile for each pair of leaf switch in your data centre that will be used for VPCs.  It will have two Switch Selectors each specifying one of the two switches.
    • This means you now have two kinds of Switch Profiles, one for Virtual Port Channels, and one for single-attached devices.
  • Define one and only one Interface Profile for each Switch Profile
  • Define one Access Port Selector for every single port that you intend to use in that particular Switch Profile.  In other words, DO NOT define interface ranges, like 1/1-10.  Instead, define 10 Interface Selectors, each with one port.  You will be happy you did the day you need to change the configuration for port 1/8 whilst leaving the others the same.

Now the the Switch Profile+Selector plus Interface Profile+Access Port Selector will need to have some policies associated with it, and this is where the concept of Interface Policy Groups comes in.

Interface Policy Groups and Interface Policies

In truth, this collective name is a little misleading – you can not define an Interface Policy Group as such, but you can define an Access Port Policy Group, or a PC Interface Policy Group or even a VPC Interface Policy Group.

 Key Point: There are three types of Interface Policy Groups – one of these is for directly attached devices, the other two for bundled interfaces.

  1. Access Port Policy Groups (APPGs) define a set of policies for a group of ports that are allowed to use the same VLAN set (via the AEP – discussed later)
  2. Port Channel Interface Policy Groups (PCIPGs) define a set of policies for a single port channel – which of course must be allowed to use the same VLAN set (via the AEP – discussed later).  This PCIPG becomes the identifier for the port-channel for later reference.
  3. Virtual Port Channel Interface Policy Groups (VPCIPGs) define a set of policies for a single virtual port channel – which of course must be allowed to use the same VLAN set (via the AEP – discussed later). This VPCIPG becomes the identifier for the port-channel for later reference.

At first look, an Interface Policy Group seems like an entity to allow you to collect together a group of similar polices (known collectively as Interface Policies) in a named entity that can later be applied to a group of ports. And this is partially true, but it is much more than that, especially when it comes to Port Channel Interface Policy Groups (PCIPGs) and to Virtual Port Channel Interface Policy Groups (VPCIPGs)

The illustration below shows two Interface Policy Groups, one is an Access Port Policy Group, and the other a VPC Interface Policy Group.  Each contains a set of Interface Policies that might seem appropriate to apply to ports that are intended to be used for attaching single or dual attached Bare Metal Hosts (BMHs). I hope the logic of my naming scheme and the completeness of the diagram will suffice for a full explanation of the relationships. I’ll assume you can work out what a CDP Policy called Disable-CDPPol will do.

 

As you can see, in each of these Interface Policy Groups, an interface policy for Link Level (where link speed is defined), CDPLLDP and for the VPC, LACP has been applied.  There are currently (v1.1) ten different polices that can be placed in an Interface Policy Group.

The following diagram shows how these Interface Policy Groups are linked via the Access Port Selector to the Switch Profile+Selector plus Interface Profile+Access Port Selector discussed earlier.

Note the important role that the Access Port Selector plays in this link.  It really doesn’t matter which Interface Profile a particular interface finds itself in, so long as on one hand it is linked to the correct switch identifier, and on the other hand is linked to the correct Interface Policy Group.  So what is really important above is that ports 1/5-7, on Leaf Switches 101 & 102 are linked to the 10GbpsBMH-APPG

This relationship is even more important for  Port Channel Interface Policy Groups  and Virtual Port Channel Interface Policy Groups, because a single PCIPG or VPCIPG can only refer to a set of ports that define a single Port Channel or Virtual Port Channel in much the same away as a single port-channel interface (such as interface port-channel1) can only refer to a single port-channel in IOS or NXOS.

Let me use an example to illustrate.

Suppose you were using interfaces 1/25-28 on both Leaf Switch 101 and 102 to create VPCs to three different Bare Metal Hosts. Although each of those ports require exactly the same set of policies, and same set of VLANs (discussed later), you will need to create three VPC Interface Policy Groups – one for each VPC – you can’t use the same Interface Policy Group and expect LACP to sort out the fact that it is port 25 on each switch that connects to BMH1, port 26 on each switch that connects to BMH2, and ports 27 and 28 on each switch that connects to BMH3.  Instead, as shown in the example above, each VPC Interface Policy Group has to be defined uniquely.

There is one more important piece of information that the Interface Policy Group provides.  And that is the set of VLANs that are permitted on the associated ports, and this is done by linking each and every Interface Policy Group with a very strange Global Policy element known as an Attachable Access Entity Profile(*)

The VLAN containers: Attachable Access Entity Profiles, Domains and VLAN Pools

To add the AEP(*) concept to the diagram, I’ll have to stretch it a little, so use the diagram below to mentally enhance part of the the top right hand side of the diagram above.

AEP-DomProf

Notice first that every Interface Policy Group needs to be associated with an Attachable Access Entity Profile (AEP).  Dwelling on this concept for a while helps conceptualise the name of the object. An Attachable Access Entity Profile. Or AEP (*) as it is commonly known.

With this concept in mind, you can think of an AEP as a collection point for a group of access ports, port-channels and virtual-port channels.  Possibly this collection of ports have been dedicated to a single Tenant (you probably will want at least one AEP per Tenant) so I have included the Tenant’s name in the name of the AEP above – RedNectar:GP-AEP.  The “GP” just means “General Purpose”, so for now just think of this AEP as a single Entity, or Entity Profile for Attachable things, like Access Ports and Port-Channels and Virtual Port-Channels. Hence – an Attachable Access Entity Profile.

But these ports also need access to a set of VLANs, which leads to the concepts of VLAN Pools and Domains.  You will get a better mental picture if you think of Domains(*) as VLAN Domains, as I will explain shortly.

A VLAN Pool is nothing more than a collection of VLAN IDs assigned in blocks.  And every (VLAN) Domain needs a single VLAN Pool associated with it.  And every AEP can be associated with a bunch of (VLAN) Domains.

So this makes the AEP a meeting point for VLANs and ports (sorry – Attachable Access Entities) to get together and party. Any VLAN in and Domain that needs access to any port/port-channel will need to be at the party, and of course the corresponding port/port channel will also need to be present.

Attachable Access Entity Party

A more formal description of an AEP is that an AEP is a container that allows you to define which VLANs – via its relationship to Domains – are allowed to exist on which Attachable Entities (Ports/Port Channels/Virtual Port-Channels) – via its relationship to Interface Profiles

So an AEP is more or less the equivalent of:

switchport trunk allowed vlan 10-999, 1000-1999

If you wanted to specifically specifically restrict a port (or PC/VPC) to a single specific VLAN (say VLAN 100), you would need to:

  1. Create a VLAN Pool that had only one VLAN (VLAN 100)
  2. Create a Domain of the appropriate type (see below) – let’s call it VLAN100-PhysDom
  3. Allocate that VLAN Pool to the VLAN100-PhysDom Domain
  4. Create an AEP – let’s call it OneVLAN-AEP
  5. Allocate ONLY the VLAN100-PhysDom to OneVLAN-AEP
  6. Link the Interface Profile that defined the particular port (or PC/VPC) that you wished to restrict to the OneVLAN-AEP

Which would be the equivalent of (in IOS/NXOS terminology)

switchport trunk allowed vlan 100

And should you every wish to expand the AEP to allow more VLANs, you would need to either:

  1. Add more VLANs to the VLAN Pool; or
  2. Add more Domains to the AEP

A word about Domains

As I mentioned earlier, Domains(*) are a place to deposit a bunch of VLANs via a VLAN Pool.  But they also serve another purpose, and that is to link the physical configuration (via the AEP) to the Logical Tenant configuration.  Just how this is done depends on the type of Domain involved. And there are four type of Domains – and remember, each Domain is going to define a set of available VLANs.

  1. Physical Domains provide a mechanism to permit a set of VLANs on a given port (or PC/VPC) where a directly attached hosts or switch might be attached. This makes it possible to then map one of the allowed VLANs on a particular port (or PC or VPC) to a particular EPG.  This is discussed more in Part #5 of this tutorial.
  2. VMM Domains (Vritual Machine Manager Domains) do a little more than just provide a mechanism to permit a set of VLANs on a given port (or PC/VPC) where a directly attached (e.g.) ESXi host might be attached. A VMM Domain also provides a mechanism to integrate with the related VMM (either VMware vCenter/vShield or Microsoft SCVMM as of 2015) so that the VLANs associated with the VMM can be dynamically allocated and automatically allocated to EPGs. This is discussed more in Part #5 of this tutorial.
  3. Layer 2 External Domains provide a mechanism to permit a set of VLANs on a given port (or PC/VPC) where a External Bridged Network (aka L2Out) connects.  This makes it possible to then map one of the allowed VLANs on a particular port (or PC or VPC) to a Node Profile/Interface Profile.  This is discussed more in Part #6 of this tutorial.
  4. Layer 3 External Domains provide a mechanism to permit a set of VLANs on a given port (or PC/VPC) where a External Routed Network (aka L3Out) connects.  This makes it possible to then map one of the allowed VLANs on a particular port (or PC or VPC) to a Node Profile/Interface Profile.  This is discussed more in Part #6 of this tutorial.
TIP:RedPoint2  Life will be simpler if you only configure one AEP per Tenant, (hence the naming convention used) and within each Tenant one Physical Domain, one Layer 3 External Routed Domain and a VMM Domain for each type of hypervisor. Each Domain is given a non-overlapping range of VLANs, and all Domains are linked to the Tenant’s common AEP. If you have overlapping VLANs, you’ll need a second AEP for that Tenant.

Time to start configuring. It is going to be easier if I start by defining the re-usable general purpose objects (Interface Policies and a VLAN Pool), then a Physical Domain to hold the VLAN Pool, then move on to the physical side of things by creating an Attachable Access Entity Profile(*) then do Interface Profiles/Access Port Selectors and work back to Switch Profiles/Selectors.  In other words, I will work more-or-less backwards to the sequence I used to explain the concepts.

Configuration Steps

Interface Policies will only ever need to be configured once (unless new options are added) and will be common across all Tenants.

TIP:RedPoint2  Create a standard set of Interface Policies and use my Naming Convention (I hope to write a more complete post about this in the future).

Once you have your set of Interface Policies in place, avoid using Wizards, because they may re-create duplicates of the policies you have.

Step 1: Add new Link Level Interface Policies for 1Gbps and 10Gbps

Link Level Policy is the host interface policy, which specifies the layer 1 parameters of host facing ports. [Adapted from ACI Help]

A Link Level Interface Policy contains the Physical Layer policies applied to a port when a linked Interface Policy Group is applied to a physical interface via an Access Port Selector. [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Interface Policies > Polices > Link Level

  • (+) Create Link Level Policy
    • Name: 1Gbps-Link
      • Speed: 1 Gbps
  • (+) Create Link Level Policy
    • Name: 10Gbps-Link
      • Speed: 10 Gbps

 Step 2: Add new CDP Interface Policies to enable/disable CDP

The CDP interface policy is primarily used to obtain protocol addresses of neighboring devices and discover the platform of those devices. CDP can also be used to display information about the interfaces your router uses. CDP is media- and protocol-independent, and runs on all Cisco-manufactured equipment including routers, bridges, access servers, and switches [From ACI Help]

A CDP  Interface Policy contains the CDP policy (enabled/disabled) applied to a port when a linked Interface Policy Group is applied to a physical interface via an Access Port Selector. [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Interface Policies > Polices > CDP Interface

  • (+) Create CDP Interface Policy
    • Name: Enable-CDP
      • Admin State: (x) Enabled
  • (+) Create CDP Interface Policy
    • Name: Disable-CDP
      • Admin State: (x) Disabled

 Step 3: Add new LLDP Interface Policies to enable/disable LLDP

The LLDP interface policy defines a common configuration that will apply to one or more LLDP interfaces. LLDP uses the logical link control (LLC) services to transmit and receive information to and from other LLDP agents. [From ACI Help]

An LLDP  Interface Policy contains the LLDP policy (Tx/Rx enabled/disabled) applied to a port when a linked Interface Policy Group is applied to a physical interface via an Access Port Selector. [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Interface Policies > Polices > LLDP Interface

  • (+) Create LLDP Interface Policy
    • Name: Enable-LLDP
      • Receive State: (x) Enabled
      • Transmit State: (x) Enabled
  • (+) Create LLDP Interface Policy
    • Name: Disable-LLDP
      • Receive State: (x) Disabled
      • Transmit State: (x) Disabled

 Step 4: Add new PortChannel Policies for LACP variations

The Port Channel policy enables you to bundle several physical ports together to form a single port channel. LACP enables a node to negotiate an automatic bundling of links by sending LACP packets to the peer node. [From ACI Help]

A PortChannel Interface Policy contains the policies to be used when bundling pairs of ports together to from a PortChannel or Virtual PortChannel. The policy is applied to a port when a linked Interface Policy Group is applied to a physical interface via an Access Port Selector. [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Interface Policies > Polices > PortChannel Policies

  • (+) Create PortChannel Policy
    • Name: Static-PC
      • Mode: (x) Static Channel – Mode On
  • (+) Create PortChannel Policy
    • Name: ActiveLACP-PC
      • Mode: (x) LACP Active
  • (+) Create LACP Interface Policy
    • Name: PassiveLACP-PC
      • Mode: (x) LACP Passive
  • (+) Create LACP Interface Policy
    • Name: MacPinning-PC
      • Mode: (x) Mac Pinning

VLAN Pools can be considered re-usable objects, but I would advise creating a new VLAN Pool for every Domain. Otherwise, if you have the same VLAN Pool linked to multiple Domains, changing the VLAN Pool will affect all those Domains. Hence, you will see that in the naming of the pools below, I have included the Tenant name as part of the VLAN Pool name

Step 5: Create VLAN Pools.

The VLAN range namespace policy defines for ID ranges used for VLAN encapsulation. [From ACI Help]

A VLAN Pool defines a set of VLAN IDs used which can be made available to a Domain (Physical, External Bridged, External Routed or VMM). [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Pools > VLAN

  • (+) Create VLAN Pool
    • Name: RedNectar:BMH-VLAN.Pool
    • Allocation Mode: (x) Static Allocation
    • (+) Encap Blocks
      • Range 10 to 10
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]
    • (+) Encap Blocks
      • Range 11 to 11
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]
    • (+) Encap Blocks
      • Range 12 to 12
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]
    • (+) Encap Blocks
      • Range 13 to 13
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]
    • (+) Encap Blocks
      • Range 14 to 14
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]
    • (+) Encap Blocks
      • Range 15 to 15
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]
TIP:RedPoint2  You probably have guessed that the above Encap Blocks could have been added in a single range: 10-15. However, by adding Static VLAN IDs individually, you will have more flexibility in the future if you even need to remove any of the VLANs from this pool.
  • (+) Create VLAN Pool
    • Name: RedNectar:ExtL2-VLAN.Pool
    • Allocation Mode: (x) Static Allocation
    • (+) Encap Blocks
      • Range 1000 to 1000
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]
    • [Repeat substituting the numbers 1-5 for <n> in the following]
    • (+) Encap Blocks
      • Range 100<n> to 100<n>
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]
  • (+) Create VLAN Pool
    • Name: RedNectar:ExtL3-VLAN.Pool
    • Allocation Mode: (x) Static Allocation
    • (+) Encap Blocks
      • Range 2000 to 2000
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]
    • [Repeat substituting the numbers 1-5 for <n> in the following]
    • (+) Encap Blocks
      • Range 200<n> to 200<n>
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]
  • (+) Create VLAN Pool
    • Name: RedNectar:vCenter-VLAN.Pool
    • Allocation Mode: (x) Dynamic Allocation [Default]
    • (+) Encap Blocks
      • Range 3000 to 3966 [#Because VLAN 3967 is the recommended value for the Infrastructure VLAN, and VLAN IDs above 3967 are not supported on all Cisco products]
      • Allocation Mode: (x) Inherit allocMode from Parent [Default]

Now move on to the physical configuration, starting with a Physical Domain – I will create the other Domains (External Bridged, External Routed and VVM Domains when they are required)

Step 6: Create a Physical Domain for your Tenant.

The physical domain profile stores the physical resources (ports and port-channels) and encap resources (VLAN/VXLAN) that should be used for endpoint groups associated with this domain. [From ACI Help]

A Physical Domain stores the encapsulation resources (VLAN IDs) and, via any associated Attachable Access Entity Profiles, the physical resources (ports and [virtual] port-channels) that may be used with EndPoint Groups associated with this Domain. [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Physical and External Domains > Physical Domains

  • (+) Create Physical Domain
    • Name: RedNectar:BMH-PhysDom
    • Associated Attachable Entity Profile(*): [Leave Blank – you haven’t created one yet]
    • VLAN Pool: RedNectar:BMH-VLAN.Pool

Step 7: Create an Attachable Access Entity Profile for your Tenant.

The attached entity profile(*) provides a template to deploy hypervisor policies on a large set of leaf ports. This also provides the association of a Virtual Machine Management (VMM) domain(*) and the physical network infrastructure. [Adapted from ACI Help]

The Attachable Access Entity Profile provides a connection between the physical resources (ports and [virtual] port-channels) via associated Interface Policy Groups to a range of available encapsulation resources (VLAN IDs) via Domain associations. [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Global Policies > Attachable Access Entity Profiles

  • (+) Create Attachable Access Entity Profile
    • Name: RedNectar:GP-AEP
    • Enable Infrastructure VLAN: [x] Checked [Default]
    • (+) Domains (VMM, Physical or External) To Be Associated To Interfaces:
      • Domain Profile: RedNectar:BMH-PhysDom

Before beginning to define Interface Policy Groups, let’s take another look at the planned interface uses from the table in Part #1:

Port range Use
1/1-2 External L3 Routers
1/3-4 External L2 Switches
1/5-10
1/5-7 10Gb/s
1/8-10 1Gb/s
 Single Attached Bare Metal Hosts
1/11-14 VPCs to ESXi  Hosts
1/15-20
1/15-17 10Gb/s
1/18-20 1Gb/s
Single Attached ESXi  Hosts
1/21-24 VPCs to UCS B Series
1/25-28 VPCs to Bare Metal Hosts
1/29-32 Port Channels to Bare Metal Hosts
1/33-40 FEX connections
1/41 SPAN Port
1/42-44 APIC attachments

I will deal with connections to External L2/L3 devices and the SPAN port later, so for now I will start with the requirement that we have single attached Bare Metal Hosts, some at 10Gbps and some at 1Gbps.  Similarly, we have some single attached ESXi hosts at 10Gbs and 1Gbps.

For reasons that I will explain fully in Part #5,  we will want CDP and LLDP to be enabled when connecting to an ESXi host (basically because the connection will be to a virtual switch inside the ESXi host) whereas I would normally disable these protocols when connection to a host.  This means we will need four different Access Port Policy Groups to cater for the combinations of link speed and CDP/LLDP policy.  I will name these:

  • RedNectar:1GbpsBMH-APPG
  • RedNectar:10GbpsBMH-APPG
  • RedNectar:1GbpsESXi-APPG
  • RedNectar:10GbpsESXi-APPG

Step 8: Create Access Port Policy Groups for RedNectar’s single attached hosts

The interface policy group enables you to specify the interface policies you want to use. [From ACI Help]

The Access Port Policy Group enables you to specify the interface policies you want to use with a particular set of ports (when the Policy Group is linked to an Access Port Selector), and enables you to specify the VLANs permitted on those ports via the associated Attachable Access Entity Profile. [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Interface Policies > Policy Groups

  • (+) Create Access Port Policy Group
    • Name: RedNectar:1GbpsBMH-APPG
      • Link Level Policy: 1Gbps-Link
      • CDP Policy: Disable-CDP
      • LLDP Policy: Disable-LLDP
      • Attached Entity Profile: RedNectar:GP-AEP
  • (+) Create Access Port Policy Group
    • Name: RedNectar:10GbpsBMH-APPG
      • Link Level Policy: 10Gbps-Link
      • CDP Policy: Disable-CDP
      • LLDP Policy: Disable-LLDP
      • Attached Entity Profile: RedNectar:GP-AEP
  • (+) Create Access Port Policy Group
    • Name: RedNectar:1GbpsESXi-APPG
      • Link Level Policy: 1Gbps-Link
      • CDP Policy: Enable-CDP
      • LLDP Policy: Enable-LLDP
      • Attached Entity Profile: RedNectar:GP-AEP
  • (+) Create Access Port Policy Group
    • Name: RedNectar:10GbpsESXi-APPG
      • Link Level Policy: 10Gbps-Link
      • CDP Policy: Enable-CDP
      • LLDP Policy: Enable-LLDP
      • Attached Entity Profile: RedNectar:GP-AEP

Some servers will be attached via VPCs and PCs, so I will configure these next.

 Key Point: Unlike directly attached devices where a “general” Access Port Policy Group can be used for multiple attachments, a VPC Interface Policy Group or PC Interface Policy Group will need to be created for each VPC/PC.

Recall that the following ports will be used for VPCs and PCs

Port range Use
1/11-14 VPCs to ESXi  Hosts
1/21-24 VPCs to UCS B Series
1/25-28 VPCs to Bare Metal Hosts
1/29-32 Port Channels to Bare Metal Hosts

Step 9: Create PC and VPC Interface Policy Groups for BMH and ESXi attached hosts

The bundle interface group enables you to specify the interface policy you want to use. [From ACI Help]

The bundle (VPC or PC) Interface Policy Group defines a VPC or PC instance and enables you to specify the interface policies you want to use with the specific bundle of ports (when the Policy Group is linked to an Access Port Selector), and enables you to specify the VLANs permitted on those ports via the associated Attachable Access Entity Profile. [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Interface Policies > Policy Groups

  • (+) Create VPC Interface Policy Group
    • Name: RedNectar:10GbpsBMH1-VPCIPG
      • Link Level Policy: 10Gbps-Link
      • CDP Policy: Disable-CDP
      • LLDP Policy: Disable-LLDP
      • LACP Policy: ActiveLACP-PC
      • Attached Entity Profile: RedNectar:GP-AEP

Now repeat for each BMH VPC – substituting the numbers 2-4 for <n> in the following

  • (+) Create VPC Interface Policy Group
    • Name: RedNectar:10GbpsBMH<n>-VPCIPG
      • Link Level Policy: 10Gbps-Link
      • CDP Policy: Disable-CDP
      • LLDP Policy: Disable-LLDP
      • LACP Policy: ActiveLACP-PC
      • Attached Entity Profile: RedNectar:GP-AEP

Now repeat for each ESXi VPC – substituting the numbers 1-4 for <n> in the following

  • (+) Create VPC Interface Policy Group
    • Name: RedNectar:10GbpsESXi<n>-VPCIPG
      • Link Level Policy: 10Gbps-Link
      • CDP Policy: Enable-CDP
      • LLDP Policy: Enable-LLDP
      • LACP Policy: ActiveLACP-PC
      • Attached Entity Profile: RedNectar:GP-AEP

And again for VPCs to UCS B Series chassis (ports 1/21-24).  In this case, I will assume that at least some of the servers in the chassis will be ESXi servers, so I will ensure CDP and LLDP is enabled (for reasons I’ll explain in Part #5).  Also, connections to UCS B Series will be via devices known as Fabric Interconnects.  If you are now familiar with the UCS B series or Fabric Interconnects, see Cisco’s descriptions here. These Fabric Interconnects are known as FI-A and FI-B.  I will assume that there will be 2x10G links to FI-A from each leaf, and 2x10G links to FI-B, so only one VPCIPG is required, but I will still use the number “1” in the name to allow for a future number “2” VPC.

  • (+) Create VPC Interface Policy Group
    • Name: RedNectar:10GbpsFIA1-VPCIPG
      • Link Level Policy: 10Gbps-Link
      • CDP Policy: Enable-CDP
      • LLDP Policy: Enable-LLDP
      • LACP Policy: ActiveLACP-PC
      • Attached Entity Profile: RedNectar:GP-AEP
  • (+) Create VPC Interface Policy Group
    • Name: RedNectar:10GbpsFIB1-VPCIPG
      • Link Level Policy: 10Gbps-Link
      • CDP Policy: Enable-CDP
      • LLDP Policy: Enable-LLDP
      • LACP Policy: ActiveLACP-PC
      • Attached Entity Profile: RedNectar:GP-AEP

Finally, the port-channel interfaces (on port 1/29-32 of each switch) will need Interface Policy Groups – for my example I will use pairs of ports, so I will need four PCIPGs (Port Channel Interface Policy Groups), two for each leaf switch.

  • (+) Create PC Interface Policy Group
    • Name: RedNectar:10GbpsBMH1-PCIPG
      • Link Level Policy: 10Gbps-Link
      • CDP Policy: Disable-CDP
      • LLDP Policy: Disable-LLDP
      • LACP Policy: ActiveLACP-PC
      • Attached Entity Profile: RedNectar:GP-AEP

Now repeat for each BMH PortChannel – substituting the numbers 2-4 for <n> in the following

  • (+) Create PC Interface Policy Group
    • Name: RedNectar:10GbpsBMH<n>-PCIPG
      • Link Level Policy: 10Gbps-Link
      • CDP Policy: Disable-CDP
      • LLDP Policy: Disable-LLDP
      • LACP Policy: ActiveLACP-PC
      • Attached Entity Profile: RedNectar:GP-AEP

 

That completes the VPC/PC Interface Policy Groups. You will need these when you add Access Port Selectors to the Interface Profiles that I will configure next.

First of all, recall my earlier advice – only I have re-ordered the options to suit the order of configuration.

TIP:RedPoint2
  • Define one and only one Interface Profile for each Switch Profile
  • Define one Access Port Selector for every single port that you intend to use in that particular Switch Profile.  In other words, DO NOT define interface ranges, like 1/1-10.  Instead, define 10 Interface Selectors, each with one port.  You will be happy you did the day you need to change the configuration for port 1/8 whilst leaving the others the same.
  • Define one Switch Profile for each individual leaf switch in your data center.  It will have one Switch Selector that specifies that single switch.
  • Define one Switch Profile for each pair of leaf switch in your data center that will be used for VPCs.  It will have two Switch Selectors each specifing one of the two switches.
    • This means you now have two kinds of Switch Profiles, one for Virtual Port Channels, and one for single-attached devices.

 

Step 10: Create Interface Profiles for each switch collection

The interface profile enables you to specify the interface you want to configure.

The Host Port Selector(*) is used for grouping ports between the node and the host (such as hypervisor). [From ACI Help]

The Interface Profile enables you to group together a collection of Access Port Selectors that need to be linked to a Switch Profile.

The Access Port Selector is used to link specific ports in the Interface Profile to an Interface Policy Group. [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Interface Policies > Profiles

  • #Start with an Interface Profile that will define the (pairs of) ports used for VPCs
  • (+) Create Interface Profile
    • Name: 101..102-IntProf
      • #Start with the four VPCs for ESXi hosts
      • Add (+) Interface Selectors:
        • Name: 1:11
        • Interface IDs: 1/11
        • Interface Policy Group: RedNectar:10GbpsESXi1-VPCIPG
      • Add (+) Interface Selectors:
        • Name: 1:12
        • Interface IDs: 1/12
        • Interface Policy Group: RedNectar:10GbpsESXi2-VPCIPG
      • Add (+) Interface Selectors:
        • Name: 1:13
        • Interface IDs: 1/13
        • Interface Policy Group: RedNectar:10GbpsESXi3-VPCIPG
      • Add (+) Interface Selectors:
        • Name: 1:14
        • Interface IDs: 1/14
        • Interface Policy Group: RedNectar:10GbpsESXi4-VPCIPG
      • #Next, the interfaces connected to Fabric Interconnect-A and Fabric Interconnect-B – these could be added individually for even greater flexibility.
      • Add (+) Interface Selectors:
        • Name: 1:21..22
        • Interface IDs: 1/21-22
        • Interface Policy Group: RedNectar:10GbpsFIA1-VPCIPG
      • Add (+) Interface Selectors:
        • Name: 1:23..24
        • Interface IDs: 1/23-24
        • Interface Policy Group: RedNectar:10GbpsFIB1-VPCIPG
      • #Finally, the four VPCs connecting to BMHs
      • Add (+) Interface Selectors:
        • Name: 1:25
        • Interface IDs: 1/25
        • Interface Policy Group: RedNectar:10GbpsBMH1-VPCIPG
      • Add (+) Interface Selectors:
        • Name: 1:26
        • Interface IDs: 1/26
        • Interface Policy Group: RedNectar:10GbpsBMH2-VPCIPG
      • Add (+) Interface Selectors:
        • Name: 1:27
        • Interface IDs: 1/27
        • Interface Policy Group: RedNectar:10GbpsBMH3-VPCIPG
      • Add (+) Interface Selectors:
        • Name: 1:28
        • Interface IDs: 1/28
        • Interface Policy Group: RedNectar:10GbpsBMH4-VPCIPG

Each individual switch will need it’s own Interface Profile for single attached devices and ordinary Port Channel.

  • #Start with an Interface Profile for Leaf Switch 101
  • (+) Create Interface Profile
    • Name: 101-IntProf
      • #Start by adding Interface Selectors for 10Gbps attached BMHs
      • Add (+) Interface Selectors:
        • Name: 1:05 #The leading 0 keeps things in order
        • Interface IDs: 1/5
        • Interface Policy Group: RedNectar:10GbpsBMH-APPG
      • [Repeat substituting the numbers 06-07 for <n> in the following]
      • Add (+) Interface Selectors:
        • Name: 1:<n>
        • Interface IDs: 1/<n>
        • Interface Policy Group: RedNectar:10GbpsBMH-APPG
      • #Next, add Interface Selectors for 1Gbps attached BMHs
      • [Repeat substituting the numbers 08-10 for <n> in the following]
      • Add (+) Interface Selectors:
        • Name: 1:<n>
        • Interface IDs: 1/<n>
        • Interface Policy Group: RedNectar:1GbpsBMH-APPG
      • #Next, add Interface Selectors for 10Gbps attached ESXi hosts
      • [Repeat substituting the numbers 15-17 for <n> in the following]
      • Add (+) Interface Selectors:
        • Name: 1:<n>
        • Interface IDs: 1/<n>
        • Interface Policy Group: RedNectar:10GbpsESXi-APPG
      • #Next, add Interface Selectors for 1Gbps attached ESXi hosts
      • [Repeat substituting the numbers 18-20 for <n> in the following]
      • Add (+) Interface Selectors:
        • Name: 1:<n>
        • Interface IDs: 1/<n>
        • VPC Policy Group: RedNectar:1GbpsESXi-APPG
      • #Finally, add Interface Selectors for the PortChannel bundles
      • Add (+) Interface Selectors:
        • Name: 1:29..30
        • Interface IDs: 1/29-30
        • VPC Policy Group: RedNectar:10GbpsBMH1-PCIPG
      • Add (+) Interface Selectors:
        • Name: 1:31..32
        • Interface IDs: 1/31-32
        • VPC Policy Group: RedNectar:10GbpsBMH2-PCIPG
  • #Now create an Interface Profile for Leaf Switch 102.  It will be ALMOST identical to the 101-IntProf except for the PortChannel Interfaces which will carry the numbers 3 & 4 in their names rather than 1 & 2.
  • (+) Create Interface Profile
    • Name: 101-IntProf
      • #Start by adding Interface Selectors for 10Gbps attached BMHs
      • Add (+) Interface Selectors:
        • Name: 1:05
        • Interface IDs: 1/5
        • Interface Policy Group: RedNectar:10GbpsBMH-APPG
      • [Repeat substituting the numbers 06-07 for <n> in the following]
      • Add (+) Interface Selectors:
        • Name: 1:<n>
        • Interface IDs: 1/<n>
        • Interface Policy Group: RedNectar:10GbpsBMH-APPG
      • #Next, add Interface Selectors for 1Gbps attached BMHs
      • [Repeat substituting the numbers 08-10 for <n> in the following]
      • Add (+) Interface Selectors:
        • Name: 1:<n>
        • Interface IDs: 1/<n>
        • Interface Policy Group: RedNectar:1GbpsBMH-APPG
      • #Next, add Interface Selectors for 10Gbps attached ESXi hosts
      • [Repeat substituting the numbers 15-17 for <n> in the following]
      • Add (+) Interface Selectors:
        • Name: 1:<n>
        • Interface IDs: 1/<n>
        • Interface Policy Group: RedNectar:10GbpsESXi-APPG
      • #Next, add Interface Selectors for 1Gbps attached ESXi hosts
      • [Repeat substituting the numbers 18-20 for <n> in the following]
      • Add (+) Interface Selectors:
        • Name: 1:<n>
        • Interface IDs: 1/<n>
        • VPC Policy Group: RedNectar:1GbpsESXi-APPG
      • #Finally, add Interface Selectors for the PortChannel bundles
      • Add (+) Interface Selectors:
        • Name: 1:29~30
        • Interface IDs: 1/29-30
        • VPC Policy Group: RedNectar:10GbpsBMH3-PCIPG
      • Add (+) Interface Selectors:
        • Name: 1:31~32
        • Interface IDs: 1/31-32
        • VPC Policy Group: RedNectar:10GbpsBMH4-PCIPG

Now that the Interface Profiles have been created, Switch Profiles need to be created so they can be matched to appropriate Interface Profiles.

Step 11: Create Switch Profiles for each switch collection

The node profile(*) enables you to specify which nodes (Example: a leaf) to configure. [From ACI Help]

The Switch Profile enables you to define multiple Switch Selectors which in turn define a group of leaf switches that share the same Interface Profile(s). [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Switch Policies > Profiles

  • #First, create a Switch Profile for the VPC pairs
  • (+) Create Switch Profile
    • Name: 101..102-SwProf
      • Add (+) Switch Selectors:
        • Name: 101-SS
        • Blocks: 101
      • Add (+) Switch Selectors:
        • Name: 102-SS
        • Blocks: 102
  • #Next a Switch Profile for Leaf Switch 101
  • (+) Create Switch Profile
    • Name: 101-SwProf
      • Add (+) Switch Selectors:
        • Name: 101-SS
        • Blocks: 101
  • #Finally a Switch Profile for Leaf Switch 102
  • (+) Create Switch Profile
    • Name: 102-SwProf
      • Add (+) Switch Selectors:
        • Name: 102-SS
        • Blocks: 102

It would have been possible to complete the associations as step 2 of the Switch Profile creation wizard – but I avoid wizard, so I have configured the associations as a separate step.

Step 12: Associate the Interface Profiles with the Switch Profiles

FABRIC > ACCESS POLICIES > Switch Policies > Profiles > 101..102-SwProf

  • Add (+) Associated Interface Selector(*) Profile
    • Interface Select Profile(*): 101..102-IntProf

FABRIC > ACCESS POLICIES > Switch Policies > Profiles > 101-SwProf

  • Add (+) Associated Interface Selector(*) Profile
    • Interface Select Profile(*): 101-IntProf

FABRIC > ACCESS POLICIES > Switch Policies > Profiles > 102-SwProf

  • Add (+) Associated Interface Selector(*) Profile
    • Interface Select Profile(*): 102-IntProf

Although I have created the appropriate Interface Policy Groups to define each of the VPCs, it is still necessary to create an Explicit VPC Protection Group which will define the anycast address that will be used by the fabric for any VPC existing on this pair of switches.

Step 13: Create an Explicit VPC Protection Group

The VPC protection policy is a container of VPC protection groups; it enables you to select a pairing type for creating the protection groups.

VPC Protection Groups
A VPC auto-protection group. This represents a VPC domain (a protection group). You can set the type of algorithm to be used for creating the automatic groups. Based on the algorithm, APIC implicitly makes discovered nodes part of the group. [From ACI Help]

The VPC Port Channel Security Policy – Virtual Port Channel default is a container of VPC Protection Groups and defines the Pairing Type; explicit (pairs chosen by user), consecutive (101+102, 103+104,…) or reciprocal (101+103, 102+104,…)

VPC Protection Groups define pairs of switches that participate in Virtual Prot Channels. [RedNectar’s Definition]

FABRIC > ACCESS POLICIES > Switch Policies > Policies > Virtual Port Channel default

  • Add (+) Explicit VPC Protection Group
    • Name: 101..102-ExpVPC.ProtGrp
    • ID: 12 #To represent Leaf1 & Leaf2
    • Switch 1: 101
    • Switch 2: 102

Step 14: Check your configuration

The following sequence is a great way to troubleshoot Access Policies.  The idea is that you start with Switch Profiles, and work through to Domains.

FABRIC > ACCESS POLICIES > Switch Policies > Profiles

SwProfiles

Check that each Switch Profile is associated to the correct leaf switches, and has an Interface Profile associated with it

FABRIC > ACCESS POLICIES > Interface Policies > Profiles

Check that the correct interfaces have been assigned

FABRIC > ACCESS POLICIES > Interface Policies > Profiles > 101-IntProf
FABRIC > ACCESS POLICIES > Interface Policies > Profiles > 101..102-IntProf
FABRIC > ACCESS POLICIES > Interface Policies > Profiles > 102-IntProf

IntProf-IntPolicyGroupCheck each Interface Profile in turn, and check that each has been assigned to an Interface Policy Group, and that it is the correct Interface Policy Group.

FABRIC > ACCESS POLICIES > Interface Policies > Policy Groups

CheckIntPolGroups

Check each Interface Policy Group, and most particularly, make sure that it is linked to an Attachable Access Entity Profile (labeled ATTACHED ENTITY PROFILE).  Can you spot my deliberate mistake?

FABRIC > ACCESS POLICIES > Global Policies > Attachable Access Entity Profiles > RedNectar:GP-AEP

CheckAEP

Check that the AEP is linked to the Physical Domain

FABRIC > ACCESS POLICIES > Global Policies > Attachable Access Entity Profiles > RedNectar:GP-AEP | [OPERATIONAL]

CheckAEP-Operational

This is a handy place to reverse-check which Interface Policy Groups are linked to this AEP. Can you see that the mistake I pointed out earlier has been fixed?

FABRIC > ACCESS POLICIES > Physical and External Domains > Physical Domains > RedNectar:BMH-PhysDom

Check that the Domain is linked to a VLAN Pool.

FABRIC > ACCESS POLICIES > Pools > VLAN

Check VLAN

Check that the VLANS are correct for each pool

Step 15: Verify your physical connections

You can check that the physical ports and PC/VPCs are up by going to:

FABRIC > INVENTORY > Pod1 > Leaf1 (Node-101) > Interfaces > Physical Interfaces
FABRIC > INVENTORY > Pod1 > Leaf1 (Node-101) > Interfaces > PC Interfaces
FABRIC > INVENTORY > Pod1 > Leaf1 (Node-101) > Interfaces > VPC Interfaces
FABRIC > INVENTORY > Pod1 > Leaf1 (Node-102) > Interfaces > Physical Interfaces
FABRIC > INVENTORY > Pod1 > Leaf1 (Node-102) > Interfaces > PC Interfaces
FABRIC > INVENTORY > Pod1 > Leaf1 (Node-102) > Interfaces > VPC Interfaces

Each of these screens will give a summary of the physical state of each port/PC/VPC.

Summary Picture

The illustration below shows a summary of the Access Policies that I have configured so far.

AccessPolicies Summary

But your configuration is still not complete.  Remember when you created EPGs (in Part #2), there was a link to a list of Domains.  If you are ready to start Connecting servers to the fabric switches, then continue to the next tutorial where the Domain will be linked to the EPGs, and ports assigned to the EPGs.

Next…

RedNectar

Note:RedPoint If you would like the author or one of my colleagues to assist with the setup of your ACI installation, contact acimentor@housley.com.au and refer to this article. Housley works mainly around APJC, but are not restricted to this area.

Footnotes

 

Attachable Access Entity Profile

Note: The Attachable Access Entity Profile object is referred to variously as an Attachable Entity Profile; an Attached Entity Profile; an Access Entity Profile and possibly other terms, but most often is referred to simply as an AEP. Although I struggle with reconciling AEP to mean Attachable Access Entity Profile, I will bow to the common practice and stick with the convention others have put in place before me. Don’t be confused. I was.

Domains

Note: Domains  are sometimes referred to as Domain Profiles (in EPG configuration and in the help documentation). Don’t confuse them with Bridge Domains, they are entirely different concepts – think of Domains as VLAN Domains and you will be closer to the mark. The use of Domains will come to prominence in parts #5 and #6 of this tutorial.

Interface Profile and Access Port Selector

Note: The Interface Profile object is referred to as an associated Interface Selector Profile or Interface Select Profile on the Switch Profile page. On the other hand, the Access Port  Selector object is also referred to in various places as Interface Selector or the Host Port Selector. Don’t be confused. I was.

Switch Profile

Note: The Switch Profile object is referred to as a Node Profile on the Switch Profile help page.

 

Posted in Access Policies, ACI, ACI configuration, Cisco, configuration tutorial, Data Center, Data Centre, Nexus, Nexus 9000, SDN, Software Defined Networking | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , | 1 Comment

New CLI coming for Cisco ACI/APIC

This video on youtube shows off the NX OS style CLI coming for Cisco ACI with version 1.2 code for the APIC and version 11.2 ACI switch code.

The slides shown at the beginning of the video mention a release date of December 2015. That’s not far away!

I watched the video (20mins) and noticed a couple of interesting things in the CLI

  1. Application Profiles are configured using the application configuration command, although the speaker still referred to it as an “Application Network Profile”
  2. Private Networks are configured using the vrf context  configuration command. I love consistency!
  3. From a snippet of code he showed, it looks like filters are referenced within subjects as access-groups. Clearly trying to appeal to the old CLI jockeys rather than making a clean correlation to the already flawed and inconsistent GUI, or even a correlation to the the API classes and distinguished names.

RedNectar

Note:RedPoint If you would like the author or one of my colleagues to assist with the setup of your ACI installation or ACI training needs, contact acimentor@housley.com.au and refer to this article. Housley works mainly around APJC, but are not restricted to this area.

 

 

Posted in ACI, ACI configuration, Cisco, Data Center, Data Centre, NX OS | Leave a comment

Cisco ACI Tutorial 3 – Sing a new song for sorting server groups and policy

Sing a new song for sorting server groups and policy

Cisco ACI Tutorial – Part 3

If you have been responsible for defining groups of servers and devices to which policy can be applied, you have probably used structures called Access Control Lists, or ACLs.

ACLs are typically based on IP addresses and IP address ranges, with added refinements for UDP/TCP port numbers.

ACI has no concept of ACLs.  In this part of the tutorial I’ll explain how ACI uses the concept of Application Profiles (APs) and End Point Groups (EPGs) to identify devices and policy groupings.  Later I will add Contracts and Filters to complete the story, but for now I want to relate these new concepts to the picture I presented in the previous tutorial.

TIP:RedPoint2 If you have not read Cisco ACI Tutorial – Part #1, now would be a good time. It contains a diagram of the topology and some important notes about conventions used. Similarly, you will get more familiar with the context of this tutorial if you also take a look at Cisco ACI Tutorial – Part #2 

The following diagram adds the relationship between  Application ProfilesEnd Point Groups, and Bridge Domains to the diagram from Part #2.

EPGs are children of the AP and the AP is a child of the Tenant. The Tenant owns the AP and the AP owns the EPGs.  But each EPG must be linked to a Bridge Domain. And here’s the catch.  The Bridged Domain that the EPG is is linked to doesn’t have to belong to the same Tenant as the EPG itself.

In the diagram above, you can see that the AP called  AP1 has three EPGs. The AP and all its EPGs then belong to the Tenant called RedNectar, but the EPG called DB-EPG is linked to a BD in the common tenant.  The common tenant is a special tenant where shared configuration can be defined. I have used this relationship here just to illustrate that there is a little chink in the hierarchical model, in that entities that are linked rather than related (parent-child) can be linked outside the parent-child relationship.

As far as IP addressing goes, devices in the PublicServers-EPG could be in either the 203.0.113.0/24 subnet, or the 10.1.4.0/24 subnet. Traffic between devices on the 203.0.113.0/24 subnet will be bridged in the normal way based on MAC addresses, but traffic between 203.0.113.0/24 devices and 10.1.4.0/24 devices will be routed freely (remember there is NO restriction between devices in the same EPG) without policy applied.

 Key Point: End Point Groups are basic unit of policy enforcement.  End Points (=something with an IP address) within the same EPG can communicate freely without restrictions – ping each other, share files… Later I will show how a Contract is needed to permit devices in different EPGs to communicate.

However the more interesting arrangement above concerns the two EPGs linked to the Clients-BD. Note that there are two EPGs but only one subnet.  This means that some End Points that are on the same subnet will NOT be able to freely communicate with each other – they will need a Contract to be able to communicate.


 

Time out – re read that last paragraph

Yes – it says that you can easily configure different communication  policies between devices on the same subnet. Something that would involve tedious editing of ACLs with 32 but masks in a traditional environment.

The result is something akin to what you might have used Cisco’s Private VLANs for in the past – each End Point uses the same default gateway IP address, but only End Points in the same EPG can communicate freely.


For our configuration model I won’t make it as sophisticated as the diagram.  I will simply start by configuring an AP with two EPGs.

Note the naming used for the EPGs. It is no accident that they closely relate to the BDs that they are linked to.  It is possible to have multiple EPGs linked to a single BD (as in the example above), but unless you are specifically attempting fine grained separation, I wouldn’t recommend having more than one EPG linked to a BD.

The EPG is the atomic unit for applying policy. All devices in the same EPG therefore have unrestricted access to each other, but for traffic to travel between EPGs there will need to be a Contract.

For our example, I will begin by creating one Application Profile (AP), and two End Point Groups (EPGs) within the AP.

Configuration Steps

Step 1: Add a new Application Profile

The application profile is a set of requirements that an application instance has on the virtualizable fabric. The policy regulates connectivity and visibility among endpoints within the scope of the policy.

Application profiles define the policies, services and relationships between End Point Groups (EPGs). Each application profile contains one or more EPG that can communicate with the other EPGs in the same application profile and with EPGs in other application profiles according to the contract rules. [Adapted from ACI Help]

The Application Profile is a container object for End Point Groups. An Application Profile can also be used to limit the scope of a Contract. [RedNectar’s Definition]

TENANTS > RedNectar > Tenant RedNectar > Application Profiles

  • (+) Create Application Profile
    • Name: AP1

Step 2: Add two new Application End Point Groups

An End Point Group (EPG) is a collection of physical and / or virtual end points that require common services and policies. End point examples include servers, virtual machines, storage, or clients on the Internet. An End Point Group example is a set of servers providing a common application, function or service. [From ACI Help]

An End Point Group (EPG) is a group of End Points that require common services and policies, such a server farm.  End Points are defined by IP and/or MAC addresses.  End Point Groups must be associated with a single Bridge Domain to enable Layer 2/3 connectivity, and End Point Groups must be associated with at least one (VLAN) Domain, either a Physical Domain or a VMM Domain  so that the ACI Fabric can detect incoming traffic from an End Point.

The ACI Fabric uses 802.1Q VLAN tags (or VXLAN VNIDs) to identify which EPG incoming traffic is associated with.  [RedNectar’s Definition]

TENANTS > RedNectar > Tenant RedNectar > Application Profiles > AP1 > Application EPGs

  • (+) Create Application EPG
    • Name: InternalClients-EPG
    • Bridge Domain: RedNectar/Clients-BD
TIP:RedPoint2 Take note of the fact that there is a section in the EPG configuration where you can associate the EPG to a list of Domains(*).  Domains play a very important role in the EPG as you will see in part #5 of this Tutorial.
  • (+) Create Application EPG
    • Name: PublicServers-EPG
    • Bridge Domain: RedNectar/PublicServers-BD

So now your configuration can be encapsulated by the following illustration:

Tn with DB and SN and AP and EPG


Filters, Subjects and Contracts

Before you can apply policy to traffic flowing between EPGs, you will need to create the Contracts that define the policy to be applied. And since Contracts have Subjects and Subjects are linked to Filters, you will need to define those too.

The following diagram shows the relationship between EPGsContractsSubjects and Filters.

Filters are easy to understand.  A filter simply allows traffic on a particular UDP/TCP port (or traffic on a particular protocol like ICMP).  So in my example I have one filter called HTTP-Fltr with two entries – one I have named TCP80 which filters on destination TCP port 80, the other named UDP80, which filters on destination UDP port 80. Another filter is called HTTPS-Fltr with only one entry – TCP443 and the last one is called App-Fltr and has one entry called TCP88.  No prizes for guessing how that naming scheme works!

However, I would like to point out that there is a subtle difference between the first two filters and the last. Filters are defined within a Tenant space, so I have defined the first two in the common tenant, which means they will be available to all Tenants. The third one is defined within the RedNectar tenant.

Now Contracts are not directly linked to Filters.  Instead, Cisco has added an extra layer of flexibility (and complexity) by giving Contracts a child object called a Subject.  And it is the Subjects that are linked to Filters – in my example the Contract ServerToClient.PN.Ct has two Subjects and one of those Subjects (Web-Subj) is linked to two Filters from the common tenant.

Finally, the crux of the matter is the way policy is applied through the Contract.  For a Contract to pass traffic, the EPG either has to Consume a Contract or Provide a Contract. and every Contract must have at least one consumer and one provider to be of any use.  Later I will discuss how Contracts can also be used to redirect traffic to services such as Firewalls and Load Balancers. but for now, notice how the name of the contract indicates the direction of consumption.

There are two things to consider when naming contracts; direction and scope.  With the name ServerToClient-PN.Ct I have indicated that the direction of consumption is from Server To Client, and in the PN in the name indicates that the scope of the contract is Private Network.  When you are looking at a list of contracts and trying to apply one to an EPG, knowing the scope of each contract is important.

For my example, I will continue by examining the pre-defined filters in the common Tenant, then creating a new Filter in the common Tenant for Web traffic which will filter on TCP ports 80 and 443.

I will then create a new Contract in the common Tenant – for ping traffic. In the process, Subjects will be added to the Contract and linked to the Filters that were examined/created earlier.

For diversity, I will then repeat the process of creating a Filter and Contract in the RedNectar Tenant.

Configuration Steps

Step 3: Examine pre-defined Filters

A filter policy is a group of resolvable filter entries. Each filter entry is a combination of network traffic classification properties. [Adapted from ACI Help]

TENANTS > common > Tenant common > Security Policies > Filters

  • Note that there are a set of pre-defined filters defined in the common Tenant.  These filter can be linked to any Subject in any Contract in any other Tenant.

PreDefinedFilters

Step 4: Create a common Filter for web traffic

TENANTS > common > Tenant common > Security Policies > Filters

  • (+) Create Filter
    • Name: Web-Fltr
    • Add (+) Entries:
      • Name: TCP80
        EtherType: IP
        IP Protocol: tcp
        Source Port Range From: 1024 To: 65535 [See Note below]
        Destination Port Range From: http To: http
    • Add (+) Entries:
      • Name: TCP443
        EtherType: IP
        IP Protocol: tcp
        Source Port Range From: 1024 To: 65535
        Destination Port Range From: 443 To: 443
Note:RedPoint For the second entry I could have selected the Destination Port Range From: https To: https, but I wanted to illustrate that it is possible to type port numbers in these fields – but it is tricky. Type quickly and hit Tab as soon as you have finished typing.

Step 5: Create a common Contract for web traffic and a common Contract for ping traffic.

A service contract/deal can exist between two or more participating peer entities, such as two applications running and talking to each other behind different endpoint groups, or between providers and consumers, such as a DNS contract between a provider entity and a consumer entity.

A subject is a sub-application running behind an endpoint group (for example, an Exchange server). A subject is parented by the contract, which can encapsulate multiple subjects. An endpoint group associated to a contract is providing one or more subjects or is communicating with the subject as a peer entity. An endpoint group always associates with a subject and defines rules under the association for consuming/providing/peer-to-peer communications to that subject.

One or more subjects within a contract use filters to specify the type of traffic that can be communicated and how it occurs.  Each filter entry is a combination of network traffic classification properties. [Adapted from ACI Help]

TENANTS > common> Tenant common > Security Policies > Contracts

  • (+) Create Contract
    • Name: Ping-TnCt
    • Scope: Tenant
    • Add (+) Subjects:
      • Name: Ping-Subj
      • Add (+) FILTERS
        • Name: common/icmp

Step 6: Create a tenant Filter for custom application traffic

Note:RedPoint Make sure you do NOT do the following configuration in the common Tenant – I am switching back to Tenant RedNectar’s configuration.

TENANTS > RedNectar > Tenant RedNectar > Security Policies > Filters

  • (+) Create Filter
    • Name: App-Fltr
    • Add (+) Entries:
      • Name: TCP88
        EtherType: IP
        IP Protocol: tcp
        Source Port Range From: 1024 To: 65535
        Destination Port Range From: 88 To: 88

Step 7: Create a tenant Contract for web traffic and custom application traffic

TENANTS > RedNectar > Tenant RedNectar > Security Policies > Contracts

  • (+) Create Contract
    • Name: PubSrvsToIntClnts-PN.Ct 
    • Scope: Private Network [Default]
    • Add (+) Subjects:
      • Name: Web-Subj
      • Add (+) FILTERS
        • Name: common/Web-Fltr
    • Add (+) Subjects:
      • Name: App-Subj
      • Add (+) FILTERS
        • Name: RedNectar/App-Fltr
Note:RedPoint The above result could have also been achieved using a single Subject with two filters added. Separating each filter into it’s own subject allows for future flexibility is you say wanted to add QoS policies to one of the subjects.

Indeed, the above could also have been achieved by creating two contracts, one for Web and one for App traffic.  It all depends on the level of granularity required.

 Consuming and Providing Contracts

Now that you have all of the building blocks – EPGs, Filters and Contracts defined, you can start defining how policy is to be applied to allow traffic between our EPGs.  To do this, you will return to the EPGs you created earlier, and link the contracts in a provider-consumer relationship.  The PubSrvsToIntClnts-PN.Ct contract will be be linked as a provided contract to the PublicServers-EPG and as a consumed contract to the InternalClients EPG.  This will allow devices in the InternalClients-EPG to ask for TCP services on ports 80, 443 and 88 from devices in the PublicServer-EPG.  Return traffic will be allowed of course, but devices in the PublicServer-EPG will not be able to consume Web traffic from the InternalClients-EPG even if those devices have services listening on TCP ports 80, 443 or 88.

However, the Ping-Tn.Ct contract is a little different.  In this case, you will want devices in the PublicServer-EPG to be able ping the InternalClients-EPG devices and  the InternalClients-EPG devices to be able to ping the PublicServer-EPG devices.  So to make this happen, each EPG will have to be both a provider to and a consumer of the contract.

[Edit: 2015-11-27. Since ACI v1.1, ICMP filters only need to be applied in one direction. Once an EPG has been defined either a  provider to or a consumer of a contract with an ICMP filter it is able to both send and receive ICMP traffic to a corresponding consumer or provider. However, you still need both a consumer andprovider.  Two providers can’t ping each other, nor can two consumers.]

Configuration Steps

Step 8: Link EPGs with contacts in their appropriate consumer-provider relationship.

TENANTS > RedNectar > Tenant RedNectar > Application Profiles > AP1 > Application EPGs > EPG InternalClients-EPG > Contracts

  •  (+) Add Consumed Contract
    • Contract: RedNectar/PubSrvsToIntClnts-PN.Ct
  •  (+) Add Consumed Contract
    • Contract: common/Ping-TnCt
  •  (+) Add Provided Contract [Edit: 2015-11-27. This is now unnecessary]
    • Contract: common/Ping-TnCt

TENANTS > RedNectar > Tenant RedNectar > Application Profiles > AP1 > Application EPGs > EPG PublicServers-EPG > Contracts

  •  (+) Add Provided Contract
    • Contract: RedNectar/PubSrvsToIntClnts-PN.Ct
  •  (+) Add Consumed Contract [Edit: 2015-11-27. This is now unnecessary]
    • Contract: common/Ping-TnCt
  •  (+) Add Provided Contract
    • Contract: common/Ping-TnCt

Step 9: Verify your configuration

TENANTS > RedNectar > Tenant RedNectar > Application Profiles > AP1 > Application EPGs

  • You should see icons representing your EPGs and your Contracts.  You can move these icons around, and without too much trouble, you should be able to get your screen looking something like this:

ApplicationEPGs view

Note that the arrows show the direction from Provider  to Consumer.  In the case of the Ping-TnContr, the arrows are pointing away from the contract, indicating that both EPGs are consuming the contract, but unfortunately there is no indication that both EPGs are also providing the contract as well.

TIP:RedPoint2 If your screen seems to have arrows going different ways to my illustration, select some other top menu item (like SYSTEM) then navigate back to TENANTS > RedNectar > Tenant RedNectar > Application Profiles > AP1 > Application EPGs.

TENANTS > RedNectar > Tenant RedNectar > Application Profiles > AP1 > Application EPGs > EPG PublicServers-EPG | [OPERATIONAL] | [CONTRACTS]
TENANTS > RedNectar > Tenant RedNectar > Application Profiles > AP1 > Application EPGs > EPG InternalClients-EPG | [OPERATIONAL] | [CONTRACTS]

checkContracts

Note the filters that are produced when you create contracts.

Discussion

You now have policies in place to control traffic between devices in the two EPGs you have created. But at the moment there is absolutely no use for your AP, your EPGs or any other policy related configuration that you have completed, because you have yet to tell the APIC which connected devices and servers belong to which EPG.

And before you can do that, you will have to tell the APIC how to identify traffic and on which interfaces to expect traffic for these EPGs.

And to do that you will need to learn all about Access Policies – the new “interface range” command.

Next…

RedNectar

Note:RedPoint If you would like the author or one of my colleagues to assist with the setup of your ACI installation, contact acimentor@housley.com.au and refer to this article. Housley works mainly around APJC, but are not restricted to this area.

Footnotes

Domains

Note: Domains  are sometimes referred to as Domain Profiles (in EPG configuration and in the help documentation).   Don’t confuse them with Bridge Domains, they are entirely different concepts – think of Domains as VLAN Domains and you will be closer to the mark. The use of Domains will come to prominence in parts #5 and #6 of this tutorial.
Posted in ACI, ACI configuration, Cisco, configuration tutorial, Data Center, Data Centre, Nexus, Nexus 9000, SDN, Software Defined Networking | Tagged , , , , , , , , , , , , | 3 Comments

Cisco ACI Tutorial 2 – Goodbye to VLANs. Well… not quite

Goodbye to VLANs. Well… not quite

Cisco ACI Tutorial – Part 2

ACI does not use VLANs or even subnets to isolate policy groups. I’m sorry if this is a shock for you, but before I discuss how policy groups are bound by an object known as an End Point Group I’ll use this tutorial to take you through the replacement concepts for VLANS and IP addressing.  The containers used by ACI to provide IP addressing isolation are the Tenant, and the Private Network.  Bridge Domains are used to provide multicast and broadcast isolation (similar to VLANs) and you’ll be pleased to know that we still use Subnets for IP addressing.

In this part of the tutorial I am going to take you through the configuration of the primary containers in ACI as you setup a Tenant, a Private Network, two Bridge Domains and two Subnets.

TIP:RedPoint2 If you have not read Cisco ACI Tutorial – Part 1, now would be a good time. It contains a diagram of the topology and some important notes about conventions used.

First a little theory

ACI uses the concepts of Tenants, Private NetworksBridge Domains, and Subnets in a hierarchy to contain routed traffic.  Note that the concept of VLANs is entirely absent. Later I will add the most important policy containment concept, End Point Groups (EPGs) to the mix.

This diagram shows the hierarchy of Tenants, Private NetworksBridge Domains, and Subnets.

Note that some relationships are linked such as Bridge Domains are linked to Private Networks, while others are parent-child relationships, such as Subnets, which are children of Bridge Domains which in turn are children of Tenants.  Note that these relationships are also one-to many – a Tenant may have multiple Private Networks, Private Networks can be linked to multiple Bridge Domains, and Bridge Domains many multiple child Subnets.

Note:RedPoint Typically you will have only one Subnet per Bridge Domain, so in that sense a Bridge Domain “resembles” a VLAN, and a Subnet “resembles” a VLAN interface.

The naming and numbering of the items also reveals some of the isolation as well. Note that the names BD1 and BD2 have been used in each Tenant, and that Tenant Large has three Private Networks (VRFs), two of which have an instance of the 10.1.1.1/24 subnet associated with it.

Note: RedPoint The subnet numbering is on purpose. 10.1.1.1/24 actually defines the Distributed Gateway address for the subnet 10.1.1.0/24, but it is convenient to use the numbering to indicate both in this diagram.

Configuration Steps

Step 1: Add a new Tenant

A tenant is a logical container for application policies that enable an administrator to exercise domain-based access control. A tenant represents a unit of isolation from a policy perspective, but it does not represent a private network. Tenants can represent a customer in a service provider setting, an organization or domain in an enterprise setting, or just a convenient grouping of policies. [From ACI Help]

Tenant is a logical container for isolating routing and switching functions.  A Tenant will require at least one Private Network to provide routing functionality, and at least one Bridge Domain to provide layer 2 isolation. [RedNectar’s Definition]

TENANTS > ADD TENANT

  • Name: RedNectar
Note:RedPoint Clicking NEXT, and FINISHED is assumed since I have come to the end of the item.  I almost always avoid using the Wizard steps because Wizards are magical creatures that are particularly good at hiding information. By avoiding the Wizards, you will get to know where the individual configuration items are placed.

Step 2: Give the new Tenant a Private Network

A private network is a layer 3 context (a VRF) that provides IP address space isolation for tenants. Each tenant can have one or more private networks, or share one default private network with other tenants when there is no overlapping IP addressing being used in the ACI fabric. [From ACI Help. I added the emphasis to illustrate how the terms private network,  [layer 3] context and VRF are used interchangeably  in ACI terminology]

TENANTS > RedNectar > Tenant RedNectar > Networking > Private Networks

  • (+) Create Private Network
TIP:RedPoint2 The Create Private Network option is reached by right-clicking on the Private Networks element in the Navigation Pane or by clicking the ACTIONS menu at the top right hand side of the Work Pane
    • Name: VRF1
    • Create A Bridge Domain [ ] Unchecked

Step 3: Give the new Tenant two Bridge Domains

A bridge domain represents a L2 forwarding construct within the fabric. One or more EPG can be associated with one bridge domain or subnet. A bridge domain can have one or more subnets associated with it. One or more bridge domains together form a tenant network. [From ACI Help]

Bridge Domain is a Layer 2 (broadcast) domain that must be associated with a Tenant‘s Private Network. A Bridge Domain may contain one or more Subnets that can provide routing services for associated End Point Groups. [RedNectar’s Definition]

TENANTS > RedNectar > Tenant RedNectar > Networking > Bridge Domains

  • (+) Create Bridge Domain
    • Name: Clients-BD
    • Network: RedNectar/VRF1
  • (+) Create Bridge Domain
    • Name: PublicServers-BD
    • Network: RedNectar/VRF1

Step 4: Give the new Bridge Domains a Subnet each

A subnet, which represents routes that will be exported. While a context defines a unique IP address space, that address space can consist of multiple subnets. These subnets are defined per bridge domain. A bridge domain can contain multiple subnets, but a subnet is contained within a single bridge domain. [From ACI Help]

Subnet is defined by an IP address/mask and provides a routing gateway service for  End Point Groups that are associated to the Subnet‘s parent Bridge Domain.  A Bridge Domain can contain multiple Subnets, but a Subnet is contained within a single Bridge Domain. [RedNectar’s Definition]

TENANTS > RedNectar > Tenant RedNectar > Networking > Bridge Domains > Clients-BD > Subnets

  • (+) Create Subnet
    • Gateway IP: 192.168.10.1/24

TENANTS > RedNectar > Tenant RedNectar > Networking > Bridge Domains > PublicServers-BD > Subnets

  • (+) Create Subnet
    • Gateway IP: 203.0.113.1/24
    • Scope:
      [   ] Private Subnet [Unchecked]
      [x] Public Subnet

Step 5: Verify your configuration

Navigate your way around the Navigation Pane and expand the relevant sections to verify that you have. It should look something like this:

Verify Tenant Config

And diagrammatically, it could be represented like this:

Discussion

Note that you have created a structure to contain your subnets that does not involve VLANs.  If you have come from a networking background, this will be a foreign concept for you.  Subnets are however contained by another scope called a Bridge Domain.  In many ways a Bridge Domain is similar to a VLAN, but is not a container for policy enforcement.  For Policy enforcement, you will need to sing a new song for sorting server groups and policy, namely Application Profiles and End Point Groups.

Next…

RedNectar

Note:RedPoint If you would like the author or one of my colleagues to assist with the setup of your ACI installation, contact acimentor@housley.com.au and refer to this article. Housley works mainly around APJC, but are not restricted to this area.

Footnotes

Quotations: The quotations supplied to describe the various elements are taken directly from the ACI Help.  I did not write this content and want to be clear about the source.  Which also should give you a heads up: It is a good idea to click the little i symbol in the top right-hand corner of many panes and dialogues.
Naming Conventions: Note also the naming convention used.  You will find that I almost always give an object a descriptive name followed by a dash followed by an identifier – such as Clients-BD – the name only has one dash. If a separator is needed in either part of a name, a dot . or a colon : or an underscore _ is used.  Sometimes a generic name is appropriate, especially if the purpose is not clear (you CAN’T change names later).  If I had have planned multiple VRFs (Private Networks) I may have named them something like DMZ-VRF and Internal-VRF, but in this case I chose a generic VRF1 as a name.
Posted in ACI, ACI configuration, Cisco, configuration tutorial, Data Center, Data Centre, Nexus, Nexus 9000, SDN, Software Defined Networking | Tagged , , , , , , , , , , , , | 2 Comments

Cisco ACI Tutorial – A Configuration Guide

Cisco ACI Tutorial – A Configuration Guide

Cisco ACI Tutorial – Part 1

Note:RedPoint This is the first of a series of at least eight blog posts that I plan to publish over the coming weeks. Make sure you follow my blog so you don’t miss out on the continuing story.

If you are new to Cisco Application Centric Infrastructure (ACI) then you may well be daunted at this new method of configuring switches.  In this multi-part tutorial, I hope to take some of the configuration pain out of your headache.

I will take you through some of the underlying concepts, but do not attempt to explain the theory of ACI, such as VXLAN overlays and the like.

Objective

This is a configuration tutorial. It takes you through the steps needed to configure a sample ACI fabric.  The journey will give you important foundations in naming conventions that will help you understand your configuration in the future, and establish some best practices and conventions that will guide you long into the future.

At the completion of the tutorial, you will have a fully configured ACI fabric ready for testing.  You are responsible for making sure that all systems are production ready before deployment in a live network.

Prerequisite

To get the most out of this tutorial, you will need some basic understanding of Cisco ACI and the role of the Application Policy Infrastructure Controller (APIC).  Ideally, you will have access to an APIC and an ACI fabric, or the ACI Simulator.  Cisco partners/customers with sufficient rights to their CCO login, will be able to get access to an ACI simulator for a few hours at a time by logging on to Cisco dcloud.  See this post for a dcloud tutorial.

If you are not comfortable with terms like Leaf, Spine, Clos topology, End Point Groups and Application Profiles, then you will do well to do some research before you begin.  I’d recommend the following:

Let’s Begin

Assumption

To get the ball rolling, I will assume that:

  • Your ACI devices are racked and stacked, and cabled according to the Topology below
  • You have completed the assigned Out of Band (OOB) Management IP addresses to your APIC(s) and assigned addresses to the TEP pool as described in the “Setting up the APIC” section of the APIC Getting Started Guide.
  • You have a management station logged into the APIC GUI ready to begin configuring your ACI fabric.
TIP:RedPoint2 If you have real lab equipment (rather than using a simulator), then you will also be able to physically test your configuration if you have:

  • at least two devices that can act as Bare Metal Servers (ie servers that do NOT run a hypervisor), ideally with two NICs (and a 3rd NIC for RDP or VNC access would also be useful in the lab environment)
  • at least one ESXi server, and
    • VMware vCenter v5.5 installed with a NIC on the OOB Management Network.  For my test network I had an ESXi with three NICs and ran the vCenter as a VM on the ESXi server

Structure

This tutorial has seven parts, including this one.

  1. Let’s Begin
    • That’s this first tutorial where I cover the topology I will use, the Lab setup and conventions.
  2. Goodbye to VLANs. Well… not quite
    • Tenants, Private Networks, Bridge Domains and Subnets
  3. Sing a new song for sorting server groups and policy
    • Application Profiles, End Point Groups and Contracts
  4. All about Access Policies – the new “interface range” command
    • Access Port Profiles and Interface Profiles
    • And all that goes with it (Interface Policies, Policy Groups…)
  5. Connecting servers to the fabric switches
    • Directly connected bare-metal servers and switches without applying policy
    • VMware Virtual Machines
  6. Connecting the outside world
    • External bridged devices with applied policy
    • External routed devices
  7. Serving up services
    • Firewalls, load balancers, SSL offload…

The Topology

You probably know that an ACI Topology consists of Leaf and Spine switches.  For this tutorial, I will use two of each, although in truth the spine switches will not feature much at all in the configuration.  What will be more important is the equipment that is connected to the leaf switches.

ACI Topology

Note that there are two leaves.  To make life simple, I am going to use the following ports on each leaf switch for the following purposes:

 

Port range Use
1/1-2 External L3 Routers
1/3-4 External L2 Switches
1/5-10
1/5-7 10Gb/s
1/8-10 1Gb/s
 Single Attached Bare Metal Hosts
1/11-14 VPCs to ESXi  Hosts
1/15-20
1/15-17 10Gb/s
1/18-20 1Gb/s
Single Attached ESXi  Hosts
1/21-24 VPCs to UCS B Series
1/25-28 VPCs to Bare Metal Hosts
1/29-32 Port Channels to Bare Metal Hosts
1/33-40 FEX connections
1/41 SPAN Port
1/42-44 APIC attachments

I will refer back to this table in  Tutorial #4 All about Access Policies – the  new “interface range” command.

The setup

AS I mentioned above, I’ll assume you have completed the assigned Out of Band Management IP addresses to your APIC(s) and assigned addresses to the TEP pool as described in the “Setting up the APIC” section of the APIC Getting Started Guide and can now log into your APIC’s GUI.

Already your fabric has begun the discovery process using LLDP, but before you can continue, you will have to register each leaf and spine switch in the APIC.  At the completion of this process, the APIC controllers will discover each other and form an APIC Cluster to become a single management unit.

Conventions

I will refer to menu items in the APIC in the following manner:

FABRIC > INVENTORY > Pod 1> Leaf 1

means that you should select the main menu item FABRIC, the sub-menu item INVENTORY then expand the POD 1 branch of the Navigation pane, and select the item Leaf 1.  Leaf 1 is in bold print because Leaf 1 is a user assigned name – in your case you may have decided to call it L-101 or something different – so you will have to be on the lookout when menu paths have bolded items, if you don’t follow my names exactly, you may have differences.

Menu Bar, Submenu Bar, Navigation Pane and Work Pane

Sometimes, you will need to click in the Work Pane to perform some tasks, and in this case you may find that the work pane has tabs across the top, and possibly another row of tabs under that or even more tabs under that. When you need to click on a tab, the tab name will appear in square brackets after a vertical bar after the item you need to select in the navigation pane, like FABRIC > INVENTORY > Pod 1 | [OPERATIONAL] | [SWITCHES] | [LEAVES], which says, “After you have selected FABRIC > INVENTORY > Pod 1 in the navigation pane, select the OPERATIONAL tab in the Work Pane, then select the tab under that called SWITCHES, and the tab under that called LEAVES.

NavigationPath

Configuration Steps

Step 1: Identify the fabric nodes

Note:RedPoint If you are using Cisco dcloud, you may find that this step has already been completed.

FABRIC > INVENTORY > Fabric Membership

You should see one item in the list with a NODE ID of 0 and no Node Name.  This device is the Leaf switch that your APIC is connected to.

Double-click on this entry and change the following:

  • NODE ID: 101
  • NODE NAME: Leaf1
Note:RedPoint Any text you have to enter will be shown in bold italics. Items that you have to select from a list or click on will be shown in bold only.

You will have to click on UPDATE to complete the change, but from now on you will have to assume that when I reach the end of a configuration item, you may need to click items like NEXT, FINISH, UPDATE and SUBMIT to complete the configuration.

After some time, the spine switches will be discovered via LLDP.  Pretty soon your screen should look something like this:

FabricMembershipProgress

As each Spine switch is discovered, double-click on the entry and change the following:

  • NODE ID: 201
  • NODE NAME: Spine1
  • NODE ID: 202
  • NODE NAME: Spine2

And finally the second leaf will be discovered. Update to

  • NODE ID: 102
  • NODE NAME: Leaf2

Step 2: Verify your config

Verify your configuration by selecting FABRIC > INVENTORY > Pod 1 | [OPERATIONAL] |[SWITCHES] | [LEAVES] and FABRIC > INVENTORY > Pod 1 | [OPERATIONAL] |[SWITCHES] | [SPINES].

You should see that each switch has been given an INFRASTRUCTURE IP that has come from the range of TEP addresses you gave the APIC in the initial setup, and that the STATUS shows in-serviceVerify Leaves

Assuming all is well, it is time to get ready to say Goodbye to VLANs. Well… not quite

Next…

RedNectar

Note:RedPoint If you would like the author or one of my colleagues to assist with the setup of your ACI installation, contact acimentor@housley.com.au and refer to this article. Housley works mainly around APJC, but are not restricted to this area.
Posted in ACI, ACI configuration, Cisco, configuration tutorial, Data Center, Data Centre, Nexus, Nexus 9000, SDN, Software Defined Networking | Tagged , , , , , , , , , , , , | 3 Comments