Archive | Cisco RSS feed for this section

FYI: Cisco UCS doesn’t work with VVOLs oob

FYI, ran into an issue tonight working with VVOLs.

I had just upgraded my PureStorage array to Purity 5 GA release and wanted to jump into VVOLs on my lab.

Customers will not be able to use vvols unless they manually install FNIC driver or higher.

VVOLs are not supported on any VICs with the drivers installed from the current Cisco-branded v6.5 and 6.5U1 ESXi ISOs.

This is Cisco bug CSCux64473 Specific cause is because included driver did not support “Secondary LUN IDs”.

If a customer tries, their Protocol Endpoint will just never show up.

Symptom: UCS server with VIC adapter does not support VMware Virtual Volume feature
Conditions: When attempting to access Virtual Volume from UCS servers installed with VIC card.
Workaround: Virtual volume is not supported on any the VIC cards / FNIC drv ( Their but report isn’t totally true, I tested driver and it worked great!)

  1. Latest driver release does work:


Now, after updating the driver and rebooting my servers, VVOLs is working awesome!

PureStorage does VVOLS right!


Leave a comment Continue Reading →

#STBY Cisco Clock Issue – 18month life failure

Cisco announced that a clock module in certain products may have a failure after 18months. This one is big, and this one is bad!
Partial list of systems with issues (ordered by my own priority of which I think will hit my customers the worst)

ASA 5506, ASA 5506W, ASA 5506H, ASA 5508, and ASA 5516 <---- ahh crap! Meraki MS350, MB84 Nexus 9000 Series N9K-C9504-FM-E/N9K-C9508-FM-E/N9K-X9732C-EX NCS1K-CNTLR Cisco ISA3000 SR4331, ISR4321, ISR4351 and UCS-E120

Click to continue reading “#STBY Cisco Clock Issue – 18month life failure”

Leave a comment Continue Reading →


Please click here to goto the main NS0-504 Study Notes blog post to see all sections and additional information


3.1 Prepare site for installation.

3.1.1 Be able to review implementation flowchart with customer and assign task areas.


3.1.2 Verify site infrastructure including: dual power, floor space, floor loading plan, HVAC.

Sample cDot Site Survey Questions (not including networking) (from an cDot old Survey to Quote document)


Questions to help get the equipment racked and powered properly.

  • Does the customer plan on using NetApp Cabinets?
    • If Yes, what PDUs does the customer want in the cabinets?
      • What country will these racks be installed in?
      • Will redundant power be able to be configured?
    • If No, what is the height in RackU of the customer’s cabinets?
      • Will customer provide redundant power for the equipment?
  • Does the customer want NetApp to supply PDU-to-Head/Shelf power cords?
    • If so, what length?
    • If not, country-specific power cords will be included.
  • Does the customer want NetApp Universal Rail kits?
    • If so, 2 post or 4 post?

Questions to help gauge cable lengths to connect cluster elements together.

  • How does the customer plan on running cables between racks?
    • Down, under the floor and back up?
    • Up, through ladder racks/trays and back down?
    • Through cabinet sides (not recommended)?
    • What cable length(s) will be required?
  • How will cabinets be set up on the datacentre floor?
    • All adjacent with no gaps or other equipment in between?
    • Separated by an aisle or other equipment?
    • If not all adjacent, please build diagram of cabinet layout to facilitate cable length calculation with PS


Questions to help gauge SFPs and cable lengths to connect cluster to host/client side networks.

Clusters with NAS, iSCSI & FCoE protocols:

  • What kind and quantity of Ethernet ports does the customer require?
    • o   10GigE with optical SFP+
    • o   10Gig CNA with optical SFP+
    • o   10GigE bare cage
    • o   10Gig CNA bare cage
    • o   1GigE with optical SFP
    • o   1GigE with RJ45 connector
  • Does the customer want NetApp to supply cables for any of the above?
    • If so, what length(s)?
  • For bare cage 10GigE & 10GigE CNA, does the customer want copper 10Gig cables?
    • If so, what length(s)?

Clusters with FC protocol:

  • What kind and quantity of FC ports does the customer require?
    • o   8Gbps with optical SFP
    • o   4Gbps with optical SFP
    • o   8Gbps bare cage
    • o   4Gbps bare cage
  • Does the customer want NetApp to supply cables for any of the above?
    • If so, what length(s)?


  • Read the Netapp Site Requirement guide
  • Review the BTU and Ton conversion of the heat output of the controllers and disk shelves.
  • Know the NEMA vs IEC power cables and voltage
    • C13 – C14
    • NEMA L5-15/L5-20
    • C19-C20

3.1.3 Validate equipment move path to installation location.

  • This sounds stupid, but I have seen racks fall over, or bust plastic wheels.
  • Walk it!

3.1.4 Validate logistics plan for staging and installation of equipment.

  •  Will this be built in a lab room? Built in production rack? Etc.

3.1.5 Verify Ethernet cabling plan and availability of cable supports.

Valid 1gb Administration Switches

  • Cisco 2960
  • Netapp 1601 (may not have existed at time of test creation)

Valid 10gb Cluster Interconnect Switches

  • Cisco 5010 – 12 or 18 node clusters
  • Cisco 5020 – Up to 32 nodes (not like thats possible, realistically 24 nodes)
  • Cisco 5548UP
  • Cisco 5596UP – Up to 40 nodes
  • Cisco CN1610 (rebranded Broadcom BCM53716-16FE or larger port count)


Sample cabling plan:

  • Netapp CN1610 Cabling.

Overview Cabling Diagram


Sample Cabling Guide of a Flexpod – For illustration only. Dont use on modern production.



3.1.6 Verify fiber cabling plan and availability of cable supports.




3.2 Following the rack diagram, install systems and FC switches.

Example Rack Diagram


3.3 Storage System Configuration Tasks.

3.3.1 Data ONTAP 8.1.1 Cluster-Mode Setup Tasks


3.3.2 Storage Provisioning and Vserver Setup Tasks

  • vserver create
  • protocol configuration
  • volume create
  • Adding to namespace
  • lun create
  • NFS: Export policy creation


  • Other tasks
    • Lif Migrate
    • ARL Aggregate Relocate
    • DataMotion for Volumes

3.3.3 FC, FCoE, and iSCSI Connectivity Tasks


  • We are almost always deploying a “switched fabric” with a core-edge topology with FC.
    • Core-edge topology: In this design, storage is always put in the core, and hosts are always attached at the edge. This design is effective because SAN traffic flows are typically not peer to peer but instead many to one (hosts to storage). (Definition from Cisco MDS documentation)
    • Edge-core-edge topology: This common design (storage edge to core to host edge) is used when a core-edge design provides insufficient scalability and an additional edge tier is needed to handle the large number of devices. (Definition from Cisco MDS documentation)
[caption id="" align="alignnone" width="265"] Topologies[/caption]


FC Cable comparison (because they always have annoying distance questions, not like we couldnt just google this when we need to)

Comparison (from Wikipedia)

Category Minimum modal bandwidth
850 nm / 1310 nm
100 Mb Ethernet 100BASE-FX 1 Gb (1000 Mb) Ethernet 1000BASE-SX 10 Gb Ethernet 10GBASE-SR 40 Gb Ethernet 100 Gb Ethernet
OM1 (62.5/125) 200 / 500 MHz·km up to 2000 meters (FX) 275 meters (SX) 33 meters (SR) Not supported Not supported
OM2 (50/125) 500 / – MHz·km up to 2000 meters (FX) 550 meters (SX) 82 meters (SR) Not supported Not supported
OM3 (50/125) *Laser Optimized* 1500 / 2000 MHz·km up to 2000 meters (FX) 550 meters (SX) 300 meters (SR) 100 meters330 meters QSFP+ eSR4 100 meters
OM4 (50/125) *Laser Optimized* 3500 / 4700 MHz·km up to 2000 meters (FX) 1000 meters (SX) 400 meters (SR) 150 meters550 meters QSFP+ eSR4 150 meters

Fibre Channel loop speeds/Distance from Siemons

Connection Speed and Distance by Cable Category
Type Speed Distance
OM2 1Gb/s 500m/1,640’
OM3 1Gb/s 500m/1,640’
OM2 2Gb/s 300m/900’
OM3 2Gb/s 500m/1,640’
OM2 4Gb/s 150m/492’
OM3 4Gb/s 270m/886’
OM2 8Gb/s 50m/1,64’
OM3 8Gb/s 150m/492’
Twinax copper 8Gb/s 15m max’
  • Optical for SAS between controller and shelves distances: The point-to-point (QSFP-to-QSFP) path of any multimode cable cannot exceed 150 meters for OM4 and 100 meters for OM3.  Though this isnt supported on 8.1.1.
  • Important note!:  If you are using patch panels with a different thickness (62.5/125 instead of 50.125), you should match the patch panel fiber thickness to the end host or san A transition between 62.5/125 and 50/125 may result in a possible loss of signal strength.  Read this for more information. ( )




3.3.4 LUN Connectivity Tasks

  •  ALUA information for pathing preferences is gathered by the host sending scsi inquiry (or new  REPORT_TARGET_PORT_GROUPS) command. (From Netapp Knowledge Base)
    • The storage system implements four states for a LUN:
      • Active/Optimized
      • Active/Non-Optimized
      • Unavailable
      • Transitioning
    • These map to the following existing Data ONTAP terms:
      • Local/Fast/Primary
      • Partner/Proxy/Slow/Secondary
      • Cluster IC is down, path is not functional
      • Path is transitioning to another state



3.3.5 Configure FC and Ethernet switches


3.3.6 Host Configuration Tasks

3.3.7 Virtualized Environment/Platforms: SAN Best Practices.

3.3.8 FCoE and Unified Connect Enabling Technologies

  • Relies on DataCenter Bridging (DCB)
    • Data center bridging (DCB) is a collection of extensions to the existing Ethernet standard that provides a lossless transport layer for FCoE traffic.
      • Per-priority pause (priority-based flow control)
        • Enables a device to only inhibit the transmission of frames based on user-defined priorities.
      • Enhanced transmission selection
        • Allows administrators to allocate bandwidth on a percentage basis to different priorities.
      • Congestion notification
        • Transmits congestion information.
      • DCB Exchange (DCBX) protocol
        • Exchanges connection information with directly connected peers and detects misconfigurations.
    • SAN Admin Guide Troubleshooting – Page 70-71 is a great resource for seeing this in action
      • Default priorities is 3 for FCoE traffic & 50% Bandwidth

        • Page 20 of TR3894 states this.
      • Default priority is 0 for IP traffic.


3.3.9 FCoE and Unified Connect Hardware

3.3.10 FCoE and Unified Connect Configuration

To move:

  • Must read: FCoE End to End Deployment (TR-3800 Older – 2011, w/ Qlogic)
  • Configuring CNA/UTA Ports
    • ucadmin
    • fcp config
  • FCoE Overview for Clustered Ontap
  • FC and FCoE Zoning
    • Best practices and recommendations
      • Netapp recommends “Single Initiator Zoning”
        • A zone should include a SINGLE INITIATOR and ALL Targets the initiator is connecting to.
      • Zoning should be based on World Wide Port Name (wwpn)
        • Change an HBA card, update the zone. Change a server, update the zone.
      • 50/125 Recommended.
      • Short Wave SFPs required to connect to onboard FC.
      • Should be used when you have 4 or more hosts connected (really always)
      • Orange cable typically OM2. “Laser Optimized” needed for short length 10gb.
      • OM3 and OM4 are cyan (blue), recommended for 8gb FC, 10GB eth.
  •  Why Zone?
    • Reduces CROSS TALK between initiator HBAs
    • Reduces paths to an available port
    • Increases Security
    • Shortens troubleshooting times
  • Zoning


1 Comment Continue Reading →

Thoughts on Certifications (aka I passed my CCNP DataCenter)

CCNP Datacenter

6 required exams. 21 days.

I didn’t make a large hoopla when I passed my CCNP Datacenter and CCNA Datacenter last week. I’ve always wanted to be Cisco certified. Though, I always felt anything but CCIE was sort of bunk. (still do, though I have met MANY incapable CCIE) I’m no smarter today. I’m no better of an engineer today. The only thing I know better today, is where a few documents are. I’ve been using the Nexus product line since the 5010’s first came out a few years ago. I play with 5548s in customer environments from time to time, same with 6248 FIs.

Click to continue reading “Thoughts on Certifications (aka I passed my CCNP DataCenter)”

1 Comment Continue Reading →

Tip #3: Adding additional networks to VMware Fusion 4/5

So, you are studying for a networking certification. Maybe CCNA or CCNP, maybe F5 or Juniper. You need multiple interfaces to run GNS3/Dynamips or your favorite networking tool. Alas, the default configuration of VMWare Fusion just has the basic VMnet1 and VMnet8. Here is how to fix that!

Click to continue reading “Tip #3: Adding additional networks to VMware Fusion 4/5”

Leave a comment Continue Reading →