EXAMTOPICS / TECHNOLOGY INSIGHTS

Cisco 350-601: VXLAN EVPN in Cisco Data Centers

VXLAN EVPN gives a data-center fabric a scalable way to carry Layer 2 segments and Layer 3 tenant reachability across an IP underlay. The active 350-601 DCCOR scope includes overlay networking because modern Cisco data-center designs increasingly separate simple routed transport from endpoint reachability, segmentation, and gateway policy in the overlay.

The architecture combines VXLAN as the data-plane encapsulation with BGP EVPN as the control plane. VTEPs encapsulate tenant traffic into VXLAN, VNIs identify overlay segments, and EVPN route types advertise MAC, IP, prefix, and multihoming information. The design is powerful because the underlay can remain a predictable IP fabric while the overlay carries tenant-specific state.

Underlay and overlay solve different problems

The underlay provides IP reachability between VTEP loopbacks. It should converge quickly, support ECMP, and remain operationally simple. OSPF, IS-IS, or another routed design can be used depending on the platform and architecture. The overlay runs on top of that reachability and should not be used to compensate for a fragile underlay. If VTEPs cannot reach each other reliably, EVPN control-plane routes and VXLAN tunnels will fail regardless of overlay configuration.

This separation is useful during troubleshooting. First prove loopback reachability, MTU, ECMP, and routing stability in the underlay. Then inspect BGP EVPN sessions, route types, VNIs, and endpoint learning. Jumping directly to EVPN when the VTEP loopback is unreachable wastes time because every overlay symptom is downstream of the transport failure.

VNIs define Layer 2 and Layer 3 overlay contexts

A Layer 2 VNI identifies a broadcast domain that may span multiple leaf switches. Hosts in the same VNI can communicate at Layer 2 even when they attach to different VTEPs. A Layer 3 VNI represents a tenant routing context and is commonly associated with a VRF for symmetric integrated routing and bridging. Keeping L2 and L3 VNIs conceptually separate helps explain whether a fault is bridging within a subnet or routing between subnets.

The VRF relationship is especially important in multi-tenant fabrics. The VRF separates tenant routing state, while VNIs provide overlay identifiers that can be signaled through EVPN. If a host route is learned in the correct L2 VNI but inter-subnet traffic fails, verify the L3 VNI, VRF association, route-target policy, and anycast gateway state instead of treating the problem as generic VXLAN encapsulation.

EVPN replaces flood-and-learn with control-plane reachability

Classic VXLAN can discover remote MAC addresses through data-plane flooding and learning, but EVPN distributes endpoint information through BGP. Type 2 routes advertise MAC and optional IP bindings, allowing remote VTEPs to program reachability without first receiving unknown unicast traffic. Other route types support multihoming, inclusive multicast, IP prefixes, and Ethernet segments.

This control-plane learning improves scale and visibility, but it introduces BGP policy and route-target behavior into what previously looked like a Layer 2 problem. A missing endpoint can be caused by local learning, EVPN advertisement, import policy, route-target mismatch, or remote programming. Engineers should trace the endpoint from local attachment to BGP EVPN table to remote forwarding state.

Anycast gateways keep first-hop routing local

Distributed anycast gateways let multiple leaf VTEPs present the same default-gateway IP and virtual MAC for a subnet. A host can keep using the same gateway identity after moving to another leaf, while inter-subnet traffic is routed at the closest VTEP. This avoids hairpinning through a centralized gateway and supports east-west scale.

Consistency is mandatory. The anycast gateway MAC, SVI addressing, VNI mapping, and VRF context must align across participating VTEPs. A mismatch can produce intermittent ARP/ND behavior or traffic that works from one rack but not another. Troubleshooting should compare the gateway configuration on both the source and destination leaf rather than assuming anycast means every leaf is equivalent automatically.

Symmetric IRB simplifies large routed fabrics

In symmetric integrated routing and bridging, ingress and egress VTEPs perform routing with a Layer 3 VNI between them. This model scales because each leaf does not need every remote tenant subnet as a local Layer 2 construct. The fabric can exchange routed tenant reachability while preserving tenant isolation through VRFs and route targets.

The trade-off is a deeper packet walk. A packet can enter an L2 VNI, be routed into an L3 VNI, cross the fabric, and then be routed or bridged again at the egress VTEP. Operators should be able to identify the expected inner and outer headers, VNIs, and route lookups. That understanding turns packet capture and counter inspection into a deterministic workflow instead of trial-and-error.

BUM traffic still needs a replication strategy

Broadcast, unknown unicast, and multicast traffic cannot be delivered solely from unicast endpoint routes. VXLAN fabrics therefore need an ingress-replication or multicast-underlay method for BUM delivery, depending on the platform and design. The choice affects underlay multicast state, headend replication load, and failure behavior.

Unexpected BUM volume often indicates an endpoint-learning problem, a loop, or a design that stretches too many Layer 2 domains. Monitor replication counters and unknown-unicast behavior rather than focusing only on BGP route count. A fabric with perfect EVPN sessions can still suffer poor performance if the Layer 2 service model creates unnecessary flooding.

Multihoming and failure domains

EVPN multihoming lets an endpoint or downstream switch attach redundantly to multiple VTEPs while the control plane coordinates designated-forwarder behavior and reachability. The objective is to provide resiliency without creating duplicate forwarding or loops. Ethernet Segment Identifiers and EVPN route signaling are central to that coordination.

Failure testing should include single-link, single-leaf, and control-plane events. A dual-homed server may retain physical connectivity while losing the expected EVPN route or designated-forwarder state. Verify traffic in both directions during failover and ensure that convergence does not depend on stale MAC entries. Redundancy should be demonstrated by packet forwarding, not inferred from two cables being present.

VXLAN EVPN and Cisco ACI are related but different models

Cisco ACI also uses an overlay fabric, but its operational model is policy-driven through APIC and tenant constructs rather than a primarily NX-OS CLI/BGP EVPN workflow. Understanding ACI architecture is useful because both models separate underlay transport from overlay endpoint policy, yet their configuration, object model, and troubleshooting interfaces differ.

The distinction matters when teams move between Cisco data-center platforms. Concepts such as VRFs, endpoint learning, anycast gateways, route distribution, and encapsulation remain relevant, but the source of truth changes. In standalone VXLAN EVPN, engineers interact directly with BGP, VNIs, and NX-OS fabric state. In ACI, the controller translates intent into distributed fabric policy.

Verification should follow the packet path

A repeatable verification sequence starts with underlay IP reachability between VTEPs, including MTU. Then verify NVE/VXLAN interface state, VNI membership, BGP EVPN neighbor state, imported route targets, and endpoint routes. Confirm local MAC/IP learning and remote programming before testing inter-subnet routing through the anycast gateway.

VXLAN EVPN troubleshooting across Cisco data-center certifications depends on tracing the interaction among the routed underlay, BGP control plane, tenant VRFs, VNIs, endpoint learning, and distributed gateways. Engineers who can follow those layers in order are better prepared to diagnose partial failures where only one tenant, subnet, or endpoint is affected.

Scale validation should include both control-plane and hardware tables. A fabric can have healthy BGP sessions while approaching platform limits for MAC entries, ARP/ND state, routes, VNIs, or next hops. Growth in endpoint count is not the only dimension: multi-homing, route type 5 prefixes, and tenant segmentation can increase state even when server count stays flat. Capacity planning should therefore track the actual tables consumed by the selected design and platform.

Operational consistency across leaf software versions is also important. EVPN and VXLAN capabilities evolve, and features such as anycast gateway behavior, route types, or hardware support can vary by release and switch model. During upgrades, validate feature parity and interop in a staged fabric rather than assuming a control-plane session proves complete forwarding compatibility. A mixed-version maintenance window should have explicit success criteria for endpoint learning, inter-subnet routing, and failover.

Security policy needs an overlay-aware perspective as well. Traditional ACL placement may not map cleanly to encapsulated traffic, and hardware restrictions can differ for packets before and after VXLAN termination. Decide where policy is enforced—at the host-facing SVI, VRF, firewall service, or another control point—and verify that encapsulation does not bypass the intended inspection path. Overlay reachability and security enforcement must be tested together.

Route-target design should be reviewed as carefully as VLAN/VNI mapping. A VTEP can learn a valid EVPN route from BGP but refuse to import it into the intended tenant because the route target does not match. That failure is especially easy to miss in large fabrics where the control plane appears healthy globally. Check the route distinguisher, import/export route targets, and the specific VRF/VNI context for the endpoint or prefix being investigated. If type 5 prefix routes are used for inter-subnet reachability, validate next-hop resolution and whether the receiving leaf installs them into the correct VRF.

Change windows should include route-control validation as well as endpoint tests. Adding or removing a VNI, VRF, or route target can affect many remote VTEPs after BGP convergence. Capture representative EVPN routes and endpoint reachability before the change, then compare them afterward. This makes unintended route withdrawal visible immediately instead of waiting for an application team to report that only one tenant or subnet has disappeared.

For migration into an EVPN fabric, move one boundary at a time and verify both endpoint and prefix routes after each stage. That keeps Layer 2 stretch, gateway migration, and routing-policy changes from becoming one indistinguishable event.