- Scenario 1
- Scenario 2
- Scenario 3: multicast source connected to external PIM domain
- Scenario 4: both multicast source and receiver are in external PIM domain
The objective of this article is to understand how EVPN OISM (Optimised Inter Subnet Multicast) operates in certain scenarios/designs and understanding the logical flow.
This article is best suited if you have already read the below TOI and have some trouble understanding how it would work in real world:
In above topology, host Sender multicast traffic is in Vlan 10 connected to VTEP-1 and Receiver is Vlan 20, connected to VTEP-2. Please ignore VTEP-3 for now.
On both VTEP-1 and VTEP-2 (showing output only for VTEP-1):
BGP Underlay neighbour IP 188.8.131.52 and BGP EVPN neighbour IP- 184.108.40.206 (Spine).
BGP Underlay neighbour IP 220.127.116.11 and BGP EVPN neighbour IP- 18.104.22.168 (Spine).
Points to note
- In case of directly connected multicast source/sender, the multicast traffic received from source is ALWAYS encapsulated with the L2 VNI (bridged) in any scenario in OISM. So just defining the L3VNI (VRF red) is not sufficient, L2 VNI must be configured under “interface Vxlan 1” and also there should be a mac-vrf for the Vlan defined under “router bgp” which should contain the RD and RT for the Vlan.
- The routing of multicast traffic happens on the receiving VTEP (where receiver is connected). It will receive the traffic with L2VNI of source and will route it into SBD Vlan (more on this later).
- Whether a VTEP supports OISM or not is advertised in the EVPN IMET route (type-3).
Example: see the below snapshot of IMET route advertised by VTEP-1:
- When a L3VRF (VRF red) is mapped to a VNI, a dynamic Vlan is created for the VRF. The dynamic Vlan is the backbone of OISM to enable multicast routing between a Vlan and a VRF. The dynamic Vlan is also referred to as Supplementary Bridge Domain (SBD).
- The command “evpn multicast” in the ip-vrf defined under “router bgp” is mandatory for the VTEP to perform OISM (capability advertised in IMET route) and also igmp-proxy for the L2Vlan, which is also advertised in the IMET route.
- When this command is enabled:
- A SPMSI-AD route (type-10) is generated for both mac-vrf and ip-vrf. The route contains the RT of both mac-vrf and ip-vrf and RD is that of the mac-vrf. Example (SPMSI generated by VTEP-1 for Vlan 10):Here the PMSI Tunnel Multicast group is 22.214.171.124 which is the underlay group for VRF-red. However, if we configure a dedicated multicast group for Vlan 10, then that would be used instead of 126.96.36.199.
- Another SPMSI-AD route is generated for the ip-vrf (VRF red). It contains the RD/RT only that of the ip-vrf.
Example below (generated by VTEP-1):
Scenario 1: logical flow
- Sender in Vlan 10 is sending multicast traffic to the group 188.8.131.52, however at this point no receiver wants to receive the feed.
- Receiver in Vlan 20 now sends an IGMPv2 report for 184.108.40.206 to receive the feed.
- As soon as the report is received on VTEP-2, VTEP-2 generates a SMET route for Vlan 20 mac-vrf.
- VTEP-2 would also join the underlay PIM tree (10.10.10.10, 220.127.116.11) to receive Vxlan encapsulated multicast traffic from VTEP-1
- After VTEP-1 receives the SMET route from VTEP-2, it is aware that there is a remote receiver that is interested in receiving the multicast traffic from sender (for group 18.104.22.168).
- VTEP-1 will start encapsulating the incoming multicast traffic from sender (22.214.171.124) with VNI as 10, outer source IP 10.10.10.10 and outer destination IP 126.96.36.199, which would be received by VTEP-2 and will get routed to Vlan 20.
- It is not possible to do inter Vlan Multicast routing (when sender and remote receiver are in different Vlan) without the SBD Vlan (L3 VRF- VNI).
Outputs (show commands)
After receiver sends IGMPv2 report to join 188.8.131.52 towards VTEP-2
- SMET route (generated by VTEP-2) is received on VTEP-1:
- VTEP-2 joins the underlay multicast tree (10.10.10.10, 184.108.40.206) to receive encapsulated multicast traffic:
VTEP-2…08:34:32(config-if-Et6/4)#show pim ipv4 sparse-mode route
Note that the OIL is Vlan 4094 (which has a star sign), this is the dynamic Vlan (SBD) that is mapped to the VRF red. This mroute will simply route the decapsulated multicast traffic into the VRF red. From VRF red, the multicast traffic would route into Vlan 20.
- Mroute table on VTEP-2 in VRF red (overlay mroute):
VTEP-2…08:38:43(config-if-Et6/4)#show pim ipv4 vrf red sparse-mode route
Here the IIF is Vlan 4094 (SBD Vlan) and the OIL is Vlan 20. This is how the multicast traffic sent by sender (in Vlan 10) in VTEP-1 would route from Vlan 10 to SBD Vlan and then from SBD Vlan to Vlan 20.
- Underlay mroute on VTEP-1 (to encapsulate the traffic):
VTEP-1…08:50:34(config)#show pim ipv4 sparse-mode route
- Mroute table on VTEP-1 in VRF red (overlay mroute):
VTEP-1…05:54:31(vrf:red)(config)#show ip mroute
Note that there is no OIL. This is because the multicast traffic received on VTEP-1 from a directly connected source is bridged with L2 VNI in Vxlan (not routed).
- After VTEP-1 receives the SMET route, an entry is created in SBD Vlan (Internal Vlan for VRF red) of snooping table indicating that there is an interested remote receiver:
Due to the presence of this entry (PIM-Tunnel), the incoming multicast traffic is encapsulated in Vxlan and uses L2VNI to encapsulate it. The traffic reaches VTEP-2, where it would decapsulate it and route it into SBD Vlan, and from SBD Vlan it will route it to Vlan 20 (See the previous points 2. and 3.).
In above topology, the receiver is connected to VTEP-3 where Vlan 10 doesn’t exist, the source is connected to VTEP-1 in Vlan 10 and Vlan 30 doesn’t exist on VTEP-1
Overlay multicast group (traffic sent by the end multicast source) 220.127.116.11.
Vxlan and BGP configuration
- Configuration of VTEP-1 and VTEP-2 is same as that in Scenario 1.
- Vxlan Configuration of VTEP-3:
- BGP configuration of VTEP-3:
BGP Underlay neighbour IP 18.104.22.168 and BGP EVPN neighbour IP 22.214.171.124 (Spine).
- BGP configuration of VTEP-1:
The only new configuration added is to advertise the subnet of the multicast source (192.168.1.0/24) in VRF red. If VTEP-3 receives IGMPv3 report from receiver for (126.96.36.199, 192.168.1.10) it needs to have a route to the source IP. However, if it is a v2 report (*, 188.8.131.52), then it will not check for route to source or RP, it will directly form the mroute (more on this later).
Scenario 2: logical flow
- As discussed at point 1. under “Points to Remember” earlier, any traffic sent by multicast source (directly connected) will be encapsulated with it’s L2 VNI. So the traffic from sender will be encapsulated with L2VNI of 10 and outer destination IP would be 184.108.40.206.
- The VNI-10 is not defined in VTEP-3, so the question arises: how will VTEP-3 decapsulate the packet and forward to the Receiver in Vlan 30? This is explained in below points:
- VTEP-1 advertises the SPMSI-AD (type-10) route for the mac-vrf Vlan 10, which is shown below:
In this output, you would notice that it has the MPLS Label as 10 (which is VNI-10) but it has the route-target of both mac-vrf (vlan10 → RT 10:1) and ip-vrf (VRF red → RT 100:1).
VTEP-3 receives this route, it uses the VRF red route-target (100:1) to import this route. When it imports, it maps the VNI-10 with the SBD Vlan (dynamic vlan for vrf-red). This you can see in below output:
Here, the source- “evpn multicast” means the L2 VNI (10, 20) is imported from EVPN route and mapped to SBD Vlan (4092). However, if the VNI is also defined locally (under “interface Vxlan 1”), then the local mapping would take preference. The VNI-100 is that of VRF red which is locally configured.
You will also notice that the SBD Vlan (4092) is mapped to multiple VNIs (10,20,100). A Vlan can be mapped to multiple VNIs. But a single VNI cannot be mapped to multiple Vlans.
Now, when VTEP-3 receives the encapsulated multicast feed (with L2VNI-10), it will map the VNI 10 to the SBD Vlan internally and then route the packet.
VTEP-3 may also receive the multicast feed with L3VNI-100 (which would happen if the source is not directly connected to VTEP-1 and is part of a different PIM domain). In that case as well, it will decapsulate the packet and map the VNI 100 to SBD Vlan 4092 and then route it.
- VTEP-1 advertises the SPMSI-AD (type-10) route for the mac-vrf Vlan 10, which is shown below:
- Underlay mroute on VTEP-3:
The IIF is Ethernet50/2 (towards spine) and OIL is SBD Vlan 4092.
- Mroute output on VTEP-3 in VRF red when receiver sends IGMPv2 report (*, 220.127.116.11):
The OIL is Vlan 30 as receiver is sent the join in Vlan 30 and the IIF is SBD Vlan. It directly forms the mroute. Multicast packets will be successfully delivered to the receiver.
- Mroute output on VTEP-3 in VRF red when receiver sends IGMPv3 report (192.168.1.10, 18.104.22.168) and there is no route to source IP:
The IIF is null as there is no route to source. Multicast packets, received on Vxlan1, will be dropped due to RPF check failure (fastdrop for SBD Vlan will be programmed)
- Mroute output on VTEP-3 in VRF red when receiver sends IGMPv3 report (192.168.1.10, 22.214.171.124) and there is a route to source IP:
This is the correct state and packets will be forwarded to the receiver.
Scenario 3: multicast source connected to external PIM domain
In this topology, the multicast sender (Vlan 50- 126.96.36.199) is connected to an external router which is also the gateway (FHR) for the sender. The router has a PIM neighbourship with VTEP-3. In reality there could be several other PIM routers between VTEP-3 and the FHR.
Sender is sending multicast feed to group- 188.8.131.52 and source IP is 184.108.40.206.
Scenario 3: logical flow with outputs
- Similar to Scenario-2, VTEP-3 has a route to the source (220.127.116.11) in VRF red with the next-hop pointing to the router:
- The route 18.104.22.168/32 is also advertised in VRF red under “router bgp” in VTEP-3. This creates a EVPN Type-5 route which is sent to VTEP-1 (where receiver is directly connected) and gets installed on VTEP-1:
The route to multicast source on VTEP-1 in VRF red.
- Now the receiver in Vlan 10 (connected to VTEP-1) sends an IGMPv3 report for (22.214.171.124, 126.96.36.199). This creates an IGMP snooping entry on VTEP-1 for the receiver and hence VTEP-1 generates a SMET route to indicate to other VTEPs that there is an interested remote receiver.
Note Didn’t include IGMPv2 report.Only difference in v2 would be that the mroute would be formed without the need for route to Source (similar to Scenario-2) as the VTEP acts as a RP (forms the RPT tree).
SMET route received on VTEP-3 from VTEP-1:
This SMET is generated in ip-vrf (red) as per the configuration. An SMET route for mac-vrf (Vlan 10) would also have been generated if “redistribute igmp” was configured in Vlan 10 under “router bgp” of VTEP-1. However, since Vlan 10 is missing on VTEP-3, VTEP-3 will only honour the SMET route generated for the VRF red even if a SMET for Vlan 10 was also received.
- VTEP-1 has also received a SPMSI route from VTEP-3 for the VRF red:
Using this route, VTEP-1 joins the underlay multicast tree (188.8.131.52, 184.108.40.206) (after the report from receiver is received) in order to receive the encapsulated multicast feed from VTEP3.
- Underlay PIM mroute (220.127.116.11, 18.104.22.168) on VTEP-1 (receiver side):
The IIF is the uplink (Ethernet9/1) and the OIL is the SBD Vlan (4091). VTEP-3 would encapsulate the multicast feed with L3VNI (VNI-100 → VRF red), unlike in case of directly connected source where it is encapsulated with L2VNI. VNI-100 is mapped to SBD Vlan (4091) on VTEP-1. After decapsulating the multicast traffic, it will be routed into SBD Vlan (4091).
- PIM mroute (22.214.171.124, 126.96.36.199) in VRF red on VTEP-1:
The native multicast packet would finally be routed from SBD Vlan into the Vlan 10 where receiver is present.
- Underlay PIM mroute (188.8.131.52, 184.108.40.206) on VTEP-3 (sender side):
- PIM mroute (220.127.116.11, 18.104.22.168) in VRF red on VTEP-3:
Multicast feed would be received on Ethernet47 from external PIM domain and then it will get encapsulated with L3VNI of VRF red (VNI- 100) with outer source IP as 22.214.171.124 and outer destination IP as 126.96.36.199.
Scenario 4: both multicast source and receiver are in external PIM domain
This is explained in great detail in the following TOI- https://eos.arista.com/eos-4-26-2f/multicast-evpn-transit/
Before reading the TOI, make sure you have covered all the 3 scenarios and understood the flow.