Posted on July 19, 2021 2:58 am
 |  Asked by Michael Witte
 |  56 views
RESOLVED
0
0
Print Friendly, PDF & Email

I am using vEOS to build out a spine-leaf EVPN fabric, and using CVP, IPAM and cloud builder to push out appropriate configs. Each P-P link is a vswitch then i map the appropriate port groups which then map to the correct eth ports in the vEOS switch. CVP and cloud builder all seems to work fine building the config but I was having weird connectivity issues. After a great deal of digging it appears that the mapping of the vSphere VMs port groups and hence vswitch and P-P links  to vEOS Switches are bound correctly but upon reboot remap randomly. For instance VM portgroup nics 0-4 would be nic0= mgmt ,then portgroup nic 1-4 would be eth1,eth2,eth3,eth4. so then when applying configs via cloudbuilder it would create the configs for the appropriate NIC connectivity

What i found after digging into LLDP neighbors and matching mac addreses of the VM port group NIC 1, to the say eth1  after a reboot it could become eth4 which is verified by LLDP neighbors and mac addresses of the th port.

So on my spine;

NIC 1 MAC=Eth1 MAC

NIC 2 MAC =Eth2 MAC

NIC 3 MAC =Eth3 MAC

NIC 4 MAC =Eth4 MAC

After the reboot

NIC 1 MAC =Eth4 MAC

NIC 2 MAC =Eth1 MAC

NIC 3 MAC =Eth3 MAC

NIC 4 MAC =Eth2 MAC

This is repeatable and appears to have something with the way the interfaces come up and grabs the next port group really have no explanation. I dont know how to reserve the VM port group to vEOS eth mapping so its consistent and its impossible to reliably build a lab with this issue.

0
Posted by Tamas Plugor
Answered on July 19, 2021 1:29 pm

Seems like you're hitting the same issue describe by Colin here: https://eos.arista.com/forum/veos-cloudeos-and-interface-ordering/

0
Posted by Michael Witte
Answered on August 13, 2021 2:56 am

So the issue here is as you said, it randomly chooses the ESXi hosts PCI interfaces mapping and does not properly map NIC1-NIC7 of the VM(or how many NICs you have in VMWare) to Eth1-7 in vEOS. Since I am using vCloud director to build the labs, I have to verify what mac address the vEOS switch is using for the eth port, then look in vCloud to see what NIC MAC its mapped to

in the vEOS configuration for spine-leaf, I am using say eth1 on leaf1 to eth1 on spine 1. However the mac address assigned to these eth interfaces are assigned to maybe NIC2 of the leaf VM and NIC3 of the spine VM. I then needed to connect NIC2 of leaf1VM to NIC3 of spine 1 VM(which then connect eth1 leaf1 to eth1 spine1 under the covers).

This was very painful to find but thanks for pointing me in the right direction. From a lab student they wont see it, but from a lab builder perspective its a virtual cabling nightmare.

Post your Answer

You must be logged in to post an answer.