Running vEOS on ESXi 5.5

What is vEOS?

Arista Networks vEOS is a software only version of the EOS network operating system. vEOS is meant to be run in a virtual machine environment. vEOS is useful for feature testing and especially for development of scripts and extensions. vEOS can be run on many different virtualization platforms like Virtual Box, VMware Fusion or Workstation as well as ESXi. Arista Networks has previously published how to documentation for running vEOS on other virtualization platforms and this document will extend that documentation to ESXi.

What is ESXi?

VMware ESXi is a server virtualization platform that supports hypervisor clustering, distributed virtual switching and vMotion in addition to a very rich set of enterprise virtualization features. ESXi is meant to be run on server hardware for the purposes of virtualizing production workloads in the data center.

Why run vEOS on ESXi?

Having a persistent vEOS environment in a lab environment can be beneficial when compared to the limitations of running vEOS on a laptop or desktop machine. Server or workstation hardware have greater scalability of resources especially memory and CPU cores. This allows one to run more vEOS instances, with more front panel interfaces and multiple supporting machines that can act as servers for ZTP, Chef, etc. or to have multiple VMs attached to virtual switches in your test environment. This scalability lends itself to a richer lab environment where L2 MLAG and L3 ECMP environments are possible. The document is written from the perspective of running on ESXi standalone and it is not the intention of the author to suggest that advanced VMware features like vMotion or DRS are of specific benefit.

Scope

In this post you will learn how to create a vEOS VM instance. Next you will learn how to connect vEOS switches together with vSwitches acting as ISL links. We will connect vEOS-04, a new instance to an existing L2 MLAG environment. Finally we will show the configuration for adding the new vEOS instance as an MLAG member switch to the previously built MLAG domain.


Note: This document was written using ESXi 5.5 standalone without vCenter server using the vSPhere client. Distributed Virtual Switching (DVS) is out of scope as well as the updated web interface. Experimentation with Distributed Virtual Switches (DVS) as interconnects for vEOS MLAG did not work as expected and are not recommended by the author.

Note: If you are running ESXi 5.1 this article should still be relevant, as they say your mileage may vary, but do know that there is a bug prior to ESXi 5.1 Update 1 that may prevent your vEOS instance from booting properly. Please apply Update 1 or higher to avoid this outcome. See the following URL for reference:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2035777


Pre-requisites

vEOS topology diagram

ESXi vEOS lab November 2013

Four switch vEOS MLAG diagram

Build a vEOS VM in vSphere

From the vSphere Client create a new custom VM

Screen Shot 2013-11-21 at 3.50.01 PM

Choose a name, for example we called our machine vEOS-04. Choose the appropriate datastore for your environment. Select virtual machine version 8. Do not upgrade the virtual machine hardware unless you are using the web interface with a vCenter server. Otherwise you will lose the ability to manage the VM through vSphere client.

original1.Screen Shot 2013-11-21 at 3.50.45 PMoriginal1.Screen Shot 2013-11-21 at 3.54.03 PM

original1.Screen Shot 2013-11-21 at 3.54.35 PM

Choose Linux and “Other 2.6.x Linux (32-bit)” for the guest operating system

original1.Screen Shot 2013-11-21 at 3.56.50 PM

Select on socket and one CPU. Allocate 2GB RAM to the VM

original1.Screen Shot 2013-11-21 at 3.57.03 PM

original1.Screen Shot 2013-11-21 at 4.00.07 PM

Start with four network interfaces, this will provide a management interface and three front panel interfaces. More interfaces will be needed for larger topologies and can be added later. Choose VM Network for your default port-group, this will be changed later when we create the vswitches for interconnection.

original1.Screen Shot 2013-11-21 at 4.00.26 PM

Keep the default SCSI Controller as “LSI Logic Parallel”, in fact it will not be used. Next choose “Do not create disk”. We will select the VMDK later. Finally click finish but DO NOT POWER UP the VM yet.

original1.Screen Shot 2013-11-21 at 4.02.43 PM

original1.Screen Shot 2013-11-21 at 4.04.28 PM

Upload the Aboot and VMDK files to the directory of the new vEOS VM (vEOS-04 in our case). You will need to browse the datastore that you selected earlier and upload to the new folder that was created.

original1.Screen Shot 2013-11-21 at 4.10.04 PM

original1.Screen Shot 2013-11-21 at 4.09.38 PM
original1.Screen Shot 2013-11-21 at 4.11.19 PM

Next step is to change the virtual machine settings. First, remove the floppy drive. Next select the CD/ DVD Drive and specify the Aboot file as the “Datastore ISO File”. Finally, check mark the “connect at power on” box. The vEOS VM will use the Aboot ISO to boot each time, so keep it connected.

original1.Screen Shot 2013-11-21 at 4.37.28 PM

Add a new device, type Hard Disk. Select “Use an existing virtual disk…”. Then browse to the datastore under the VM folder and choose the VMDK file you uploaded earlier. Accept “IDE (0:0)” as the Virtual Device Node. Close the Add Hardware wizard and the Virtual Machine Properties dialog.

original1.Screen Shot 2013-11-21 at 4.18.46 PM

original1.Screen Shot 2013-11-21 at 4.18.56 PM

Boot the vEOS VM. Once booted, login with username admin. Password will be blank at default configuration.

original1.Screen Shot 2013-11-25 at 2.03.35 PM

Build ESXi vSwitches and Port-Groups

Each inter-switch link will require a vSwitch and a port-group. This will not work with a single vSwitch with multiple port-groups, we know because we tried this and failed miserably.

  • Each vEOS management interface (Network adapter 1) will connect to the VM Network on vSwitch0
  • vEOS-01 and vEOS-02 will connect on vSwitch1 with port-group PEER_LINK
  • vEOS-02 and vEOS-03 will connect on vSwitch2 with port-group MLAG_LEFT
  • vEOS-02 and vEOS-03 will connect on vSwitch3 with port-group MLAG_RIGHT
  • vEOS-04 requires an additional vSwitch (vSwitch5) for its ISLs to vEOS-01 and vEOS-02 which will be created as part of this document in the steps that follow.

vSwitch configuration showing vEOS-01, vEOS-02, and vEOS-03.

original1.Screen Shot 2013-11-21 at 4.22.46 PM original1.Screen Shot 2013-11-21 at 4.24.35 PM

Click on the ESXi host in the left hand navigation pane of vSphere.

  • Click on the Configuration tab in the right hand work pane
  • Choose Networking and Add Networking
  • Choose Virtual Machine as the connection type
  • Choose “Create a vSphere standard switch
  • Do not assign any physical interfaces to the vSwitch

original1.Screen Shot 2013-11-21 at 4.29.51 PM original1.Screen Shot 2013-11-21 at 4.32.24 PM

In the Port Group Properties

  • Create a Network Label, for vEOS-04 we are calling it “MLAG_LEFT2”
  • Change the VLAN ID to “All (4095)”
  • Edit the vSwitch and change the MTU to 9000
  • NOTE: Promiscuous Mode defaults to Reject. Some L3 configurations may require changing to accept. Some users report MLAG does not work without Promiscuous Mode set to accept on the peer-link
  • Repeat these steps to add a vSwitch for the MLAG_RIGHT2 port-group

original1.Screen Shot 2013-11-25 at 2.25.19 PM original1.Screen Shot 2013-11-25 at 2.31.11 PM

Make the port group assignments to vEOS-04 to connect it to the vSwitches

  • Right click on vEOS-04 and edit settings
  • Click Network adapter 2
  • Select MLAG_LEFT2 under Network Label
  • Check the box for “Connected” and “Connect at power on”
  • Repeat using MLAG_RIGHT2 for Network adapter 3
  • TIP: Uncheck “Connected” and “Connect at power on” for any adapters not currently in use

Screen Shot 2013-12-03 at 12.06.08 PM

When completed your vSwitch configuration will look similar to this. Note: vSwitch4 in this configuration was created for a different project, don’t be concerned if your configuration uses different vSwitch numbers.

Screen Shot 2013-12-28 at 9.26.30 PM

Screen Shot 2014-02-24 at 1.43.28 PM

Adding additional network interfaces

You may need to add additional ethernet interfaces to your vEOS instance as you build out larger configurations. This can be achieved simply by editing the settings of the vEOS VM.

Right click on the VM in vSphere and click Edit Settings

original1.Screen Shot 2013-12-03 at 12.05.08 PM

Click Add under the Hardware tab

 original1.Screen Shot 2013-12-03 at 12.05.36 PM

Choose Ethernet Adapter
original1.Screen Shot 2013-12-03 at 12.05.45 PM

Choose E1000, leave Network label at default (VM Network in our case) Click next and finish.

original1.Screen Shot 2013-12-03 at 12.05.58 PM

Choose the correct Network label according to your vSwitch setup and connectivity. You may need to add another vSwitch.

Screen Shot 2013-12-03 at 12.06.08 PM

Conclusion

vEOS can be run successfully on ESXi 5.5. ESXi enables larger configurations due to the capabilities of ESXi with respect to server class hardware containing many CPU cores and large quantities of memory. With this configuration a large vEOS topology can be constructed for lab testing L2 and L3 configurations and developing EOS extensions and scripts as well as training. With these instructions and a bit of patience you should be able to build a custom VM, configure the VM for use as a vEOS instance by attaching the Aboot and VMDK images. Finally, you should be able to interconnect the vEOS instances to fully leverage the capabilities of vEOS on ESXi 5.5.

Author’s note

Experimentation with Distributed Virtual Switches as interconnects for vEOS MLAG were largely a failure and not recommended by the author.