• Taking packet captures on Arista devices

 
 
Print Friendly, PDF & Email

Control-plane packet capture

TCPDUMP on physical ports and SVIs. This will help in capturing only control plane traffic but no data plane traffic.

Running tcpdump natively in EOS

#tcpdump interface Management1 filter ether proto 0x88cc
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ma1, link-type EN10MB (Ethernet), capture size 65535 bytes

11:33:47.750573 00:1c:73:00:44:d5 (oui Arista Networks) > 01:80:c2:00:00:0e (oui Unknown), ethertype LLDP (0x88cc), length 187: LLDP, length 173: s7151.lab.local

Running tcpdump from bash

bash ifconfig

et1       Link encap:Ethernet  HWaddr 00:1C:73:00:44:D6
         UP BROADCAST MULTICAST  MTU:9214 Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

et11       Link encap:Ethernet  HWaddr 00:1C:73:00:44:D6
         UP BROADCAST MULTICAST  MTU:9214 Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
[...]

ma1       Link encap:Ethernet  HWaddr 00:1C:73:00:44:D5
         inet addr:192.168.1.202  Bcast:255.255.255.255 Mask:255.255.252.0
         UP BROADCAST RUNNING MULTICAST  MTU:1500 Metric:1
         RX packets:2658 errors:0 dropped:0 overruns:0 frame:0
         TX packets:1579 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:394599 (385.3 KiB)  TX bytes:322439 (314.8 KiB)
         Interrupt:21
switch#bash tcpdump -i et11 stp
tcpdump: WARNING: et11: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on et11, link-type EN10MB (Ethernet), capture size 65535 bytes

11:55:39.244615 00:1c:73:00:44:e1 (oui Arista Networks) > 01:80:c2:00:00:00 (oui Unknown), 802.3, length 119: LLC, dsap STP (0x42) Individual, ssap STP (0x42) Command, ctrl 0x03: STP 802.1s, Rapid STP, CIST Flags [Proposal, Agreement], length 102

Data-plane packet capture

Monitor session

The Advanced Mirroring functionality on select Arista switches provides the ability to mirror to the CPU some data-plane traffic, whose internal path would normally never cross the control-plane since it is forwarded in hardware. Such data-plane traffic is exposed in the control-plane through an interface mirror, which can be listened to by the software of your choice; for example TCPdump

Below is the syntax:

monitor session <name> source interface <interface list> [ rx | tx | both ]
monitor session <name> destination <interface>

If there is a dedicated collector device to capture packets from the monitor session then we can use that port as a destination.

In case there is no dedicated device, we can punt the traffic to CPU to take the captures. It will not overwhelm the CPU as policies rate limiting traffic to CPU are in place.

(config)#[ no ] monitor session <sessionName> source <interface> [ rx | tx |  both ]
(config)#[ no ] monitor session <sessionName> destination cpu

Running tcpdump for data-plane traffic natively in EOS

(config)# monitor session MIRROR-CPU source ethernet4/4 [ rx | tx |  both ]
(config)#[ no ] monitor session <sessionName> destination cpu

Switch#show monitor session
Session MIRROR-CPU
------------------------
Source Ports:
   Rx Only: Et4/4


Destination Ports:               
   Cpu :  active (mirror0)

Switch#tcpdump monitor MIRROR-CPU

23:20:30.666829 00:50:56:99:fe:47 (oui Unknown) > 00:1c:73:85:bd:61 (oui Arista Networks), ethertype 802.1Q (0x8100), length 152: vlan 101, p 0, ethertype IPv4, 10.10.101.201.58504 > 10.10.200.101.4789: VXLAN, flags [I] (0x08), vni 10003
00:50:56:99:11:19 (oui Unknown) > 00:50:56:99:77:52 (oui Unknown), ethertype IPv4 (0x0800), length 98: 192.168.1.100 > 192.168.1.200: ICMP echo reply, id 45575, seq 6, length 64

Running tcpdump for data-plane traffic from bash

#show monitor session
Session MIRROR-CPU
------------------------
Source Ports:
   Rx Only: Et4/4


Destination Ports:               
   Cpu :  active (mirror0)


bash tcpdump -i mirror0
tcpdump: WARNING: mirror0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on mirror0, link-type EN10MB (Ethernet), capture size 65535 bytes
23:20:30.666829 00:50:56:99:fe:47 (oui Unknown) > 00:1c:73:85:bd:61 (oui Arista Networks), ethertype 802.1Q (0x8100), length 152: vlan 101, p 0, ethertype IPv4, 10.10.101.201.58504 > 10.10.200.101.4789: VXLAN, flags [I] (0x08), vni 10003
00:50:56:99:11:19 (oui Unknown) > 00:50:56:99:77:52 (oui Unknown), ethertype IPv4 (0x0800), length 98: 192.168.1.100 > 192.168.1.200: ICMP echo reply, id 45575, seq 6, length 64

Running tcpdump for data-plane traffic from bash for 7050/7060/7260 devices using mirror to GRE

Arista(config)#monitor session 1 source <source> rx
Arista(config)#monitor session 1 destination tunnel mode gre
                              source <sourceIp>
                              destination <destIp>
                              [ ttl <value> ]
                              [ dscp <value> ]
                              [ protocol <value> ]
                              [ vrf <value> ]
Arista#show monitor session 1 detail


Session 1
------------------------
Source Ports:
   Rx Only: Et4/4


Destination Ports:
     status     source       dest ttl dscp proto fwd-drop
  Gre1 : active 100.100.100.1 200.200.200.1 128 0 0x88be no
 next hop interfaces: Et4/10/1 Et4/10/2

GRE Payload Type: inner-packet


GRE Key Metadata: ingress-port vlan

    1. The rx keyword specifies that incoming packets should be mirrored. Only ingress mirroring is currently supported.
    2. The tunnel keyword specifies that the destination is a tunnel. Only the GRE mode is currently supported.
    3. The source keyword specifies the IP address to use as the source IP in the outer IP header of the GRE-encapsulated mirrored packet.
    4. The destination keyword specifies the IP address to use as the destination IP in the outer IP header of the GRE-encapsulated mirrored packet. The destination must be reachable.

Test setup

In the above scenario, a ping is initiated from SW2 to loopback0 on SW1. The ICMP echo request will be data plane traffic for SW1. We will configure “mirroring to GRE” to capture this data plane traffic.

We have created a GRE session on SW1 with a unique source IP to identify the switch from where the packets are being received while analysing the captures.

The destination is configured as IP address of Ethernet 5 of SW2.

After initiating a ping we can take tcpdump on Ethernet 5 of SW2 to check for gre encapsulated packets.

PING 1.1.1.1 (1.1.1.1) 72(100) bytes of data.
80 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=0.193 ms
80 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=0.083 ms
80 bytes from 1.1.1.1: icmp_seq=3 ttl=64 time=0.080 ms
80 bytes from 1.1.1.1: icmp_seq=4 ttl=64 time=0.080 ms
80 bytes from 1.1.1.1: icmp_seq=5 ttl=64 time=0.080 ms

Configuration

monitor session test source Ethernet5 rx
monitor session test destination tunnel mode gre source 11.11.11.11 destination 10.0.0.2 protocol 0x0800

Show commands

(config)#show monitor session

Session test

------------------------

Source Ports:

 Rx Only:     Et5

Destination Ports:

           status    source  dest ttl dscp proto fwd-drop

    Gre1 :  active   11.11.11.11        10.0.0.2 128 0 0x88be       no

      next hop interfaces: Et5

Viewing the packet capture on CLI

(config)#tcpdump int et5  filter inbound
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on et5, link-type EN10MB (Ethernet), capture size 65535 bytes
10:15:35.765323 44:4c:a8:2e:17:51 > 44:4c:a8:2e:23:6f, ethertype IPv4 (0x0800), length 152: 11.11.11.11 > 10.0.0.2: GREv0, proto unknown (0x88be), length 118: gre-proto-0x88be
10:15:35.765380 44:4c:a8:2e:17:51 > 44:4c:a8:2e:23:6f, ethertype IPv4 (0x0800), length 114: 1.1.1.1 > 10.0.0.2: ICMP echo reply, id 20220, seq 1, length 80
10:15:35.765507 44:4c:a8:2e:17:51 > 44:4c:a8:2e:23:6f, ethertype IPv4 (0x0800), length 152: 11.11.11.11 > 10.0.0.2: GREv0, proto unknown (0x88be), length 118: gre-proto-0x88be
10:15:35.765542 44:4c:a8:2e:17:51 > 44:4c:a8:2e:23:6f, ethertype IPv4 (0x0800), length 114: 1.1.1.1 > 10.0.0.2: ICMP echo reply, id 20220, seq 2, length 80

Viewing the packet capture on wireshark:

We can view the original ICMP packet encapsulated in GRE:

 

Sflow

Implementing sFlow on an Arista switch consists of configuring the following agent parameters:

  • Collector location address
  • Agent source address
  • Polling interval
  • Sampling rate

(Optionally, sFlow can be configured to include output interface and traffic class information in samples using the sflow sample command.)

To enable sflow globally

Configuring the collector location

The sflow destination command specifies the IP address and UDP port of an sFlow collector. The switch supports multiple collectors.

Example : This command configures the switch to send sFlow data to collectors at 10.42.15.12, port 6100 and 10.52.12.2 port 6343 (the default sFlow port).

switch(config)#sflow destination 10.42.15.12 6100
switch(config)#sflow destination 10.52.12.2 

Configuring the agent source address

The sflow source command specifies the source address that the switch places in all sFlow datagrams that it sends to the collector. This address is normally set to an IP address configured on the switch.

Example: This command configures 10.2.9.21 as the sFlow source address.

switch(config)#sflow source 10.2.9.21 

The sflow source-interface command can be alternatively used to specify the interface from which an IP address is derived that the switch places in all sFlow datagrams that it sends to the collector. This address is normally set to an IP address configured on the switch.

Example: This command configures VLAN interface 25 as the sFlow source interface. The switch enters the IP address for VLAN 25 in the source field of sFlow datagrams.

switch(config)#sflow source-interface vlan 25 

Note: The running-config cannot simultaneously contain sflow source and sflow source-interface commands.

Configuring the polling interval

The sflow polling-interval command specifies the interval for sending counter data to the sFlow collector. The default interval is two seconds.

Example: This command configures the switch to send sFlow data every ten seconds.

switch(config)#sflow polling-interval 10 

Configuring the sampling rate and sample contents

The sflow sample command sets the packet sampling rate. Packets are sampled at random intervals to avoid inaccurate sampling of periodic events. A rate of 16384 corresponds to an average sample of one per 16,384 packets. The default rate is 1048576.

Example : This command configures the sFlow sampling rate as 65536 (one per 65,536 packets).

switch(config)#sflow sample 65536 

The sflow sample command can also optionally configure sample packets to include information about the traffic class of the sample. Traffic class is communicated by rewriting the DSCP field in the sample packet.

By default, samples include information about the output interface. To remove this information, use the [no] sflow sample output interface command. These commands configure sFlow to include traffic class information in samples but to exclude output interface data.

switch(config)#no sflow sample output interface 
switch(config)#sflow sample rewrite dscp

Enabling sflow

The sflow run command globally enables sFlow on the switch. The sflow enable command controls sFlow operation on Ethernet and port channel interfaces when sFlow is globally enabled. The sflow enable command has no effect when sFlow is globally disabled.

Example : These commands enable sFlow on the switch, then disables sFlow on Ethernet interface 10.

switch(config)#sflow run 
switch(config)#interface ethernet 10 
switch(config-if-Et10)#no sflow enable

Show commands

The show sflow command displays configured sFlow parameters, operational status, and statistics.

The show sflow interfaces command displays the interfaces where sFlow is enabled.

switch#show sflow

Warning: displaying counters that may be stale sFlow Configuration

-------------------

Destination IP: 10.67.90.3

Destination Port: 6343 ( default )

Source IP: 0.0.0.0 ( default )

Sample Rate: 16384

Polling Interval (sec): 2.0 ( default )

  Status
   ------
   Running: Yes
   Polling On: Yes ( default )
   Sampling On: Yes ( default )
   Send Datagrams: No ( default )
   Hardware Sample Rate: 16384

  Statistics
   ----------
   Total Packets: 20334189
   Number of Samples: 1201
   Sample Pool: 19677184
   Hardware Trigger: 1205
   Number of Datagrams: 356


switch#show sflow detail

Warning: displaying counters that may be stale sFlow Configuration

-------------------

Destination IP: 10.67.90.3

Destination Port: 6343 ( default )

Source IP: 0.0.0.0 ( default )

Sample Rate: 16384

Polling Interval (sec): 2.0 ( default )

  Status
   ------
   Running: Yes
   Polling On: Yes ( default )
   Sampling On: Yes ( default )
   Send Datagrams: No ( default )
   Hardware Sample Rate: 16384
   Hardware Sampling On: No

  Statistics
   ----------
   Total Packets: 20334189
   Number of Samples: 1201
   Sample Pool: 19677184
   Hardware Trigger: 1205
   Number of Datagrams: 356
   Number of Samples Discarded: 0

Running tcpdump for data-plane traffic from bash for 7050/7060 devices using sflow

In the setup above Sw2 is representing 7050/7060 series devices.

For testing a ping will be initiated from Sw1 to Sw3 and the traffic passing through switch 2 will be dataplane.

Sflow tool is used to sample traffic on the device itself. The idea is to punt the traffic on the wire to CPU to capture data plane traffic.

Configuration

sflow sample 1  <<< This can be changed as per requirement. If sampling is done for each packet then
disable sflow on ALL interfaces (i.e. sflow interface disable) and then only enable on the interface of interest.
sflow destination 127.0.0.1
sflow source 127.0.0.1
sflow run
!
interface Ethernet19
   sflow enable

Ping initiated from Sw1 to Sw3:
bash ping 20.1.1.11
PING 20.1.1.11 (20.1.1.11) 56(84) bytes of data.
64 bytes from 20.1.1.11: icmp_seq=1 ttl=64 time=0.202 ms
64 bytes from 20.1.1.11: icmp_seq=2 ttl=64 time=0.191 ms

Show commands

(config)#show sflow
sFlow Configuration
-------------------
Destination(s):
 127.0.0.1:6343 (VRF: default)
Source(s):
  127.0.0.1 (VRF: default)
  :: (default) (VRF: default)
Hardware Sample Rate for SW sFlow: 1
Polling Interval (sec): 2.0 (default)
Rewrite DSCP value: No
Status
------
Running: Yes
Polling On: Yes (default)
Sampling On: Yes (default)
Send Datagrams:
  Yes (VRF: default)
BGP Export:
  No (VRF: default)
Hardware Sample Rate for SW sFlow: 1

Viewing the packet capture on CLI

(config)#bash sflowtool  -t | tcpdump -r - -vv icmp
listening
reading from file -, link-type EN10MB (Ethernet)
06:12:55.000000 00:1c:73:e1:68:ef (oui Arista Networks) > 00:1c:73:57:2d:03 (oui Arista Networks), ethertype IPv4 (0x0800), length 102: (tos 0x0, ttl 64, id 25511, offset 0, flags [DF], proto ICMP (1), length 84)
    20.1.1.2 > 20.1.1.11: ICMP echo request, id 25486, seq 18, length 64
06:12:55.000000 00:1c:73:57:2d:03 (oui Arista Networks) > 00:1c:73:e1:68:ef (oui Arista Networks), ethertype IPv4 (0x0800), length 102: (tos 0x0, ttl 64, id 49141, offset 0, flags [none], proto ICMP (1), length 84)
    20.1.1.11 > 20.1.1.2: ICMP echo reply, id 25486, seq 18, length 64

Limitations

  1. For sub-interfaces/port-channels the packet capture needs to be done on physical ports.
  2. While using GRE as a mirroring method make sure that collector/ destination of monitor session is via a L3 port.
  3. Monitor session destined to CPU on 7050/ 7060 devices will be available in near future, please use GRE or Sflow as an option till then.
  4. Sflow can only sample traffic on ingress.

References

  1. https://eos.arista.com/eos-4-15-2f/mirror-cpu/
  2. https://eos.arista.com/introduction-to-port-mirroring/
  3. https://eos.arista.com/eos-4-20-1f/advanced-mirroring-features/#Limitations
  4. https://eos.arista.com/introduction-to-managing-eos-devices-troubleshooting/#361Configuring_mirroring_to_the_CPU

Follow

Get every new post on this blog delivered to your Inbox.

Join other followers: