• Model validating PTP timestamping in a distributed Tap-Agg network

 
 
Print Friendly, PDF & Email

-- Download Model validating PTP timestamping in a distributed Tap-Agg network as PDF --


Abstract

This is a test setup using common servers to validate PTP timestamping accuracy of Arista 7150 switches.

Introduction

Tap Aggregation devices are a growing monitoring model, allowing monitoring applications to analyze a huge set of data sources. In some application monitoring, it’s critical to correlate time to achieve high precision information, where we can measure the precise time in a packet path. For this purpose we have hardware time-stamping, a feature where the tap aggregation device stamp a time reference in each packet. This subject is explored in this article. Devices performing timestamp in packets must use a common clock reference to synchronize all devices’ clock time. This can be accomplished using PTP.

Common issues when dealing with PTP

While deploying PTP is not hard, when it comes to control high precision applications where hundreds of nano seconds would make a difference in monitoring, we have to be careful about clock drifts between devices. Every device drifts its clock independently and in a PTP deployment, devices synchronize with a common clock, the Grand Master Clock. The PTP protocol exchanges messages to keep all devices in sync with the GM. Common issues found in real world deployments:

  • High offset values from PTP. Some offset is normal, but when we have high offsets values constantly, let’s say hundreds of nanoseconds, this must be fixed. It would be a jitter on PTP packets.
  • GM clock has a problem and is drifting or losing connection.

Concept Lab

To demonstrate a precision hardware time stamping, we’re using a concept lab that can be easily built using Linux servers and Arista DCS-7150S switches, using a topology in the following figure: Tap Lab TimestampThe Lab works as following:

  • The PTP GM server is a Linux server using a Ethernet line card with HW Time Stamping support. Most modern Ethernet Network cards has this support. We adopted Linux PTP project for our Lab, please refer to its project page for detailed information.
  • Arista switch SW1 is connected directly to PTP GM in boundary mode and providing PTP clock messages to Arista Sw2 and Sw3 switches.
  • The Traffic Generator can be a Linux Perf device to demonstrate this concept.
  • The TG sends data traffic into Arista SW1, configured also as a Tap Aggregation device. This switch replicates the same traffic to Arista Sw2 and Sw3 at same time.
  • Arista SW2 and Sw3 receive the same traffic and timestamps it to the Packet Capture Server, which is a Wireshark/TCPDump/TShark sniffer to save both traffic received from Sw2 and Sw3 in a Pcap file.

With this data, we expect to have both timestamped packets from Sw2 and Sw3 with pretty close UTC time values. For this analysis we used the PCAP Decoder script to extract the UTC values from both traffic. The PCAP decoder and keyframe usage is described in detail in this link.

Key details on Lab

The Linux PTP server is not a high precision device, its clock is not as stable comparing to a real PTP GM device. Thus, for this concept lab we need to make some tuning to have a more aggressive rate and avoid jittering clocks from devices. In the PTP GM we used the following /etc/ptp4l.conf config setup

[global]
priority1 64
priority2 64
verbose 1
logging_level 6
summary_interval 0
logSyncInterval -2

Switches configs

All switches were configured in Tap Aggregation mode and had tap and tools ports mapped. SW1 is acting as a split device and SW2 and SW3 as timestamping Tap Aggregation switches. An important note is when you configure switch in Tap Aggregation mode, you must configure non-tap ports as no-errdisable to use them for PTP synchronization . SW1 configuration notes:

ptp mode boundary
!
tap aggregation
   mode exclusive
   mode exclusive no-errdisable Ethernet1
   mode exclusive no-errdisable Ethernet13
   mode exclusive no-errdisable Ethernet15
!
interface Ethernet1
   ptp enable
!
interface Ethernet13
   ptp enable
!
interface Ethernet14
   switchport mode tool
   switchport tool group set Test
!
interface Ethernet15
   ptp enable
!
interface Ethernet16
   switchport mode tool
   switchport tool group set Test
!
interface Ethernet20
   switchport mode tap
   switchport tap default group Test
!
management api http-commands
   protocol http
   no shutdown

SW2 configuration notes:

ptp mode boundary
!
tap aggregation
   mode exclusive
   mode exclusive no-errdisable Ethernet13
!
interface Ethernet13
   ptp enable
!
interface Ethernet14
   switchport mode tap
   switchport tap identity 10
   switchport tap default group Test10
!
interface Ethernet24
   mac timestamp before-fcs
   switchport mode tool
   switchport tool identity dot1q
   switchport tool group set Test10
!
platform fm6000 keyframe sw2-kf interface Ethernet24 1.1.1.1 001c.73ab.1111
platform fm6000 keyframe sw2-kf device 10
platform fm6000 keyframe sw2-kf rate 100
platform fm6000 keyframe sw2-kf vlan 10
!
management api http-commands
   protocol http
   no shutdown

SW3 configuration notes:

ptp mode boundary
!
tap aggregation
   mode exclusive
   mode exclusive no-errdisable Ethernet15
!
interface Ethernet15
   ptp enable
!
interface Ethernet16
   switchport mode tap
   switchport tap identity 20
   switchport tap default group Test20
!
interface Ethernet24
   mac timestamp before-fcs
   switchport mode tool
   switchport tool identity dot1q
   switchport tool group set Test20
!
platform fm6000 keyframe sw3-kf interface Ethernet24 2.2.2.2 001c.73ab.2222
platform fm6000 keyframe sw3-kf device 20
platform fm6000 keyframe sw3-kf rate 100
platform fm6000 keyframe sw3-kf vlan 20
!
management api http-commands
   protocol http
   no shutdown

Concept Lab Results

For a 1 Mbps traffic rate, there was no deviation bigger than ~50 ns in a sample of 5K stamped packets, demonstrating great precision in timestamped packets. PTP_deviation_graph  

Follow

Get every new post on this blog delivered to your Inbox.

Join other followers: