[Linuxptp-devel] Despite the patch, "timed out while polling for tx timestamp" keeps happening

[Frantisek R. <Fra...@po...>]

Dear ptp4l developers,

this message is FYI, a "problem report from the wild"...

I have the privilege to play around with your software, on a 
Skylake-based platform containing an Intel i219LM 
(alongside i210 which seems to give comparably mediocre results).

I also have the privilege to test your software against a 
latest-generation Meinberg GrandMaster (HPS100).

I first tried your software on the stock kernel in Debian 8 
(3.16.something if memory serves) - and the "timed out while ..." 
errors prompted me to upgrade to the latest vanilla, which happens to 
be 4.13.12 at the time of this writing.
The errors are now less frequent, but they do occur.
I've noticed this debate and patch:
https://sourceforge.net/p/linuxptp/mailman/message/35810919/
https://patchwork.ozlabs.org/patch/758160/
I can see in the source code of 4.13.12 that the patch has been 
applied = it's present in the vanilla.

Attached you'll find an example of my error from the ptp4l output.
Probably not much new in there.
And, a GNUplot graph of the offset+delay (gnawed from ptp4l output). 
The needles on "offset" (the green trace) are a result of ptp4l 
resetting the port upon each occurrence of the bug, during which time 
it stops disciplining the clock for a while...

Notice how the errors become more frequent after 9 a.m.
- I came to the machine and started a PCAP sniffer 
on that same port. 
Yesterday I have uninstalled the NetworkManager 
(which had sneaked in as a "recommended" dependency
of the X desktop) so the network ports are no longer 
messed around with by dhclient. I have static addresses
in /etc/network/interfaces. And the machine ran overnight
without any PCAP activity, just the static IP address on eth0
and ptp4l querying the GM every second.

The one thing that has stunned me in a nice way:
I first ran ptp4l against the GM through two managed switches,
not very loaded, but unaware of PTP.
That resulted in the "offset" jumping within +/- 700 "units".
After I got the idea to plug the client straight into the GM,
the "offset" now wanders within +/- 10 "units". 
That's an improvement of two decimal orders :-D
Wonderful :-)
What are those units BTW? PPB? Nanoseconds?
Or, ticks of the PHC oscillator ?

Any comment welcome...

Makes me think of ASPM, I haven't checked that yet in the BIOS,
and it's possibly not tweakable in this "embedded/desktop" machine, 
but I guess the Intel NIC drivers now disable ASPM by default
in the NIC, so I don't expect to find any low-hanging fruit in 
there...

My attempts at recording the raw PTP traffic are obviously motivated
by a desire to be able to browse/analyze the logs afterwards.
As for the disruptive effect of the capturing activity, the 
particular application using libpcap doesn't make a difference 
(tcpdump, wireshark or ptpTrackHound - all the same).

The Debian on the box is using systemd, if that should 
make any difference... (I don't think so).

Thanks for the software gem that you have created,
and for including support for Intel hardware.

Frank Rysanek