linuxptp-users

[Linuxptp-users] sawtooth from linux timestamp?

[jmfriedt <jmf...@fe...>]

Thanks to the help I received earlier I am now able to control the
Compute Module 4 (CM4) on (White Rabbit) PTP, NTP or GPS. I am trying
to understand the meaning of the 9 digits Linux is providing when
reading an inode state with "stat -c %y" and how many decimals are
actually relevant.

To do this I am connecting an Ettus Research X310 software defined
radio to the CM4 and configuring to start collecting data upon
detection of the WR 1PPS rising edge. The SDR input is connected to the
1PPS to assess that the record always stays at the same time with
respect to the 1PPS and it does. I record a known number of samples in
file.bin at a known datarate (5 MS/s) and look at the timestamp of the
last file access as the last field of stat -c %y file.bin, and I try
and analyze the factional part of this timestamp.

I get the attached chart which is a 16 hours long measurement with one
measurement every 16 s. I understand that the slope of the sawtooth
(inset) is the frequency difference between the oscillator driving the
processor and the PTP time as I was able to check by clocking the CM4
with an external synthesizer locked to the WR source (hydrogen maser).

What I cannot understand is the sawtooth shape and the relation to the
kernel CONFIG_HZ_250 setting, since changing to CONFIG_HZ_300 or
CONFIG_HZ_1000 does change the sawtooth amplitude from 4 to 3 ms and
then to negligible (~ms but not looking like a sawtooth anymore)
amplitude. I have also checked that by clocking the CM4 CPU with a
synthesizer locked to the WR GM reference, the sawtooth is gone and I
am left with the +/-5 us I attribute to kernel syscall latency
fluctuation.

Can anyone point me to some literature describing the transfer from PHY
PTP time to Linux kernel time and how it affects these timestamps? also
where the sawtooth comes from and its relation to the CONFIG_HZ kernel
setting? In general I wonder, when I read NTP to be ms accuracy and PTP
being nanosecond, how do users actually use such resolution (which I
can indeed observe on the PTP 1-PPS output from the PHY) at the
operating system level when the kernel itself timestamps with several
ms granuarity? With White Rabbit I can access the 10 MHz and 1PPS
output that allow me synthesizing any frequency I need in an
application, but PTP only provides the timestamp information which I
understand to be transferred to the operating system by phc2sys and I
see then the impact of the operating system timer granularity it seems.
I also cannot understand the long term (7-8 hour period) slow drift
visible on the top chart. I have verified that the PHY oscillator
frequency does not impact the result, only the CPU oscillator, which
might be expected if PTP is not only sharing the time transfer delay
but also the time to correct for PHY clock drift.

Thank you, Jean-Michel

-- 
JM Friedt, FEMTO-ST Time & Frequency, 26 rue de l'Epitaphe, 25000
Besancon, France

Re: [Linuxptp-users] sawtooth from linux timestamp?

[Miroslav L. <mli...@re...>]

On Sat, Jul 22, 2023 at 04:12:32PM +0200, jmfriedt wrote:
> To do this I am connecting an Ettus Research X310 software defined
> radio to the CM4 and configuring to start collecting data upon
> detection of the WR 1PPS rising edge. The SDR input is connected to the
> 1PPS to assess that the record always stays at the same time with
> respect to the 1PPS and it does. I record a known number of samples in
> file.bin at a known datarate (5 MS/s) and look at the timestamp of the
> last file access as the last field of stat -c %y file.bin, and I try
> and analyze the factional part of this timestamp.

How exactly you are writing to that file? Any buffering or fsync()?

I think the modified timestamp of a file is captured by the kernel
when it's updating the inode. The system clock might be precise to few
tens of nanoseconds, but that update of the inode certainly cannot be
timed to such precision.

-- 
Miroslav Lichvar

Re: [Linuxptp-users] sawtooth from linux timestamp?

[jmfriedt <jmf...@fe...>]

On Mon, 24 Jul 2023 09:17:10 +0200
Miroslav Lichvar <mli...@re...> wrote:

> On Sat, Jul 22, 2023 at 04:12:32PM +0200, jmfriedt wrote:
> > To do this I am connecting an Ettus Research X310 software defined
> > radio to the CM4 and configuring to start collecting data upon
> > detection of the WR 1PPS rising edge. The SDR input is connected to
> > the 1PPS to assess that the record always stays at the same time
> > with respect to the 1PPS and it does. I record a known number of
> > samples in file.bin at a known datarate (5 MS/s) and look at the
> > timestamp of the last file access as the last field of stat -c %y
> > file.bin, and I try and analyze the factional part of this
> > timestamp.
> 
> How exactly you are writing to that file? Any buffering or fsync()?
> 
> I think the modified timestamp of a file is captured by the kernel
> when it's updating the inode. The system clock might be precise to few
> tens of nanoseconds, but that update of the inode certainly cannot be
> timed to such precision.

This is exactly what I am interested in learning: the impact of the
operating system and the filesystem on the timestamping capability at
the userspace level. The file storage is handled at a very high level
by a Python script generated by GNU Radio Companion using a File Sink
with unbuffered(False) so indeed there is buffering occuring, thank you
for pointing that out.

Attached is the result whan the CPU clock is generated by a synthesizer
controlled by the GM reference. In tihs case the system clock v.s PHC
clock offset is cancelled and I am only left with what I believe to be
system jitter, which I calculate as about 3 us after removing outliers.

So I would now like to try and synchonize the CPU clock on the PHC
information. However I fail to identify the time (frequency) offset
without the external truth of the SDR records. I don't understand the
"freq" field of ptp4l but I believe this is related to the PHY
oscillator, not the CPU oscillator as this field is not changing
significantly when I change the CPU frequency. Also I try to read
testptp -k 10 but that too I cannot understand: since phc2sys is
constantly updating sys with phc I believe, how come that testptp -k
tells me system/phc clock time offset is 37 s? Furthermore this
quantity also does not represent the frequency offset of the CPU
driving oscillator wrt its nominal frequency. I tried reading the
various outputs of pmc but since this is the PTP management part,
probably it will not relate to system time. So I fail to identify an
indicator of system frequency vs PHC frequency difference that would
allow me to steer the CPU clock to its nominal frequency.

Thank you, Jean-Michel

-- 
JM Friedt, FEMTO-ST Time & Frequency, 26 rue de l'Epitaphe, 25000
Besancon, France

Re: [Linuxptp-users] sawtooth from linux timestamp?

[jmfriedt <jmf...@fe...>]

> So I would now like to try and synchonize the CPU clock on the PHC

sorry for the typo, that should read "and synthonize the CPU clock",
Jean-Michel

Re: [Linuxptp-users] sawtooth from linux timestamp?

[jmfriedt <jmf...@fe...>]

I have just realized that the freq field in the /var/log/messages log
file of phc2sys is representative of the CPU (system clock) and should
provide the information needed for synthonization.

(and I now realize the 37 s from my previous post is UTC-TAI)

Thanks, Jean-Michel

-- 
JM Friedt, FEMTO-ST Time & Frequency, 26 rue de l'Epitaphe, 25000
Besancon, France