linuxptp-users — Open discussion on using the linuxptp software.

[Linuxptp-users] ts2phc master offset value periodically spikes

[Cole W. <ce....@li...>]

Hello,

I'm looking for some assistance getting to the root of an issue where ts2phc will be stable for a period of time and then experience a spike in the master offset value. The debug logs show a few interesting things and I'm looking for input on what steps to take next.

My setup:

linuxptp version 3.1.1

I have two NICs using the Intel ICE driver. NIC0 has an integrated GNSS module and I am using ts2phc to sync them.

ethtool -i enp81s0f0
driver: ice
version: 1.7.16
firmware-version: 3.10 0x80009be5 1.3086.0
expansion-rom-version:
bus-info: 0000:51:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes


tsphc config - the pulsewidth and extts_polarity values are taken from the user guide for the NICs.

[global]
leapfile /usr/share/zoneinfo/leap-seconds.list
logging_level 7
ts2phc.nmea_serialport /dev/ttyGNSS_5100_0
ts2phc.pulsewidth 100000000

[enp81s0f0]
ts2phc.extts_polarity rising

[enp138s0f0]
ts2phc.extts_polarity rising



Log sample:

2022-03-15T04:50:26.000 controller-0 ts2phc: debug [1036434.421] nmea delay: 379832699 ns
2022-03-15T04:50:26.000 controller-0 ts2phc: debug [1036434.422] enp81s0f0 extts index 0 at 1647319864.000000000 corr 0 src 1647319864.620189416 diff 0
2022-03-15T04:50:26.000 controller-0 ts2phc: debug [1036434.422] enp81s0f0 master offset          0 s2 freq      +0
2022-03-15T04:50:26.000 controller-0 ts2phc: debug [1036434.422] nmea delay: 379832699 ns
2022-03-15T04:50:26.000 controller-0 ts2phc: debug [1036434.422] enp138s0f0 extts index 0 at 1647319863.999999999 corr 0 src 1647319864.620839770 diff -1
2022-03-15T04:50:26.000 controller-0 ts2phc: debug [1036434.422] enp138s0f0 master offset         -1 s2 freq      +1
2022-03-15T04:50:27.000 controller-0 ts2phc: debug [1036435.421] nmea delay: 319026253 ns
2022-03-15T04:50:27.000 controller-0 ts2phc: debug [1036435.422] enp81s0f0 extts index 0 at 1647319865.000000000 corr 0 src 1647319865.680996587 diff 0
2022-03-15T04:50:27.000 controller-0 ts2phc: debug [1036435.422] enp81s0f0 master offset          0 s2 freq      +0
2022-03-15T04:50:27.000 controller-0 ts2phc: debug [1036435.422] nmea delay: 319026253 ns
2022-03-15T04:50:27.000 controller-0 ts2phc: debug [1036435.422] enp138s0f0 extts index 0 at 1647319865.000000000 corr 0 src 1647319865.681640151 diff 0
2022-03-15T04:50:27.000 controller-0 ts2phc: debug [1036435.422] enp138s0f0 master offset          0 s2 freq      +1
2022-03-15T04:50:28.000 controller-0 ts2phc: debug [1036436.421] nmea delay: 314155539 ns
2022-03-15T04:50:28.000 controller-0 ts2phc: debug [1036436.422] enp81s0f0 extts index 0 at 1647319866.000000000 corr 0 src 1647319866.685874828 diff 0
2022-03-15T04:50:28.000 controller-0 ts2phc: debug [1036436.422] enp81s0f0 master offset          0 s2 freq      +0
2022-03-15T04:50:28.000 controller-0 ts2phc: debug [1036436.422] nmea delay: 314155539 ns
2022-03-15T04:50:28.000 controller-0 ts2phc: debug [1036436.422] enp138s0f0 extts index 0 at 1647319866.000000001 corr 0 src 1647319866.686513484 diff 1
2022-03-15T04:50:28.000 controller-0 ts2phc: debug [1036436.422] enp138s0f0 master offset          1 s2 freq      +2
2022-03-15T04:50:29.000 controller-0 ts2phc: debug [1036437.422] nmea delay: 790744992 ns

# Things go weird here
# The src value appears to be behind the previous one?
2022-03-15T04:50:29.000 controller-0 ts2phc: debug [1036437.422] enp138s0f0 extts index 0 at 1647319866.999999999 corr 0 src 1647319866.209929593 diff 999999999
2022-03-15T04:50:29.000 controller-0 ts2phc: debug [1036437.422] enp138s0f0 master offset  999999999 s2 freq +900000000
2022-03-15T04:50:30.000 controller-0 ts2phc: debug [1036437.434] nmea delay: 790744992 ns
2022-03-15T04:50:30.000 controller-0 ts2phc: debug [1036437.434] enp81s0f0 extts index 0 at 1647319867.000000000 corr 0 src 1647319866.221685020 diff 1000000000
2022-03-15T04:50:30.000 controller-0 ts2phc: debug [1036437.434] enp81s0f0 master offset 1000000000 s2 freq +900000000
2022-03-15T04:50:30.000 controller-0 ts2phc: debug [1036438.351] nmea delay: 234127497 ns
2022-03-15T04:50:30.000 controller-0 ts2phc: debug [1036438.351] enp81s0f0 extts index 0 at 1647319867.111604645 corr 0 src 1647319868.695211166 diff -888395355
2022-03-15T04:50:30.000 controller-0 ts2phc: debug [1036438.351] enp81s0f0 master offset -888395355 s2 freq -888395355
2022-03-15T04:50:30.000 controller-0 ts2phc: debug [1036438.351] nmea delay: 234127497 ns
2022-03-15T04:50:30.000 controller-0 ts2phc: debug [1036438.351] enp138s0f0 extts index 0 at 1647319867.100432414 corr 0 src 1647319868.695795473 diff -899567586
2022-03-15T04:50:30.000 controller-0 ts2phc: debug [1036438.351] enp138s0f0 master offset -899567586 s2 freq -899567584



The offset eventually stabilizes and carries on for a while before this symptom occurs again.
I have made sure that ntpd is disabled and I am not running any other ptp services.



Looking at the ts2phc code, it appears to me that the behaviour occurs in ts2phc_slave.c    ts2phc_slave_offset() when this check isn't true.

if (source_ts.tv_nsec > 500000000) {
    source_ts.tv_sec++;
}

So something is resulting in source_ts.tv_nsec being a bad value.


I would appreciate your thoughts on this issue. Should I be looking into the driver code, or is there more I can investigate in tsphc?

Let me know if additional log output would be useful.


Thanks for your help,

Cole

Re: [Linuxptp-users] Config section for /var/log/ptp4lro creates an additional port

[Martin P. <pec...@fe...>]

>>> You can make two global options, one for each flavor.
>> I'm sorry, I don't understand your suggestion. Can you explain in a bit more
>> detail, please?
> Right now we have:
>
> uds_address	/var/run/ptp4l
> uds_ro_address	/var/run/ptp4lro
>
> You could add:
>
> uds_address_filemode	0660
> uds_ro_address_filemode	0666
>
> Hm?

That would be ideal, but I'm out of ideas with the implementation. The 
file mode setting is needed in uds.c, but the uds_open() function has no 
flag telling it whether it is RW or RO (or some other?) UDS port.

Maybe a little bit hacky but working solution would be to compare name 
and uds_path. For the RO port, name is /var/run/ptp4lro and uds_path is 
/var/run/ptp4l. But I can imagine this can be pretty fragile detection.

Re: [Linuxptp-users] Config section for /var/log/ptp4lro creates an additional port

[Richard C. <ric...@gm...>]

On Thu, Mar 31, 2022 at 10:03:10AM +0200, Martin Pecka wrote:
> 
> > On Wed, Mar 30, 2022 at 04:57:14PM +0200, Martin Pecka wrote:
> > > That works, but it loses the beauty of having a "public" RO port and a
> > > root-only RW port.
> > You can make two global options, one for each flavor.
> 
> I'm sorry, I don't understand your suggestion. Can you explain in a bit more
> detail, please?

Right now we have:

uds_address	/var/run/ptp4l
uds_ro_address	/var/run/ptp4lro

You could add:

uds_address_filemode	0660
uds_ro_address_filemode	0666

Hm?

Thanks,
Richard

Re: [Linuxptp-users] Config section for /var/log/ptp4lro creates an additional port

[Martin P. <pec...@fe...>]

> On Wed, Mar 30, 2022 at 04:57:14PM +0200, Martin Pecka wrote:
>> That works, but it loses the beauty of having a "public" RO port and a
>> root-only RW port.
> You can make two global options, one for each flavor.

I'm sorry, I don't understand your suggestion. Can you explain in a bit 
more detail, please?

Thank you

Martin

Re: [Linuxptp-users] Config section for /var/log/ptp4lro creates an additional port

[Richard C. <ric...@gm...>]

On Wed, Mar 30, 2022 at 04:57:14PM +0200, Martin Pecka wrote:
> That works, but it loses the beauty of having a "public" RO port and a
> root-only RW port.

You can make two global options, one for each flavor.

Thanks,
Richard

Re: [Linuxptp-users] Config section for /var/log/ptp4lro creates an additional port

[Martin P. <pec...@fe...>]

> Make it a global option and not a port specific option.

That works, but it loses the beauty of having a "public" RO port and a 
root-only RW port.

Martin

Re: [Linuxptp-users] Config section for /var/log/ptp4lro creates an additional port

[Richard C. <ric...@gm...>]

On Wed, Mar 30, 2022 at 01:10:23PM +0200, Martin Pecka wrote:
> 
> > Right, the config sections are for network interfaces only.
> > 
> > > Is there a way to configure the RO socket without having this side-effect?
> > What are you trying to accomplish?
> 
> That goes along with my proposal (on linuxptp-devel) to allow specifying
> file mode for the UDS sockets. Ideally, I'd set 0660 to the RW socket and
> 0666 to the RO socket. Unfortunately, when the socket is being created, it
> doesn't have the information whether it is the RW or RO socket, it knows
> only its name. So it doesn't seem possible to define e.g. two config options
> file_mode_rw and file_mode_ro.

Make it a global option and not a port specific option. 

(The UDS interfaces are not PTP ports, but they are implemented as
ports for the sake of code simplicity.)

Thanks,
Richard

Re: [Linuxptp-users] Config section for /var/log/ptp4lro creates an additional port

[Martin P. <pec...@fe...>]

> Right, the config sections are for network interfaces only.
>
>> Is there a way to configure the RO socket without having this side-effect?
> What are you trying to accomplish?

That goes along with my proposal (on linuxptp-devel) to allow specifying 
file mode for the UDS sockets. Ideally, I'd set 0660 to the RW socket 
and 0666 to the RO socket. Unfortunately, when the socket is being 
created, it doesn't have the information whether it is the RW or RO 
socket, it knows only its name. So it doesn't seem possible to define 
e.g. two config options file_mode_rw and file_mode_ro.

Thanks,

Martin

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Miroslav L. <mli...@re...>]

On Wed, Mar 30, 2022 at 01:41:18AM +0300, Vladimir Oltean wrote:
> I think your comparison between NTP and PTP is a bit flawed because they
> serve different purposes, PTP being there for 'precise' synchronization
> rather than following the time of day.

I don't think they are that different. But even if they were, I don't
see why it would matter for the initialization of the PHC.

PHCs are not used only for PTP. There are other protocols. It's just a
confusing naming with the P in PHC standing for PTP.

> This is true even to the extreme. There is a non-negligible amount of
> systems that use PTP without any need whatsoever to track UTC, where
> just local machine-to-machine sync is required. Whereas CLOCK_REALTIME
> represents the machine's best guess of the current time of day.

The same thing can be done with NTP. Isolated networks with no
reference clocks are a common use case.

> You then use the purpose for which NTP exists to justify why the kernel
> should make a PTP timer behave exactly like it.

The sanity limit is not specific to a protocol. It could be
implemented in ptp4l and it would make it more resilient to some
failures.

> I admit I haven't studied PTP security to any extent, and I've come up
> empty handed while searching for "certificate" in my copy of IEEE 1588.

If NTS-KE is used in PTP, it will need to work with certifcates. Here
is a draft:
https://datatracker.ietf.org/doc/draft-langer-ntp-nts-for-ptp/

> But I wouldn't expect that ptp4l would use the PHC time to check
> validity of certificates, for the simple reason that it has no reliable
> guarantee that the PHC tracks the time of day even broadly, and that was
> never a requirement it had.

With how things currently are I agree that it would need to use the
system clock, but it would be nice if it could follow the original
idea of working with just one clock.

> As for the panic threshold, again I admit deep-seated ignorance on the
> topic, but my thinking is that maybe there is a reason why this threshold
> is part of the actual RFC 5905 spec, while it has no direct equivalent
> in IEEE 1588. In my view, this could be explained in the same note:
> with PTP there is no expectation that the local clock tracks a particular
> time source before it starts tracking the GM selected by BMCA.
> Why do you think this is?

The PTP spec doesn't say anything about clock control, so I'd not
expect any suggestions about panic or other thresholds there.

> To summarize, my belief is that the individual kernel drivers with good
> intentions that write the current CLOCK_REALTIME value into the PHC were
> written with the implicit assumption that user space would want to do
> that later anyway. I think it is more important that the kernel consistently
> enforces the message to users that the initial PHC time is unreliable
> for the PTP protocol's purpose, rather than unreliably tries to bring
> the PHC time closer to an imaginary target.

I don't agree with that, but it's not a big deal for me.

I'm just trying to point out that the magnitude of an acceptable clock
error depends on the context. A clock being synchronized is not a
boolean. An initial value coming from RTC->system clock->PHC can be
perfectly fine in some use cases.

-- 
Miroslav Lichvar

Re: [Linuxptp-users] Config section for /var/log/ptp4lro creates an additional port

[Richard C. <ric...@gm...>]

On Tue, Mar 29, 2022 at 09:29:06PM +0200, Martin Pecka wrote:
> I added the following to my config to set some properties of the read-only
> UDS port:
> 
> [/var/run/ptp4lro]
> # options
> 
> However, when I start ptp4l, it creates a superfluous socket that brings in
> many problems:

Right, the config sections are for network interfaces only.

> Is there a way to configure the RO socket without having this side-effect?

What are you trying to accomplish?

Thanks,
Richard

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Vladimir O. <ol...@gm...>]

Hi Miroslav,

On Tue, Mar 29, 2022 at 09:10:59AM +0200, Miroslav Lichvar wrote:
> On Fri, Mar 25, 2022 at 02:45:04AM +0200, Vladimir Oltean wrote:
> > In general, we all seem to agree that the initial PTP time is largely
> > irrelevant until you start looking at it for some particular reason
> > (debugging). Or at least, almost everyone seems to agree. I remember
> > Miroslav Lichvar argued that an initial offset of 37 seconds is somehow
> > better than an initial offset from 1970 from 2022 because of the smaller
> > clock jump? I admit I didn't quite get that.
> 
> The way I see it is that an offset of 37 seconds is about 44 million
> times better than an offset of 52 years. A smaller offset is easier to
> comprehend for humans. It allows tighter sanity checks to be enabled.
> 
> For example, it would work with the default panic threshold of 1000
> seconds in ntpd. The fact that ptp4l doesn't have such a sanity check
> doesn't mean that nobody cares about things like that. If a TLS-based
> authentication mechanism is implemented in ptp4l, it won't be able to
> use the PTP clock to check validify of certificates.
> 
> I don't see a reason why a PHC should intentionally be set to
> completely bogus time. Clocks are never perfectly accurate. It's
> always about minimizing the error. If the network drivers can improve
> the initial error by 8 orders of magnitude simply by starting at
> the current system time, then I think that is what should be done.
> 
> The system clock is set from the RTC, even though RTC is not
> synchronized to anything. If you turn off your computer for a month,
> you will likely start with an offset similar to those 37 seconds. I
> don't remember seeing anyone proposing that we should stop using RTC
> for the initial setting because the clock will be corrected later by
> NTP/PTP.

My opinion below may be wrong partly due to poor judgement and partly
due to ignorance, and I don't mean to second guess you. My knowledge of
the details of NTP is sub-par even as a user, yet I'm still saying what
I'm about to say, hoping that Cunningham's law will do its job and make
me learn something.

I think your comparison between NTP and PTP is a bit flawed because they
serve different purposes, PTP being there for 'precise' synchronization
rather than following the time of day.

This is true even to the extreme. There is a non-negligible amount of
systems that use PTP without any need whatsoever to track UTC, where
just local machine-to-machine sync is required. Whereas CLOCK_REALTIME
represents the machine's best guess of the current time of day.

You then use the purpose for which NTP exists to justify why the kernel
should make a PTP timer behave exactly like it.

I admit I haven't studied PTP security to any extent, and I've come up
empty handed while searching for "certificate" in my copy of IEEE 1588.
But I wouldn't expect that ptp4l would use the PHC time to check
validity of certificates, for the simple reason that it has no reliable
guarantee that the PHC tracks the time of day even broadly, and that was
never a requirement it had.

As for the panic threshold, again I admit deep-seated ignorance on the
topic, but my thinking is that maybe there is a reason why this threshold
is part of the actual RFC 5905 spec, while it has no direct equivalent
in IEEE 1588. In my view, this could be explained in the same note:
with PTP there is no expectation that the local clock tracks a particular
time source before it starts tracking the GM selected by BMCA.
Why do you think this is?

To summarize, my belief is that the individual kernel drivers with good
intentions that write the current CLOCK_REALTIME value into the PHC were
written with the implicit assumption that user space would want to do
that later anyway. I think it is more important that the kernel consistently
enforces the message to users that the initial PHC time is unreliable
for the PTP protocol's purpose, rather than unreliably tries to bring
the PHC time closer to an imaginary target.

I suppose you could counter my argument by insisting more on the idea
that "an offset of 37 seconds is about 44 million times better than an
offset of 52 years".

[Linuxptp-users] Config section for /var/log/ptp4lro creates an additional port

[Martin P. <pec...@fe...>]

I added the following to my config to set some properties of the 
read-only UDS port:

[/var/run/ptp4lro]
# options

However, when I start ptp4l, it creates a superfluous socket that brings 
in many problems:

ptp4l[215.643]: ioctl SIOCETHTOOL failed: No such device
ptp4l[215.752]: port 1 (enp2s0f0): INITIALIZING to SLAVE on INIT_COMPLETE
ptp4l[215.752]: port 2 (/var/run/ptp4lro): INITIALIZING to LISTENING on 
INIT_COMPLETE
ptp4l[215.753]: port 0 (/var/run/ptp4l): INITIALIZING to LISTENING on 
INIT_COMPLETE
ptp4l[215.753]: port 0 (/var/run/ptp4lro): INITIALIZING to LISTENING on 
INIT_COMPLETE

port2 is what I normally don't have.

Is there a way to configure the RO socket without having this side-effect?

Thanks for help,

Martin

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Chris C. <680...@po...>]

> In the Master I didn't use phc2sys in my first tests,
> and I was getting -300ms latency values between
> the two setups.

That is a different issue than first described.  I was referring to the
problem of the date from the master being set to 2018 rather than the
current date.
I do not remember seeing a description of how the date from your GPS
receiver was transmitted to the system, but assuming that the host
processor is handling GPS information through ntpd or chronyd then you
need to get the system date set into the NIC PTP hardware clock for PTP to
utilize the set system time.

-- 
Chris Caudle

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Miroslav L. <mli...@re...>]

On Fri, Mar 25, 2022 at 02:45:04AM +0200, Vladimir Oltean wrote:
> In general, we all seem to agree that the initial PTP time is largely
> irrelevant until you start looking at it for some particular reason
> (debugging). Or at least, almost everyone seems to agree. I remember
> Miroslav Lichvar argued that an initial offset of 37 seconds is somehow
> better than an initial offset from 1970 from 2022 because of the smaller
> clock jump? I admit I didn't quite get that.

The way I see it is that an offset of 37 seconds is about 44 million
times better than an offset of 52 years. A smaller offset is easier to
comprehend for humans. It allows tighter sanity checks to be enabled.

For example, it would work with the default panic threshold of 1000
seconds in ntpd. The fact that ptp4l doesn't have such a sanity check
doesn't mean that nobody cares about things like that. If a TLS-based
authentication mechanism is implemented in ptp4l, it won't be able to
use the PTP clock to check validify of certificates.

I don't see a reason why a PHC should intentionally be set to
completely bogus time. Clocks are never perfectly accurate. It's
always about minimizing the error. If the network drivers can improve
the initial error by 8 orders of magnitude simply by starting at
the current system time, then I think that is what should be done.

The system clock is set from the RTC, even though RTC is not
synchronized to anything. If you turn off your computer for a month,
you will likely start with an offset similar to those 37 seconds. I
don't remember seeing anyone proposing that we should stop using RTC
for the initial setting because the clock will be corrected later by
NTP/PTP.

-- 
Miroslav Lichvar

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Keller, J. E <jac...@in...>]

> -----Original Message-----
> From: Vladimir Oltean <ol...@gm...>
> Sent: Thursday, March 24, 2022 6:13 PM
> To: Keller, Jacob E <jac...@in...>
> Cc: ch...@ch...; lin...@li...
> Subject: Re: [Linuxptp-users] phc2sys not synchr. to right time and date
> 
> On Fri, Mar 25, 2022 at 03:00:40AM +0200, Vladimir Oltean wrote:
> > On Fri, Mar 25, 2022 at 12:52:02AM +0000, Keller, Jacob E wrote:
> > > > -----Original Message-----
> > > > From: Vladimir Oltean <ol...@gm...>
> > > > Sent: Thursday, March 24, 2022 5:45 PM
> > > > To: Keller, Jacob E <jac...@in...>
> > > > Cc: ch...@ch...; lin...@li...
> > > > Subject: Re: [Linuxptp-users] phc2sys not synchr. to right time and date
> > > >
> > > > On Fri, Mar 25, 2022 at 12:27:53AM +0000, Keller, Jacob E wrote:
> > > > > > (there might be other reasons why his setup is working, but)
> > > > > > Some kernel Ethernet drivers have a bad habit of writing the current
> > > > > > (UTC) system time (derived from the battery-backed RTC) into the PHC
> > > > > > current (TAI) time at boot. You're off by 37 seconds, but hey, at least
> > > > > > it's 2022, so the time _must_ be right!
> > > > > >
> https://elixir.bootlin.com/linux/latest/source/drivers/ptp/ptp_qoriq.c#L504
> > > > >
> > > > > If there's a strong recommendation against doing this, should we go
> > > > > fix the in-tree drivers to avoid doing so?
> > > >
> > > > Everybody seems to have their own opinion on the topic.
> > > >
> > > > At one point I submitted a patch to change the ptp_qoriq driver (shared
> > > > by a bunch of Freescale/NXP Ethernet drivers) to stop doing this,
> > > > Richard said it's not worth it to make the change.
> > > >
> > >
> > > I think most of the Intel parts also initialize the time at driver load.
> > >
> > > > Then Sergey Organov came with a FEC driver bug which made the FEC emit
> a
> > > > duplicate TX timestamp each time the attached PHY also emitted a TX
> > > > timestamp. He said "hey, I would have figured out much quicker what was
> > > > going on, if the FEC didn't have the same rough time base as the PHY,
> > > > despite me not attempting to synchronize it to anything!". Richard
> > > > became more responsive to this argument and, as far as I see, is now
> > > > suggesting driver writers to let the PTP time be what it is when the
> > > > driver probes, instead of setting it to a basically different random value.
> > > >
> > >
> > > Leaving whatever the PTP time is alone? This typically would have it
> > > be zero in Intel hardware, which would get reported as offset since
> > > 1970..
> >
> > Yes, that would be the idea.
> >
> > > > As for myself, I had an issue with the stmmac driver where it was doing
> > > > this PTP time reinitialization with the system time, but not only at
> > > > boot, but rather each time the timestamping ioctl was sent by user space
> > > > (i.e. each time I started ptp4l). That was actively bothering me,
> > > > because I was also running phc2sys in the background, and the offset
> > > > kept resetting to 37 every time I ran a test application which enabled
> > > > timestamping, so I had to change that with a kernel patch.
> > >
> > > Yes that sounds like absolutely horrid approach. Definitely shouldn't
> > > get setting the time outside of initialization.
> > >
> > > >
> > > > I expect that if I return with the ptp_qoriq patch now, after all the
> > > > discussions that took place in the meantime, it would be accepted,
> > > > at least for the sake of uniformity if nothing else.
> > > >
> > > > In general, we all seem to agree that the initial PTP time is largely
> > > > irrelevant until you start looking at it for some particular reason
> > > > (debugging). Or at least, almost everyone seems to agree. I remember
> > > > Miroslav Lichvar argued that an initial offset of 37 seconds is somehow
> > > > better than an initial offset from 1970 from 2022 because of the smaller
> > > > clock jump? I admit I didn't quite get that.
> > >
> > > Smaller offsets are easier to correct. At least in some of the Intel
> > > hardware it can do atomic corrections for small offsets, but cannot do
> > > so for large offsets.
> >
> > At least in the NXP world I'm living in, this problem doesn't really exist.
> > A clock step is the same kind of operation regardless of offset it needs
> > to correct (either non-atomic/read-modify-write: ptp_qoriq, or atomic:
> > sja1105). As a side note I find it a bit strange to implement an atomic
> > clock step method in hardware but limit the offset...
> >
> > I realize this isn't ideal (because kernel and user space isn't updated
> > in lockstep), but maybe to account for both cases, ptp4l could make the
> > initial servo's clockadj_step() be a clockadj_set_time()?
> 
> What are these drivers you mention? ice by any means, where I see the
> atomic offset is +/- S32_MAX? If my back of the napkin calculations are
> right, 2^32 nanoseconds are ~4.3 seconds, so an initial offset of 37
> seconds is outside of the atomic range regardless, is it not?

I was referring to a couple of devices, but ice specifically, yea. I hadn't re-done the napkin math to figure out what offset 4.3 seconds sounds right though.

A bunch of other Intel hardware has even smaller atomic adjustments, and we never even bothered implementing the software for it. They're like "you can correct a couple nanoseconds". The limit is so small it makes more sense to just use frequency syntonization.

I tried to explain this and get larger atomic offsets, but it was not listened to.

We could alternatively replace things with timecounters instead of directly modifying the hardware counters. This works but has knock-on effects when dealing with hardware registers for input output pins. You have to basically reverse the timecounter corrections and that can be a bit messy. It also made multi-port situations like ice difficult: in hardware all physical functions share the underlying clock, but they use separate driver instances because its still a multifunction PCIe device.. (ugh..)

Thanks,
Jake

Re: [Linuxptp-users] Alternate ECU to became slave when primary ECU is in active

[Richard C. <ric...@gm...>]

On Mon, Mar 28, 2022 at 12:05:08PM +0530, Raj wrote:
> Does anyone have any suggestions for the below query !.

Don't use masteronly.

Just set the priority1/2 fields of the three units appropriately.

For example:

Unit  priority1
---------------
GM-1  126
GM-2  127
OC    128

Thanks,
Richard

Re: [Linuxptp-users] Alternate ECU to became slave when primary ECU is in active

[Raj <mai...@gm...>]

Hi Team,

Does anyone have any suggestions for the below query !.

Thanks in Advance.

On Fri, Mar 25, 2022 at 8:12 PM Raj <mai...@gm...> wrote:

> Hi Team,
>
> We are currently using BMCA where we have one Primary ECU(GM), An
> Alternate ECU(Passive state when Primary is in operation) and one Slave
> ECU. When Primary ECU is down , Alternate ECU will became active and will
> be grand master for the Slave ECU.
>
> Primary ECU configured as master(GM) with below parameter:
>
> priority1       240
> BMCA          ptp
> masteronly   1      ... etc
>
> Alternate ECU (Passive when primary ECU is active) configured with below
> parameter :
>
>
> priority1        248
> BMCA           ptp
> clockclass     120 ... etc
>
> Question is when Primary ECU is active and in GM role , Can we configured
> Alternate ECU as slave. And When Primary goes down Alternate will be
> active(GM) ?
>
> If it is possible to configure Alterante ECU as Slave instead of PASSIVE
> state when Primary ECU is Grand Master, Could you please let us know which
> ptp configuration parameter can be used to make Alternate as Slave ?
>
>
> Thanks,
> Raj
>
>
>

-- 
With Regards
Raj  Kishore  Chhatoi.
Mob: +91-8608747611

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Richard C. <ric...@gm...>]

On Sat, Mar 26, 2022 at 10:25:43AM +0000, Federico Murciano wrote:

> Do you have concrete instrucctions that I could try in master and
> slave in order to correct this? Do you think it is possible that due
> to the setups where I just plug in the device to electricity and
> other setup that is plugged for longer time can cause the problem?

IIRC you have one ptp4l master and two slaves, statically configured.
That will synchronize the clocks built into your MACs.

If you want the Linux system time also synchronized, then run the
phc2sys program as well.  You can configure it statically (run phc2sys -h
and read the man page) or automatically, like this:

    phc2sys -a -r -r

on each of the three computers.

HTH,
Richard

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Federico M. <Fed...@da...>]

Hi Chris,

Sorry that I didn't directly answer to your question. In the Master I didn't use phc2sys in my first tests, and I was getting -300ms latency values between the two setups. After you told me to also use phc2sys in the Master, I tried that but still I was getting this values.

I am using the next command in the master and also in the slave after ptp4l is running. The values are still negative latencies in ms which does not make sense.

sudo phc2sys -s enp0s31f6 -O 0 -m

Do you have concrete instrucctions that I could try in master and slave in order to correct this? Do you think it is possible that due to the setups where I just plug in the device to electricity and other setup that is plugged for longer time can cause the problem?

Thanks a lot for your support,

Best regards,
Federico Murciano

M.Sc. Elektrotechnik - Telecommunications Engineer
fed...@da...
Tel. +49 (0) 30/314 -74 025 / Mob. +49 157 8642 4850
Future Communication Networks
DAI-Labor
Technische Universität Berlin
Fakultät IV – Elektrotechnik & Informatik Sekretariat TEL 14 Ernst-Reuter-Platz 7
10587 Berlin, Germany
http://www.dai-labor.de/
DAI-Labor - Distributed Artificial Intelligence Laboratory Chief Executive Director: Prof. Dr. Dr. h.c. Sahin Albayrak

________________________________________
Von: Chris Caudle <680...@po...>
Gesendet: Donnerstag, 24. März 2022 21:04
An: lin...@li...
Betreff: Re: [Linuxptp-users] phc2sys not synchr. to right time and date

> I can also see in the Master devices this difference in its clocks:
> from 32 seconds to 31 in the RTC time:
> I did hwclock -w and still get this values

None of that has anything to do with the PTP hardware clock (PHC) in the
network controller.
The system clock runs on the system processor, the PTP clock runs in the
NIC.  If you want the PHC to have the same time as system time, you have
to put the system time into PHC.
I specifically asked about phc2sys, and you replied with a response about
hwclock, which is completely unrelated to PHC, so I think you
misunderstood the clock topology in a standard PC type system running PTP.

--
Chris Caudle




_______________________________________________
Linuxptp-users mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxptp-users

[Linuxptp-users] Alternate ECU to became slave when primary ECU is in active

[Raj <mai...@gm...>]

Hi Team,

We are currently using BMCA where we have one Primary ECU(GM), An Alternate
ECU(Passive state when Primary is in operation) and one Slave ECU. When
Primary ECU is down , Alternate ECU will became active and will be grand
master for the Slave ECU.

Primary ECU configured as master(GM) with below parameter:

priority1       240
BMCA          ptp
masteronly   1      ... etc

Alternate ECU (Passive when primary ECU is active) configured with below
parameter :


priority1        248
BMCA           ptp
clockclass     120 ... etc

Question is when Primary ECU is active and in GM role , Can we configured
Alternate ECU as slave. And When Primary goes down Alternate will be
active(GM) ?

If it is possible to configure Alterante ECU as Slave instead of PASSIVE
state when Primary ECU is Grand Master, Could you please let us know which
ptp configuration parameter can be used to make Alternate as Slave ?


Thanks,
Raj

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Vladimir O. <ol...@gm...>]

On Fri, Mar 25, 2022 at 03:00:40AM +0200, Vladimir Oltean wrote:
> On Fri, Mar 25, 2022 at 12:52:02AM +0000, Keller, Jacob E wrote:
> > > -----Original Message-----
> > > From: Vladimir Oltean <ol...@gm...>
> > > Sent: Thursday, March 24, 2022 5:45 PM
> > > To: Keller, Jacob E <jac...@in...>
> > > Cc: ch...@ch...; lin...@li...
> > > Subject: Re: [Linuxptp-users] phc2sys not synchr. to right time and date
> > > 
> > > On Fri, Mar 25, 2022 at 12:27:53AM +0000, Keller, Jacob E wrote:
> > > > > (there might be other reasons why his setup is working, but)
> > > > > Some kernel Ethernet drivers have a bad habit of writing the current
> > > > > (UTC) system time (derived from the battery-backed RTC) into the PHC
> > > > > current (TAI) time at boot. You're off by 37 seconds, but hey, at least
> > > > > it's 2022, so the time _must_ be right!
> > > > > https://elixir.bootlin.com/linux/latest/source/drivers/ptp/ptp_qoriq.c#L504
> > > >
> > > > If there's a strong recommendation against doing this, should we go
> > > > fix the in-tree drivers to avoid doing so?
> > > 
> > > Everybody seems to have their own opinion on the topic.
> > > 
> > > At one point I submitted a patch to change the ptp_qoriq driver (shared
> > > by a bunch of Freescale/NXP Ethernet drivers) to stop doing this,
> > > Richard said it's not worth it to make the change.
> > > 
> > 
> > I think most of the Intel parts also initialize the time at driver load.
> > 
> > > Then Sergey Organov came with a FEC driver bug which made the FEC emit a
> > > duplicate TX timestamp each time the attached PHY also emitted a TX
> > > timestamp. He said "hey, I would have figured out much quicker what was
> > > going on, if the FEC didn't have the same rough time base as the PHY,
> > > despite me not attempting to synchronize it to anything!". Richard
> > > became more responsive to this argument and, as far as I see, is now
> > > suggesting driver writers to let the PTP time be what it is when the
> > > driver probes, instead of setting it to a basically different random value.
> > > 
> > 
> > Leaving whatever the PTP time is alone? This typically would have it
> > be zero in Intel hardware, which would get reported as offset since
> > 1970..
> 
> Yes, that would be the idea.
> 
> > > As for myself, I had an issue with the stmmac driver where it was doing
> > > this PTP time reinitialization with the system time, but not only at
> > > boot, but rather each time the timestamping ioctl was sent by user space
> > > (i.e. each time I started ptp4l). That was actively bothering me,
> > > because I was also running phc2sys in the background, and the offset
> > > kept resetting to 37 every time I ran a test application which enabled
> > > timestamping, so I had to change that with a kernel patch.
> > 
> > Yes that sounds like absolutely horrid approach. Definitely shouldn't
> > get setting the time outside of initialization.
> > 
> > > 
> > > I expect that if I return with the ptp_qoriq patch now, after all the
> > > discussions that took place in the meantime, it would be accepted,
> > > at least for the sake of uniformity if nothing else.
> > > 
> > > In general, we all seem to agree that the initial PTP time is largely
> > > irrelevant until you start looking at it for some particular reason
> > > (debugging). Or at least, almost everyone seems to agree. I remember
> > > Miroslav Lichvar argued that an initial offset of 37 seconds is somehow
> > > better than an initial offset from 1970 from 2022 because of the smaller
> > > clock jump? I admit I didn't quite get that.
> > 
> > Smaller offsets are easier to correct. At least in some of the Intel
> > hardware it can do atomic corrections for small offsets, but cannot do
> > so for large offsets.
> 
> At least in the NXP world I'm living in, this problem doesn't really exist.
> A clock step is the same kind of operation regardless of offset it needs
> to correct (either non-atomic/read-modify-write: ptp_qoriq, or atomic:
> sja1105). As a side note I find it a bit strange to implement an atomic
> clock step method in hardware but limit the offset...
> 
> I realize this isn't ideal (because kernel and user space isn't updated
> in lockstep), but maybe to account for both cases, ptp4l could make the
> initial servo's clockadj_step() be a clockadj_set_time()?

What are these drivers you mention? ice by any means, where I see the
atomic offset is +/- S32_MAX? If my back of the napkin calculations are
right, 2^32 nanoseconds are ~4.3 seconds, so an initial offset of 37
seconds is outside of the atomic range regardless, is it not?

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Vladimir O. <ol...@gm...>]

On Fri, Mar 25, 2022 at 12:52:02AM +0000, Keller, Jacob E wrote:
> > -----Original Message-----
> > From: Vladimir Oltean <ol...@gm...>
> > Sent: Thursday, March 24, 2022 5:45 PM
> > To: Keller, Jacob E <jac...@in...>
> > Cc: ch...@ch...; lin...@li...
> > Subject: Re: [Linuxptp-users] phc2sys not synchr. to right time and date
> > 
> > On Fri, Mar 25, 2022 at 12:27:53AM +0000, Keller, Jacob E wrote:
> > > > (there might be other reasons why his setup is working, but)
> > > > Some kernel Ethernet drivers have a bad habit of writing the current
> > > > (UTC) system time (derived from the battery-backed RTC) into the PHC
> > > > current (TAI) time at boot. You're off by 37 seconds, but hey, at least
> > > > it's 2022, so the time _must_ be right!
> > > > https://elixir.bootlin.com/linux/latest/source/drivers/ptp/ptp_qoriq.c#L504
> > >
> > > If there's a strong recommendation against doing this, should we go
> > > fix the in-tree drivers to avoid doing so?
> > 
> > Everybody seems to have their own opinion on the topic.
> > 
> > At one point I submitted a patch to change the ptp_qoriq driver (shared
> > by a bunch of Freescale/NXP Ethernet drivers) to stop doing this,
> > Richard said it's not worth it to make the change.
> > 
> 
> I think most of the Intel parts also initialize the time at driver load.
> 
> > Then Sergey Organov came with a FEC driver bug which made the FEC emit a
> > duplicate TX timestamp each time the attached PHY also emitted a TX
> > timestamp. He said "hey, I would have figured out much quicker what was
> > going on, if the FEC didn't have the same rough time base as the PHY,
> > despite me not attempting to synchronize it to anything!". Richard
> > became more responsive to this argument and, as far as I see, is now
> > suggesting driver writers to let the PTP time be what it is when the
> > driver probes, instead of setting it to a basically different random value.
> > 
> 
> Leaving whatever the PTP time is alone? This typically would have it
> be zero in Intel hardware, which would get reported as offset since
> 1970..

Yes, that would be the idea.

> > As for myself, I had an issue with the stmmac driver where it was doing
> > this PTP time reinitialization with the system time, but not only at
> > boot, but rather each time the timestamping ioctl was sent by user space
> > (i.e. each time I started ptp4l). That was actively bothering me,
> > because I was also running phc2sys in the background, and the offset
> > kept resetting to 37 every time I ran a test application which enabled
> > timestamping, so I had to change that with a kernel patch.
> 
> Yes that sounds like absolutely horrid approach. Definitely shouldn't
> get setting the time outside of initialization.
> 
> > 
> > I expect that if I return with the ptp_qoriq patch now, after all the
> > discussions that took place in the meantime, it would be accepted,
> > at least for the sake of uniformity if nothing else.
> > 
> > In general, we all seem to agree that the initial PTP time is largely
> > irrelevant until you start looking at it for some particular reason
> > (debugging). Or at least, almost everyone seems to agree. I remember
> > Miroslav Lichvar argued that an initial offset of 37 seconds is somehow
> > better than an initial offset from 1970 from 2022 because of the smaller
> > clock jump? I admit I didn't quite get that.
> 
> Smaller offsets are easier to correct. At least in some of the Intel
> hardware it can do atomic corrections for small offsets, but cannot do
> so for large offsets.

At least in the NXP world I'm living in, this problem doesn't really exist.
A clock step is the same kind of operation regardless of offset it needs
to correct (either non-atomic/read-modify-write: ptp_qoriq, or atomic:
sja1105). As a side note I find it a bit strange to implement an atomic
clock step method in hardware but limit the offset...

I realize this isn't ideal (because kernel and user space isn't updated
in lockstep), but maybe to account for both cases, ptp4l could make the
initial servo's clockadj_step() be a clockadj_set_time()?

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Keller, J. E <jac...@in...>]

> -----Original Message-----
> From: Vladimir Oltean <ol...@gm...>
> Sent: Thursday, March 24, 2022 5:45 PM
> To: Keller, Jacob E <jac...@in...>
> Cc: ch...@ch...; lin...@li...
> Subject: Re: [Linuxptp-users] phc2sys not synchr. to right time and date
> 
> On Fri, Mar 25, 2022 at 12:27:53AM +0000, Keller, Jacob E wrote:
> > > (there might be other reasons why his setup is working, but)
> > > Some kernel Ethernet drivers have a bad habit of writing the current
> > > (UTC) system time (derived from the battery-backed RTC) into the PHC
> > > current (TAI) time at boot. You're off by 37 seconds, but hey, at least
> > > it's 2022, so the time _must_ be right!
> > > https://elixir.bootlin.com/linux/latest/source/drivers/ptp/ptp_qoriq.c#L504
> >
> > If there's a strong recommendation against doing this, should we go
> > fix the in-tree drivers to avoid doing so?
> 
> Everybody seems to have their own opinion on the topic.
> 
> At one point I submitted a patch to change the ptp_qoriq driver (shared
> by a bunch of Freescale/NXP Ethernet drivers) to stop doing this,
> Richard said it's not worth it to make the change.
> 

I think most of the Intel parts also initialize the time at driver load.

> Then Sergey Organov came with a FEC driver bug which made the FEC emit a
> duplicate TX timestamp each time the attached PHY also emitted a TX
> timestamp. He said "hey, I would have figured out much quicker what was
> going on, if the FEC didn't have the same rough time base as the PHY,
> despite me not attempting to synchronize it to anything!". Richard
> became more responsive to this argument and, as far as I see, is now
> suggesting driver writers to let the PTP time be what it is when the
> driver probes, instead of setting it to a basically different random value.
> 

Leaving whatever the PTP time is alone? This typically would have it be zero in Intel hardware, which would get reported as offset since 1970..

> As for myself, I had an issue with the stmmac driver where it was doing
> this PTP time reinitialization with the system time, but not only at
> boot, but rather each time the timestamping ioctl was sent by user space
> (i.e. each time I started ptp4l). That was actively bothering me,
> because I was also running phc2sys in the background, and the offset
> kept resetting to 37 every time I ran a test application which enabled
> timestamping, so I had to change that with a kernel patch.

Yes that sounds like absolutely horrid approach. Definitely shouldn't get setting the time outside of initialization.

> 
> I expect that if I return with the ptp_qoriq patch now, after all the
> discussions that took place in the meantime, it would be accepted,
> at least for the sake of uniformity if nothing else.
> 
> In general, we all seem to agree that the initial PTP time is largely
> irrelevant until you start looking at it for some particular reason
> (debugging). Or at least, almost everyone seems to agree. I remember
> Miroslav Lichvar argued that an initial offset of 37 seconds is somehow
> better than an initial offset from 1970 from 2022 because of the smaller
> clock jump? I admit I didn't quite get that.

Smaller offsets are easier to correct. At least in some of the Intel hardware it can do atomic corrections for small offsets, but cannot do so for large offsets.

Thanks,
Jake

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Vladimir O. <ol...@gm...>]

On Fri, Mar 25, 2022 at 12:27:53AM +0000, Keller, Jacob E wrote:
> > (there might be other reasons why his setup is working, but)
> > Some kernel Ethernet drivers have a bad habit of writing the current
> > (UTC) system time (derived from the battery-backed RTC) into the PHC
> > current (TAI) time at boot. You're off by 37 seconds, but hey, at least
> > it's 2022, so the time _must_ be right!
> > https://elixir.bootlin.com/linux/latest/source/drivers/ptp/ptp_qoriq.c#L504
> 
> If there's a strong recommendation against doing this, should we go
> fix the in-tree drivers to avoid doing so?

Everybody seems to have their own opinion on the topic.

At one point I submitted a patch to change the ptp_qoriq driver (shared
by a bunch of Freescale/NXP Ethernet drivers) to stop doing this,
Richard said it's not worth it to make the change.

Then Sergey Organov came with a FEC driver bug which made the FEC emit a
duplicate TX timestamp each time the attached PHY also emitted a TX
timestamp. He said "hey, I would have figured out much quicker what was
going on, if the FEC didn't have the same rough time base as the PHY,
despite me not attempting to synchronize it to anything!". Richard
became more responsive to this argument and, as far as I see, is now
suggesting driver writers to let the PTP time be what it is when the
driver probes, instead of setting it to a basically different random value.

As for myself, I had an issue with the stmmac driver where it was doing
this PTP time reinitialization with the system time, but not only at
boot, but rather each time the timestamping ioctl was sent by user space
(i.e. each time I started ptp4l). That was actively bothering me,
because I was also running phc2sys in the background, and the offset
kept resetting to 37 every time I ran a test application which enabled
timestamping, so I had to change that with a kernel patch.

I expect that if I return with the ptp_qoriq patch now, after all the
discussions that took place in the meantime, it would be accepted,
at least for the sake of uniformity if nothing else.

In general, we all seem to agree that the initial PTP time is largely
irrelevant until you start looking at it for some particular reason
(debugging). Or at least, almost everyone seems to agree. I remember
Miroslav Lichvar argued that an initial offset of 37 seconds is somehow
better than an initial offset from 1970 from 2022 because of the smaller
clock jump? I admit I didn't quite get that.

Re: [Linuxptp-users] phc2sys not synchr. to right time and date

[Keller, J. E <jac...@in...>]

> -----Original Message-----
> From: Vladimir Oltean <ol...@gm...>
> Sent: Thursday, March 24, 2022 11:10 AM
> To: ch...@ch...
> Cc: lin...@li...
> Subject: Re: [Linuxptp-users] phc2sys not synchr. to right time and date
> 
> On Thu, Mar 24, 2022 at 09:14:08AM -0500, Chris Caudle wrote:
> > On Thu, March 24, 2022 5:02 am,
> > lin...@li... wrote:
> > > From: Federico Murciano <Fed...@da...>
> > > Subject: Re: [Linuxptp-users] phc2sys not synchr. to right time and
> > > 	date
> > > Message-ID: <164...@da...>
> >
> > > Thank you for your answer. Yes this is in the slave computer. In the
> > > master computer I only execute the following, which is working for my
> > > other setup (with replicated master and slave devices):
> > >
> > > cat ptp.cfg
> > > [global]
> > > tx_timestamp_timeout 10
> > > step_threshold	 1
> > >
> > > sudo ptp4l -A -i eth0 -m -f ptp.cfg
> >
> > Then what process is expected to set the time for the PHC?  That seems to
> > be a missing step in your configuration.  Perhaps the better question is
> > why does the other setup ever work?
> >
> > --
> > Chris Caudle
> 
> (there might be other reasons why his setup is working, but)
> Some kernel Ethernet drivers have a bad habit of writing the current
> (UTC) system time (derived from the battery-backed RTC) into the PHC
> current (TAI) time at boot. You're off by 37 seconds, but hey, at least
> it's 2022, so the time _must_ be right!
> https://elixir.bootlin.com/linux/latest/source/drivers/ptp/ptp_qoriq.c#L504
> 

If there's a strong recommendation against doing this, should we go fix the in-tree drivers to avoid doing so?

Thanks,
Jake

> 
> _______________________________________________
> Linuxptp-users mailing list
> Lin...@li...
> https://lists.sourceforge.net/lists/listinfo/linuxptp-users