linuxptp-users

[Linuxptp-users] PTP through a bridge interface

[Rob C. <RCo...@si...>]

Hi,

I'm trying to setup a device as the grand master clock, we using an imx6 dual lite which has ieee 1588 support in the enet controller.
This is connected to a 7-port switch (no ptp support), and this switch is configured via bridge interface br0.

What I had a problem with was receiving packets on the master (delay request) from a slave device, I noticed that ptp4l actually binds the socket to the interface specified from the command for example:

$ ptp4l -A -H -4 -I eth0

ptp4l will bind to eth0 and also joins the multicast group on eth0, In my case I must use eth0 as it has the ptp hardware support.

But to receive packets through the bridge interface I need to bind instead to br0 and join the multicast group on br0. From what I see after doing this I can successfully receive the slave's packets on the master device, does this change make sense for my circumstances?


Another thing was the Sync and Delay_Req messages had 0 in the timestamp values, I attached a capture for example.
Though when I run phc2sys on my slave the system clock does seem to get synchronized to the time on the master, which leads me to think everything is functioning OK?

I also attached logs of the commands from slave and master.

Thanks,
Rob C

Re: [Linuxptp-users] PTP through a bridge interface

[Richard C. <ric...@gm...>]

On Tue, Jun 05, 2018 at 02:37:36AM +0000, Rob Cornall wrote:
> I'm trying to setup a device as the grand master clock, we using an imx6 dual lite which has ieee 1588 support in the enet controller.
> This is connected to a 7-port switch (no ptp support), and this switch is configured via bridge interface br0.

What do you mean by "this is connected to a 7-port switch"?

Do you mean that:

1. the MAC on the imx6 is physically connected to a switch using an
   Ethernet cable?

2. or that the interface is part of a Linux software bridge, configured
   with "brctl add" ?

In the case of number 2, PTP won't work.

Thanks,
Richard

Re: [Linuxptp-users] PTP through a bridge interface

[Rob C. <RCo...@si...>]

I mean #2, a marvell (mv88e6xxx DSA) is used where the cpu port is connected the imx6 ethernet controller.

Yes It is setup as a linux software bridge via brctl.


We are aware that having a switch on the master will add some variation of latency, but is it not possible to send ptp packets and receive them through it?

Because it seems to be working, at least the master and slave are communicating and adjusting.


Thanks,
Rob C

________________________________
From: Richard Cochran <ric...@gm...>
Sent: Tuesday, June 5, 2018 6:55 AM
To: Rob Cornall
Cc: lin...@li...
Subject: Re: [Linuxptp-users] PTP through a bridge interface

On Tue, Jun 05, 2018 at 02:37:36AM +0000, Rob Cornall wrote:
> I'm trying to setup a device as the grand master clock, we using an imx6 dual lite which has ieee 1588 support in the enet controller.
> This is connected to a 7-port switch (no ptp support), and this switch is configured via bridge interface br0.

What do you mean by "this is connected to a 7-port switch"?

Do you mean that:

1. the MAC on the imx6 is physically connected to a switch using an
   Ethernet cable?

2. or that the interface is part of a Linux software bridge, configured
   with "brctl add" ?

In the case of number 2, PTP won't work.

Thanks,
Richard

Re: [Linuxptp-users] PTP through a bridge interface

[Richard C. <ric...@gm...>]

On Tue, Jun 05, 2018 at 02:40:18PM +0000, Rob Cornall wrote:
> I mean #2, a marvell (mv88e6xxx DSA) is used where the cpu port is connected the imx6 ethernet controller.
> 
> Yes It is setup as a linux software bridge via brctl.
> 
> 
> We are aware that having a switch on the master will add some variation of latency, but is it not possible to send ptp packets and receive them through it?

If you leave the switch alone, not as a DSA device, then you can use
eth0 as a PTP port.

Once you make eth0 the CPU port of a DSA device, you cannot use eth0
any more, and you can't use the br0 for PTP either.

So your choice are:

1. Don't use DSA for the Marvell and run ptp4l on eth0.

2. Use the PTP support for the Marvel DSA device (new in Linux kernel v4.17)
   and run ptp4l on the external ports.

HTH,
Richard

Re: [Linuxptp-users] PTP through a bridge interface

[Rob C. <RCo...@si...>]

Hi Richard, I really appreciate you answering my questions so quickly.

I attached my logs where I ran a patched version of ptp4l on the GM device (I also attached this patch), the patch simply binds the socket to br0 and joins the multicast group on the br0 interface instead of the eth0 interface supplied In the command.
Our routes go through br0 as expected in this situation:

$ ip ro.3 
224.0.0.0/4 dev br0  scope link
...

And our DSA is setup as a 'dumb' switch, all unknown DA frames are flooded out all ports.

Also our particular marvell switch does not have the ptp support, we are aware that some versions do.

When seeing the slave adjusting its ptp clock this leads me to think everything works okay with this patch? Is this something that could be added as an option to ptp4l and merged in the future to support linux sw bridges?

Thanks,
Rob

-----Original Message-----
From: Richard Cochran <ric...@gm...> 
Sent: Tuesday, June 5, 2018 9:14 AM
To: Rob Cornall <RCo...@si...>
Cc: lin...@li...
Subject: Re: [Linuxptp-users] PTP through a bridge interface

On Tue, Jun 05, 2018 at 02:40:18PM +0000, Rob Cornall wrote:
> I mean #2, a marvell (mv88e6xxx DSA) is used where the cpu port is connected the imx6 ethernet controller.
> 
> Yes It is setup as a linux software bridge via brctl.
> 
> 
> We are aware that having a switch on the master will add some variation of latency, but is it not possible to send ptp packets and receive them through it?

If you leave the switch alone, not as a DSA device, then you can use
eth0 as a PTP port.

Once you make eth0 the CPU port of a DSA device, you cannot use eth0 any more, and you can't use the br0 for PTP either.

So your choice are:

1. Don't use DSA for the Marvell and run ptp4l on eth0.

2. Use the PTP support for the Marvel DSA device (new in Linux kernel v4.17)
   and run ptp4l on the external ports.

HTH,
Richard

Re: [Linuxptp-users] PTP through a bridge interface

[Richard C. <ric...@gm...>]

On Tue, Jun 05, 2018 at 04:46:38PM +0000, Rob Cornall wrote:
> And our DSA is setup as a 'dumb' switch, all unknown DA frames are flooded out all ports.

In that case, just don't load the DSA driver and don't create a Linux
bridge device.
 
> When seeing the slave adjusting its ptp clock this leads me to think everything works okay with this patch? Is this something that could be added as an option to ptp4l and merged in the future to support linux sw bridges?

No, that will never work.

Consider the simple case where you take two PTP capable MACs and then
join them together into a bridge.

Thanks,
Richard

Re: [Linuxptp-users] PTP through a bridge interface

[Keller, J. E <jac...@in...>]

> -----Original Message-----
> From: Rob Cornall [mailto:RCo...@si...]
> Sent: Tuesday, June 05, 2018 9:47 AM
> To: Richard Cochran <ric...@gm...>
> Cc: lin...@li...
> Subject: Re: [Linuxptp-users] PTP through a bridge interface
> 
> Hi Richard, I really appreciate you answering my questions so quickly.
> 
> I attached my logs where I ran a patched version of ptp4l on the GM device (I also
> attached this patch), the patch simply binds the socket to br0 and joins the
> multicast group on the br0 interface instead of the eth0 interface supplied In the
> command.
> Our routes go through br0 as expected in this situation:
> 
> $ ip ro.3
> 224.0.0.0/4 dev br0  scope link
> ...
> 
> And our DSA is setup as a 'dumb' switch, all unknown DA frames are flooded out
> all ports.
> 
> Also our particular marvell switch does not have the ptp support, we are aware
> that some versions do.
> 
> When seeing the slave adjusting its ptp clock this leads me to think everything
> works okay with this patch? Is this something that could be added as an option to
> ptp4l and merged in the future to support linux sw bridges?
> 
> Thanks,
> Rob

I think you really want the bridge to learn that it's underlying devices have PTP hardware support, and then connect them in a sort of JBOD like setup. I don't have any idea how hard that would be to do.

Thanks,
Jake
 

Re: [Linuxptp-users] PTP through a bridge interface

[Rob C. <RCo...@si...>]

Hi Jake, appreciate the reply -

But just for my feasibility testing purposes if I:
--Pass eth0 (imx6 mac connected to cpu port of dsa switch) to ptp4l so it can create the clock and enable timestamping etc, and
--In my patch just bind the udp socket to 'br0' instead of 'eth0'  (and join mcast group on 'br0')
Doesn't this suffice to allow recving traffic over a bridge interface?

Thanks,
Rob

-----Original Message-----
From: Keller, Jacob E <jac...@in...> 
Sent: Tuesday, June 5, 2018 11:21 AM
To: Rob Cornall <RCo...@si...>; Richard Cochran <ric...@gm...>
Cc: lin...@li...
Subject: RE: [Linuxptp-users] PTP through a bridge interface



> -----Original Message-----
> From: Rob Cornall [mailto:RCo...@si...]
> Sent: Tuesday, June 05, 2018 9:47 AM
> To: Richard Cochran <ric...@gm...>
> Cc: lin...@li...
> Subject: Re: [Linuxptp-users] PTP through a bridge interface
> 
> Hi Richard, I really appreciate you answering my questions so quickly.
> 
> I attached my logs where I ran a patched version of ptp4l on the GM 
> device (I also attached this patch), the patch simply binds the socket 
> to br0 and joins the multicast group on the br0 interface instead of 
> the eth0 interface supplied In the command.
> Our routes go through br0 as expected in this situation:
> 
> $ ip ro.3
> 224.0.0.0/4 dev br0  scope link
> ...
> 
> And our DSA is setup as a 'dumb' switch, all unknown DA frames are 
> flooded out all ports.
> 
> Also our particular marvell switch does not have the ptp support, we 
> are aware that some versions do.
> 
> When seeing the slave adjusting its ptp clock this leads me to think 
> everything works okay with this patch? Is this something that could be 
> added as an option to ptp4l and merged in the future to support linux sw bridges?
> 
> Thanks,
> Rob

I think you really want the bridge to learn that it's underlying devices have PTP hardware support, and then connect them in a sort of JBOD like setup. I don't have any idea how hard that would be to do.

Thanks,
Jake
 

Re: [Linuxptp-users] PTP through a bridge interface

[Richard C. <ric...@gm...>]

On Tue, Jun 05, 2018 at 10:42:51PM +0000, Rob Cornall wrote:
> But just for my feasibility testing purposes if I:
> --Pass eth0 (imx6 mac connected to cpu port of dsa switch) to ptp4l so it can create the clock and enable timestamping etc, and
> --In my patch just bind the udp socket to 'br0' instead of 'eth0'  (and join mcast group on 'br0')
> Doesn't this suffice to allow recving traffic over a bridge interface?

That is really not a general solution by any means.

Thanks,
Richard

Re: [Linuxptp-users] PTP through a bridge interface

[Rob C. <RCo...@si...>]

The 06/05/2018 15:47, Richard Cochran wrote:
> On Tue, Jun 05, 2018 at 10:42:51PM +0000, Rob Cornall wrote:
> > But just for my feasibility testing purposes if I:
> > --Pass eth0 (imx6 mac connected to cpu port of dsa switch) to ptp4l so it can create the clock and enable
+timestamping etc, and
> > --In my patch just bind the udp socket to 'br0' instead of 'eth0'  (and join mcast group on 'br0')
> > Doesn't this suffice to allow recving traffic over a bridge interface?
>
> That is really not a general solution by any means.
>
> Thanks,
> Richard

I realize its not a general solution, but just for testing feasibility
at this moment?

For my other previous question I had, about Syncs and Delay_Reqs having
0 in the timestamp values (I had attached a capture) I see this also
occuring in a device with no switch connected and no bridge interface -
only an imx6 mac to a phy. I don't think this is expected behavior but it
seems to be functioning okay?

Thanks,
Rob C

Re: [Linuxptp-users] PTP through a bridge interface

[Richard C. <ric...@gm...>]

On Tue, Jun 05, 2018 at 11:14:25PM +0000, Rob Cornall wrote:
> For my other previous question I had, about Syncs and Delay_Reqs having
> 0 in the timestamp values (I had attached a capture) I see this also
> occuring in a device with no switch connected and no bridge interface -
> only an imx6 mac to a phy. I don't think this is expected behavior but it
> seems to be functioning okay?

No, that isn't okay, and the synchronization is not working.  I never
tested an imx myself, but it is supposed to work.

Thanks,
Richard

Re: [Linuxptp-users] PTP through a bridge interface

[Rob C. <RCo...@si...>]

Okay so I did some reading (should have done so earlier), I see that in section 9.5.9.4 for *two-step* mode the Sync does not actually carry the originTimestamp and it should be set to 0,

And instead the originTimestamp is in the Follow-up which is what I'm seeing so this behavior seems okay actually.


I also see in 11.3.2 the Delay_Reqs should also just have originTimestamp set to 0.



>Consider the simple case where you take two PTP capable MACs and then join them together into a bridge.


Could you run a ptp4l instance on both macs, eth0 & eth1 - oh you can't bind both instances to the same IP that would cause confusion on receiving packets?..

If this is the problem then you could use ptp's domainNumber to seperate them correct?Or Is there another issue I am not seeing?


Thanks,

Rob C


________________________________
From: Richard Cochran <ric...@gm...>
Sent: Tuesday, June 5, 2018 4:24 PM
To: Rob Cornall
Cc: Keller, Jacob E; lin...@li...
Subject: Re: [Linuxptp-users] PTP through a bridge interface

On Tue, Jun 05, 2018 at 11:14:25PM +0000, Rob Cornall wrote:
> For my other previous question I had, about Syncs and Delay_Reqs having
> 0 in the timestamp values (I had attached a capture) I see this also
> occuring in a device with no switch connected and no bridge interface -
> only an imx6 mac to a phy. I don't think this is expected behavior but it
> seems to be functioning okay?

No, that isn't okay, and the synchronization is not working.  I never
tested an imx myself, but it is supposed to work.

Thanks,
Richard