linuxptp-users — Open discussion on using the linuxptp software.

Re: [Linuxptp-users] Selecting a master

[Richard C. <ric...@gm...>]

On Tue, May 16, 2017 at 03:43:50PM +0200, Petr Kulhavy wrote:
> is it possible to explicitly tell ptp4l which master it should sync to 
> in case multiple masters are available on the network?

The protocol will pick the best master automatically.

You can force one master to be the best by configuration at the
master.

At the slave end, there is an optional feature in 1588 called
"Acceptable master table", but this is not implemented in linuxptp.

Thanks,
Richard

Re: [Linuxptp-users] Adding LinuxPTP to Arria 10 SoC

[Richard C. <ric...@gm...>]

On Tue, May 16, 2017 at 10:44:13AM -0500, John Lemonovich wrote:
> And then I ran:  make install  which put the output files into
> usr/local/sbin   and usr/local/man/man8

BTW, your can do 'make install DESTDIR=/my/embedded/fs' to install the
binaries and man pages to the proper place.

> Can I just copy those directories to my embedded Linux FS and run them?  

Yes.

Thanks,
Richard

Re: [Linuxptp-users] Selecting a master

[Petr K. <br...@ji...>]

Hi Kyle and thanks for your hints!
Priority and domainNumber relate to the local clock. And I need to to 
this over UDP.

Cheers
Petr

On 16/05/17 15:51, Mace, Kyle P. wrote:
> I have not personally done it but there might be a few options you can set up in a config file. This https://linux.die.net/man/8/ptp4l has a list of the configurations options. I would look into priority, domainNumber, and if doing L2 transport maybe ptp_dst_mac and p2p_dst_mac.
> -----Original Message-----
> From: Petr Kulhavy [mailto:br...@ji...]
> Sent: Tuesday, May 16, 2017 8:44 AM
> To: lin...@li...
> Subject: [Linuxptp-users] Selecting a master
>
> Hello,
>
> is it possible to explicitly tell ptp4l which master it should sync to in case multiple masters are available on the network?
>
> Thanks
> Petr
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
> Linuxptp-users mailing list
> Lin...@li...
> https://lists.sourceforge.net/lists/listinfo/linuxptp-users

[Linuxptp-users] Adding LinuxPTP to Arria 10 SoC

[John L. <joh...@fo...>]

Hello,

 

I am trying to add ptp support into my Altera Arria 10 SoC dev kit
(Cortex-A9).  Eventually I would like to use a 10G MAC with SGDMAs and SFP
port, but for now I am trying to get it working using the ARM HPS Gb-EMAC
and Micrel KSZ9031RN triple speed PHY .  

 

I have built Linux both using Yocto and also my own build from Altera's
LTSI kernel and Buildroot/Busybox for filesystem.  My question is how to
add LinuxPTP to my embedded Linux and add driver bindings to the MAC, etc.
so that I can run as an OC slave?  I have downloaded the LinuxPTP source,
and set my:

 

ARCH=arm

CROSS_COMPILE=CROSS_COMPILE=/localcad/skyfather/ArriaX/ltsi_4_1_33/gcc-lin
aro-5.4.1-2017.01-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-  

 

And then I ran:  make install  which put the output files into
usr/local/sbin   and usr/local/man/man8

 

Can I just copy those directories to my embedded Linux FS and run them?  

 

BTW - 

When I run ethtool -T eth0 to check capabilities I get:

 

Time stamping parameters for eth0:

Capabilities:

        hardware-transmit     (SOF_TIMESTAMPING_TX_HARDWARE)

        software-transmit     (SOF_TIMESTAMPING_TX_SOFTWARE)

        hardware-receive      (SOF_TIMESTAMPING_RX_HARDWARE)

        software-receive      (SOF_TIMESTAMPING_RX_SOFTWARE)

        software-system-clock (SOF_TIMESTAMPING_SOFTWARE)

        hardware-raw-clock    (SOF_TIMESTAMPING_RAW_HARDWARE)

PTP Hardware Clock: 0

Hardware Transmit Timestamp Modes:

        off                   (HWTSTAMP_TX_OFF)

        on                    (HWTSTAMP_TX_ON)

Hardware Receive Filter Modes:

        none                  (HWTSTAMP_FILTER_NONE)

        all                   (HWTSTAMP_FILTER_ALL)

        ptpv1-l4-event        (HWTSTAMP_FILTER_PTP_V1_L4_EVENT)

        ptpv1-l4-sync         (HWTSTAMP_FILTER_PTP_V1_L4_SYNC)

        ptpv1-l4-delay-req    (HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ)

        ptpv2-l4-event        (HWTSTAMP_FILTER_PTP_V2_L4_EVENT)

        ptpv2-l4-sync         (HWTSTAMP_FILTER_PTP_V2_L4_SYNC)

        ptpv2-l4-delay-req    (HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ)

        ptpv2-event           (HWTSTAMP_FILTER_PTP_V2_EVENT)

        ptpv2-sync            (HWTSTAMP_FILTER_PTP_V2_SYNC)

        ptpv2-delay-req       (HWTSTAMP_FILTER_PTP_V2_DELAY_REQ)

root@arria10:/#

 

 

 

Thank you,

 

John

Re: [Linuxptp-users] Selecting a master

[Mace, K. P. <Kyl...@gd...>]

I have not personally done it but there might be a few options you can set up in a config file. This https://linux.die.net/man/8/ptp4l has a list of the configurations options. I would look into priority, domainNumber, and if doing L2 transport maybe ptp_dst_mac and p2p_dst_mac.
-----Original Message-----
From: Petr Kulhavy [mailto:br...@ji...] 
Sent: Tuesday, May 16, 2017 8:44 AM
To: lin...@li...
Subject: [Linuxptp-users] Selecting a master

Hello,

is it possible to explicitly tell ptp4l which master it should sync to in case multiple masters are available on the network?

Thanks
Petr

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Linuxptp-users mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxptp-users

[Linuxptp-users] Selecting a master

[Petr K. <br...@ji...>]

Hello,

is it possible to explicitly tell ptp4l which master it should sync to 
in case multiple masters are available on the network?

Thanks
Petr

Re: [Linuxptp-users] Parsing of organization extension TLVs

[Richard C. <ric...@gm...>]

On Mon, May 15, 2017 at 08:14:52AM +0200, Morten Flå wrote:
> This is how the SMPTE 2059-2 is defined with messageId field set to
> Management Message and action set to Command followed by a
> Organization Extension TLV, so I guess someone did not think this
> through.

Oh man.

I guess we'll have to reject management messages that lack a
management TLV.

Thanks,
Richard

Re: [Linuxptp-users] Parsing of organization extension TLVs

[Morten F. <mm...@gm...>]

Hi,
Thanks for the clarification.
This is how the SMPTE 2059-2 is defined with messageId field set to
Management Message and action set to Command followed by a
Organization Extension TLV, so I guess someone did not think this
through.

Thanks,
Morten

Re: [Linuxptp-users] Parsing of organization extension TLVs

[Richard C. <ric...@gm...>]

On Fri, May 12, 2017 at 12:46:45PM +0200, Morten Flå wrote:
> In port.c:2259 we receive a packet, and the packet is rightly tagged
> as a management msg type in the common message header, and we're thus
> forwarded to the clock_manage function.

Ok, so the packet is a management message...

> In the clock_manage function there is no check on what the TLV type
> is. It's just assumed that the TLV type is 1, and thus the packet is
> incorrectly parsed.

and there is no need to check the TLV type after that.  According to
1588, management messages may have zero or one TLV (15.4.1.7), and
Table 38 makes it clear that only management TLVs are allowed.

For example:

GET action requires a single management TLV.

COMMAND action requires the "managementId field" to be set.
 
> Below is a snippet of the packet. PTP payload starts at 0x2A.
> 
> 0000   01 00 5e 00 01 81 ec 46 70 00 bf 9a 08 00 45 00  ..^....Fp.....E.
> 0010   00 80 2a 93 40 00 05 11 ba d9 ac 1b 02 64 e0 00  ..*.@........d..
> 0020   01 81 01 40 01 40 00 6c c6 e5 0d 02 00 64 00 00  ...@.@.l.....d..

Here we have ......................... ^^
   messageId: Management Message (0x0d)

> 0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 ec 46  ...............F
> 0040   70 ff fe 00 bf 9a 00 01 74 c8 04 7f ff ff ff ff  p.......t.......
> 0050   ff ff ff ff ff ff 01 01 03 00 00 03 00 30 68 97  .............0h.

and then ....................... ^^
   action: COMMAND (3)

followed by .......................... ^^ ^^
   tlvType: Organization extension (3)

That is just plain wrong.  The correct way to encode a vendor specific
management action is to use a managementId in range 0xC000-DFFF (Table 40).

For a correct example of the Organization extension, see sections

     10.5.4  Signaling message
   11.4.4.3  Follow_Up information TLV
  E.5.2.1.1  CSN TLV  (Follow_Up)

in IEEE 802.1AS-2011.  Notice how none of the those TLVs appear on
management messages.

Thanks,
Richard

Re: [Linuxptp-users] Parsing of organization extension TLVs

[Morten F. <mm...@gm...>]

Hi,
I don't believe this is the case.
By following the current code sequence from git
d1bb9ecbd5a77cdb8d93accfa8c1bbf705002de0

In port.c:2259 we receive a packet, and the packet is rightly tagged
as a management msg type in the common message header, and we're thus
forwarded to the clock_manage function.
In the clock_manage function there is no check on what the TLV type
is. It's just assumed that the TLV type is 1, and thus the packet is
incorrectly parsed.

Below is a snippet of the packet. PTP payload starts at 0x2A.

0000   01 00 5e 00 01 81 ec 46 70 00 bf 9a 08 00 45 00  ..^....Fp.....E.
0010   00 80 2a 93 40 00 05 11 ba d9 ac 1b 02 64 e0 00  ..*.@........d..
0020   01 81 01 40 01 40 00 6c c6 e5 0d 02 00 64 00 00  ...@.@.l.....d..
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 ec 46  ...............F
0040   70 ff fe 00 bf 9a 00 01 74 c8 04 7f ff ff ff ff  p.......t.......
0050   ff ff ff ff ff ff 01 01 03 00 00 03 00 30 68 97  .............0h.
0060   e8 00 00 01 00 00 00 19 00 00 00 01 01 00 ff ff  ................
0070   ff db 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0080   00 00 ff ff ff fe ae 80 ff ff ff db 00 00        ..............

Thanks,
Morten

On Fri, May 12, 2017 at 10:16 AM, Richard Cochran
<ric...@gm...> wrote:
> On Fri, May 12, 2017 at 09:03:51AM +0200, Morten Flå wrote:
>> I'm currently testing a PTP Grandmaster clock that has the ability to
>> send out Organization Extension TLVs. However, it seems like ptp4l
>> treats these messages as a normal management message. The result of
>> this is that ptp4l evaluates the organizationId field as the
>> managementId, as they are located at the same byte offset.
>
> Management messages have TLV type 1.
>
> Organization Extenstion messages have TLV type 3.
>
> Sounds like your GM is sending Organization Extenstion message with
> TLV type 1.  That would be wrong.
>
> Thanks,
> Richard
>
>
>
>

Re: [Linuxptp-users] Parsing of organization extension TLVs

[Richard C. <ric...@gm...>]

On Fri, May 12, 2017 at 09:03:51AM +0200, Morten Flå wrote:
> I'm currently testing a PTP Grandmaster clock that has the ability to
> send out Organization Extension TLVs. However, it seems like ptp4l
> treats these messages as a normal management message. The result of
> this is that ptp4l evaluates the organizationId field as the
> managementId, as they are located at the same byte offset.

Management messages have TLV type 1.

Organization Extenstion messages have TLV type 3.

Sounds like your GM is sending Organization Extenstion message with
TLV type 1.  That would be wrong.

Thanks,
Richard

[Linuxptp-users] Parsing of organization extension TLVs

[Morten F. <mm...@gm...>]

Hi,
I'm currently testing a PTP Grandmaster clock that has the ability to
send out Organization Extension TLVs. However, it seems like ptp4l
treats these messages as a normal management message. The result of
this is that ptp4l evaluates the organizationId field as the
managementId, as they are located at the same byte offset.

It seems to me that both the clock_manage and the port_manage function
should be aware of this TLV type, and disregard the content if the
organizationId or organizationSubType is not recognized as per
14.3.2.3 in 1588-2008.

Currently the behavior is that ptp4l sends out an Management Error
Message (NO_SUCH_ID), but of course the parsing result is "random"
based on the OUI.

Regards,
Morten

Re: [Linuxptp-users] ptp inducing as many hops as possible

[Richard C. <ric...@gm...>]

On Tue, Apr 25, 2017 at 03:41:12PM +0000, Mace, Kyle P. wrote:
> Why would you need multiple phc2sys processes? Shouldn't you only
> need one? Shouldn't the command phc2sys -s /dev/ptp0 -c /dev/ptp1 -w
> sync the two clocks? And would this still go through the system
> clock?

With 'phc2sys -a' and boundary_clock_jbod you don't involve the system
clock at all, but everything else Luke said is right.  You *will* have
"true" BC WRT the protocol, but this BC will introduce relatively
large errors, like a dozen usec or so.

However, if you can connect your JBODs together with a hardware
signal, like the SDPs on the Intel i210, then you can reduce the error
to 100 nanoseconds or so, but you will have to write the program to
synchronize the group of cards yourself.  Still, that is not really
the kind of performance you would expect from a proper BC.

If you really need long chains, then you should use TCs configured
like the Time Aware Bridge (TAB) of 802.1AS.  In order to avoid the
ill effects of a chain of servos, they let the clock of the TAB run
free and calculate the master/local rate ratio in order to correct the
residence times.

HTH,
Richard

Re: [Linuxptp-users] ptp inducing as many hops as possible

[Mace, K. P. <Kyl...@gd...>]

Why would you need multiple phc2sys processes? Shouldn't you only need one? Shouldn't the command phc2sys -s /dev/ptp0 -c /dev/ptp1 -w sync the two clocks? And would this still go through the system clock?

Thank you
Kyle

-----Original Message-----
From: Luke Bigum [mailto:luk...@lm...] 
Sent: Tuesday, April 25, 2017 8:40 AM
To: linuxptp-users
Subject: Re: [Linuxptp-users] ptp inducing as many hops as possible

Kyle,

I think you've misunderstood what Richard was talking about. You can't just arbitrarily set any network interface to use any PHC - the oscillator under the PHC device is physically attached to the NIC. Unless you have a multi-port network card that shares one oscillator (eg: SolarFlare 7x22), most NICs will probably have their own PHC.  That's why you're getting the device mismatch error.  

I've used multiple phc2sys processes in the past to sync different NICs, but this is not a true Boundary Clock. You'll be copying time up and down the Linux stack and through the system clock, so you are subject to all sorts of kernel scheduling jitter and PCI bus latency. If you just want some sort of PTP signal and don't care so much about accuracy though, you can do this.

Also regarding your question earlier about Gain Peaking, Richard was warning you about cumulative error.  So if you are proposing to go:

   grandmaster -> BC 1 port 1 -> BC 1 port 2 -> BC 2 port 1 -> BC 2 port 2 -> BC 3 port 1 -> BC 3 port 3

And it costs, say, +/- 10us of jitter to go between ports, by the time you get to BC3 port 3 you'll be at +/- 60us.

--
Luke Bigum
Lead Engineer

Information Systems

----- Original Message -----
From: "Mace, Kyle P." <Kyl...@gd...>
To: "Richard Cochran" <ric...@gm...>
Cc: "linuxptp-users" <lin...@li...>
Sent: Tuesday, 25 April, 2017 14:19:47
Subject: Re: [Linuxptp-users] ptp inducing as many hops as possible

Alright I am having troubles forcing the ports to use the same PHC. When I specify which PHC to use I get an error  about PHC device mismatch. Is there a way to unattach the hardware clock from a specific port?

Output of error
sudo ptp4l  -i eno1 -p /dev/ptp1 -m -s
ptp4l[61532.475]: selected /dev/ptp1 as PTP clock
ptp4l[61532.475]: port 1: PHC device mismatch
ptp4l[61532.476]: port 1: /dev/ptp1 requested, but /dev/ptp0 attached
ptp4l[61532.476]: failed to open port eno1

Or would I be able to enable the boundary_clock_jbod option in a config file and then sync the clocks off the system clock using phc2sys?

-----Original Message-----
From: Richard Cochran [mailto:ric...@gm...]
Sent: Thursday, April 20, 2017 4:27 PM
To: Mace, Kyle P.
Cc: lin...@li...
Subject: Re: [Linuxptp-users] ptp inducing as many hops as possible

On Thu, Apr 20, 2017 at 07:08:52PM +0000, Mace, Kyle P. wrote:
> I am in need to test ptp with as many 'hops' as possible. I have 
> multiple systems with dual Ethernet ports. I would like to have a 
> setup something like this: grandmaster -> BC 1 port 1 -> BC 1 port 2
> -> BC 2 port 1 -> BC 2 port 2 -> BC 3 port 1 -> BC 3 port 3 ->
> etc.. Anyone have any ideas if this is possible?

The linuxptp stack supports this.

Remember that all the ports of the BC must share the same clock (that is, the same PHC).

Also, long chains of BC are prone to "gain peaking".

HTH,
Richard

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Linuxptp-users mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxptp-users
---

LMAX Exchange, Yellow Building, 1A Nicholas Road, London W11 4AN http://www.LMAX.com/

Recognised by the most prestigious technology and business awards
 
Financial technology awards
2017 Best FX trading venue, Fund Technology & WSL Awards
2016 Best trading & execution venue, HFM US Technology Awards

FX industry awards
2016, 2015, 2014, 2013 Winner, Profit & Loss Readers' Choice Awards 2016, 2015, 2014, 2013 Winner, WSL Institutional Trading Awards

Business growth awards
2016, 2015 Winner, Deloitte UK Technology Fast 50 2015, 2014, 2013, Winner, The Sunday Times Tech Track 100 2016, 2015 Winner, Deloitte EMEA Technology Fast 500
2015 Winner, Tech City UK Future 50

---

FX and CFDs are leveraged products that can result in losses exceeding your deposit. They are not suitable for everyone so please ensure you fully understand the risks involved.

This message and its attachments are confidential, may not be disclosed or used by any person other than the addressee and are intended only for the named recipient(s). This message is not intended for any recipient(s) who based on their nationality, place of business, domicile or for any other reason, is/are subject to local laws or regulations which prohibit the provision of such products and services. This message is subject to the following terms (http://lmax.com/pdf/general-disclaimers.pdf), if you cannot access these, please notify us by replying to this email and we will send you the terms. If you are not the intended recipient, please notify the sender immediately and delete any copies of this message.

LMAX Exchange is the trading name of LMAX Limited. LMAX Limited operates a multilateral trading facility. LMAX Limited is authorised and regulated by the Financial Conduct Authority (firm registration number 509778) and is a company registered in England and Wales (number 6505809).

LMAX Hong Kong Limited is a wholly-owned subsidiary of LMAX Limited. LMAX Hong Kong is licensed by the Securities and Futures Commission in Hong Kong to conduct Type 3 (leveraged foreign exchange trading) regulated activity with CE Number BDV088.


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Linuxptp-users mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxptp-users

Re: [Linuxptp-users] ptp inducing as many hops as possible

[Mace, K. P. <Kyl...@gd...>]

Ahhhhh I understand the PHC thing now. Is there any way to induce as many "hops" as possible then?  I set a test earlier where I just looped through a switch a whole bunch of times using VLANS, but I wasn't sure if that truly counted as a "hop" I would need to check if the switch supports PTP or not. 

If push comes to shove the phc2sys might work for me.

Thank you so much for the help.
-----Original Message-----
From: Luke Bigum [mailto:luk...@lm...] 
Sent: Tuesday, April 25, 2017 8:40 AM
To: linuxptp-users
Subject: Re: [Linuxptp-users] ptp inducing as many hops as possible

Kyle,

I think you've misunderstood what Richard was talking about. You can't just arbitrarily set any network interface to use any PHC - the oscillator under the PHC device is physically attached to the NIC. Unless you have a multi-port network card that shares one oscillator (eg: SolarFlare 7x22), most NICs will probably have their own PHC.  That's why you're getting the device mismatch error.  

I've used multiple phc2sys processes in the past to sync different NICs, but this is not a true Boundary Clock. You'll be copying time up and down the Linux stack and through the system clock, so you are subject to all sorts of kernel scheduling jitter and PCI bus latency. If you just want some sort of PTP signal and don't care so much about accuracy though, you can do this.

Also regarding your question earlier about Gain Peaking, Richard was warning you about cumulative error.  So if you are proposing to go:

   grandmaster -> BC 1 port 1 -> BC 1 port 2 -> BC 2 port 1 -> BC 2 port 2 -> BC 3 port 1 -> BC 3 port 3

And it costs, say, +/- 10us of jitter to go between ports, by the time you get to BC3 port 3 you'll be at +/- 60us.

--
Luke Bigum
Lead Engineer

Information Systems

----- Original Message -----
From: "Mace, Kyle P." <Kyl...@gd...>
To: "Richard Cochran" <ric...@gm...>
Cc: "linuxptp-users" <lin...@li...>
Sent: Tuesday, 25 April, 2017 14:19:47
Subject: Re: [Linuxptp-users] ptp inducing as many hops as possible

Alright I am having troubles forcing the ports to use the same PHC. When I specify which PHC to use I get an error  about PHC device mismatch. Is there a way to unattach the hardware clock from a specific port?

Output of error
sudo ptp4l  -i eno1 -p /dev/ptp1 -m -s
ptp4l[61532.475]: selected /dev/ptp1 as PTP clock
ptp4l[61532.475]: port 1: PHC device mismatch
ptp4l[61532.476]: port 1: /dev/ptp1 requested, but /dev/ptp0 attached
ptp4l[61532.476]: failed to open port eno1

Or would I be able to enable the boundary_clock_jbod option in a config file and then sync the clocks off the system clock using phc2sys?

-----Original Message-----
From: Richard Cochran [mailto:ric...@gm...]
Sent: Thursday, April 20, 2017 4:27 PM
To: Mace, Kyle P.
Cc: lin...@li...
Subject: Re: [Linuxptp-users] ptp inducing as many hops as possible

On Thu, Apr 20, 2017 at 07:08:52PM +0000, Mace, Kyle P. wrote:
> I am in need to test ptp with as many 'hops' as possible. I have 
> multiple systems with dual Ethernet ports. I would like to have a 
> setup something like this: grandmaster -> BC 1 port 1 -> BC 1 port 2
> -> BC 2 port 1 -> BC 2 port 2 -> BC 3 port 1 -> BC 3 port 3 ->
> etc.. Anyone have any ideas if this is possible?

The linuxptp stack supports this.

Remember that all the ports of the BC must share the same clock (that is, the same PHC).

Also, long chains of BC are prone to "gain peaking".

HTH,
Richard

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Linuxptp-users mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxptp-users
---

LMAX Exchange, Yellow Building, 1A Nicholas Road, London W11 4AN http://www.LMAX.com/

Recognised by the most prestigious technology and business awards
 
Financial technology awards
2017 Best FX trading venue, Fund Technology & WSL Awards
2016 Best trading & execution venue, HFM US Technology Awards

FX industry awards
2016, 2015, 2014, 2013 Winner, Profit & Loss Readers' Choice Awards 2016, 2015, 2014, 2013 Winner, WSL Institutional Trading Awards

Business growth awards
2016, 2015 Winner, Deloitte UK Technology Fast 50 2015, 2014, 2013, Winner, The Sunday Times Tech Track 100 2016, 2015 Winner, Deloitte EMEA Technology Fast 500
2015 Winner, Tech City UK Future 50

---

FX and CFDs are leveraged products that can result in losses exceeding your deposit. They are not suitable for everyone so please ensure you fully understand the risks involved.

This message and its attachments are confidential, may not be disclosed or used by any person other than the addressee and are intended only for the named recipient(s). This message is not intended for any recipient(s) who based on their nationality, place of business, domicile or for any other reason, is/are subject to local laws or regulations which prohibit the provision of such products and services. This message is subject to the following terms (http://lmax.com/pdf/general-disclaimers.pdf), if you cannot access these, please notify us by replying to this email and we will send you the terms. If you are not the intended recipient, please notify the sender immediately and delete any copies of this message.

LMAX Exchange is the trading name of LMAX Limited. LMAX Limited operates a multilateral trading facility. LMAX Limited is authorised and regulated by the Financial Conduct Authority (firm registration number 509778) and is a company registered in England and Wales (number 6505809).

LMAX Hong Kong Limited is a wholly-owned subsidiary of LMAX Limited. LMAX Hong Kong is licensed by the Securities and Futures Commission in Hong Kong to conduct Type 3 (leveraged foreign exchange trading) regulated activity with CE Number BDV088.


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Linuxptp-users mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxptp-users

Re: [Linuxptp-users] ptp inducing as many hops as possible

[Luke B. <luk...@lm...>]

Kyle,

I think you've misunderstood what Richard was talking about. You can't just arbitrarily set any network interface to use any PHC - the oscillator under the PHC device is physically attached to the NIC. Unless you have a multi-port network card that shares one oscillator (eg: SolarFlare 7x22), most NICs will probably have their own PHC.  That's why you're getting the device mismatch error.  

I've used multiple phc2sys processes in the past to sync different NICs, but this is not a true Boundary Clock. You'll be copying time up and down the Linux stack and through the system clock, so you are subject to all sorts of kernel scheduling jitter and PCI bus latency. If you just want some sort of PTP signal and don't care so much about accuracy though, you can do this.

Also regarding your question earlier about Gain Peaking, Richard was warning you about cumulative error.  So if you are proposing to go:

   grandmaster -> BC 1 port 1 -> BC 1 port 2 -> BC 2 port 1 -> BC 2 port 2 -> BC 3 port 1 -> BC 3 port 3

And it costs, say, +/- 10us of jitter to go between ports, by the time you get to BC3 port 3 you'll be at +/- 60us.

-- 
Luke Bigum
Lead Engineer

Information Systems

----- Original Message -----
From: "Mace, Kyle P." <Kyl...@gd...>
To: "Richard Cochran" <ric...@gm...>
Cc: "linuxptp-users" <lin...@li...>
Sent: Tuesday, 25 April, 2017 14:19:47
Subject: Re: [Linuxptp-users] ptp inducing as many hops as possible

Alright I am having troubles forcing the ports to use the same PHC. When I specify which PHC to use I get an error  about PHC device mismatch. Is there a way to unattach the hardware clock from a specific port?

Output of error
sudo ptp4l  -i eno1 -p /dev/ptp1 -m -s
ptp4l[61532.475]: selected /dev/ptp1 as PTP clock
ptp4l[61532.475]: port 1: PHC device mismatch
ptp4l[61532.476]: port 1: /dev/ptp1 requested, but /dev/ptp0 attached
ptp4l[61532.476]: failed to open port eno1

Or would I be able to enable the boundary_clock_jbod option in a config file and then sync the clocks off the system clock using phc2sys?

-----Original Message-----
From: Richard Cochran [mailto:ric...@gm...] 
Sent: Thursday, April 20, 2017 4:27 PM
To: Mace, Kyle P.
Cc: lin...@li...
Subject: Re: [Linuxptp-users] ptp inducing as many hops as possible

On Thu, Apr 20, 2017 at 07:08:52PM +0000, Mace, Kyle P. wrote:
> I am in need to test ptp with as many 'hops' as possible. I have 
> multiple systems with dual Ethernet ports. I would like to have a 
> setup something like this: grandmaster -> BC 1 port 1 -> BC 1 port 2
> -> BC 2 port 1 -> BC 2 port 2 -> BC 3 port 1 -> BC 3 port 3 ->
> etc.. Anyone have any ideas if this is possible?

The linuxptp stack supports this.

Remember that all the ports of the BC must share the same clock (that is, the same PHC).

Also, long chains of BC are prone to "gain peaking".

HTH,
Richard

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Linuxptp-users mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxptp-users
---

LMAX Exchange, Yellow Building, 1A Nicholas Road, London W11 4AN
http://www.LMAX.com/

Recognised by the most prestigious technology and business awards
 
Financial technology awards
2017 Best FX trading venue, Fund Technology & WSL Awards
2016 Best trading & execution venue, HFM US Technology Awards

FX industry awards
2016, 2015, 2014, 2013 Winner, Profit & Loss Readers' Choice Awards
2016, 2015, 2014, 2013 Winner, WSL Institutional Trading Awards

Business growth awards
2016, 2015 Winner, Deloitte UK Technology Fast 50
2015, 2014, 2013, Winner, The Sunday Times Tech Track 100
2016, 2015 Winner, Deloitte EMEA Technology Fast 500
2015 Winner, Tech City UK Future 50

---

FX and CFDs are leveraged products that can result in losses exceeding your deposit. They are not suitable for everyone so please ensure you fully understand the risks involved.

This message and its attachments are confidential, may not be disclosed or used by any person other than the addressee and are intended only for the named recipient(s). This message is not intended for any recipient(s) who based on their nationality, place of business, domicile or for any other reason, is/are subject to local laws or regulations which prohibit the provision of such products and services. This message is subject to the following terms (http://lmax.com/pdf/general-disclaimers.pdf), if you cannot access these, please notify us by replying to this email and we will send you the terms. If you are not the intended recipient, please notify the sender immediately and delete any copies of this message.

LMAX Exchange is the trading name of LMAX Limited. LMAX Limited operates a multilateral trading facility. LMAX Limited is authorised and regulated by the Financial Conduct Authority (firm registration number 509778) and is a company registered in England and Wales (number 6505809).

LMAX Hong Kong Limited is a wholly-owned subsidiary of LMAX Limited. LMAX Hong Kong is licensed by the Securities and Futures Commission in Hong Kong to conduct Type 3 (leveraged foreign exchange trading) regulated activity with CE Number BDV088.

Re: [Linuxptp-users] ptp inducing as many hops as possible

[Mace, K. P. <Kyl...@gd...>]

Alright I am having troubles forcing the ports to use the same PHC. When I specify which PHC to use I get an error  about PHC device mismatch. Is there a way to unattach the hardware clock from a specific port?

Output of error
sudo ptp4l  -i eno1 -p /dev/ptp1 -m -s
ptp4l[61532.475]: selected /dev/ptp1 as PTP clock
ptp4l[61532.475]: port 1: PHC device mismatch
ptp4l[61532.476]: port 1: /dev/ptp1 requested, but /dev/ptp0 attached
ptp4l[61532.476]: failed to open port eno1

Or would I be able to enable the boundary_clock_jbod option in a config file and then sync the clocks off the system clock using phc2sys?

-----Original Message-----
From: Richard Cochran [mailto:ric...@gm...] 
Sent: Thursday, April 20, 2017 4:27 PM
To: Mace, Kyle P.
Cc: lin...@li...
Subject: Re: [Linuxptp-users] ptp inducing as many hops as possible

On Thu, Apr 20, 2017 at 07:08:52PM +0000, Mace, Kyle P. wrote:
> I am in need to test ptp with as many 'hops' as possible. I have 
> multiple systems with dual Ethernet ports. I would like to have a 
> setup something like this: grandmaster -> BC 1 port 1 -> BC 1 port 2
> -> BC 2 port 1 -> BC 2 port 2 -> BC 3 port 1 -> BC 3 port 3 ->
> etc.. Anyone have any ideas if this is possible?

The linuxptp stack supports this.

Remember that all the ports of the BC must share the same clock (that is, the same PHC).

Also, long chains of BC are prone to "gain peaking".

HTH,
Richard

Re: [Linuxptp-users] ptp inducing as many hops as possible

[Richard C. <ric...@gm...>]

On Mon, Apr 24, 2017 at 11:58:59AM +0000, Mace, Kyle P. wrote:
> That is very good news. What do you mean by "gain peaking"?

@book{gardner2005phaselock,
  title={Phaselock Techniques},
  author={Gardner, F.M.},
  isbn=9780471732686,
  url={https://books.google.at/books?id=r5yjPuWQde0C},
  year=2005,
  publisher={Wiley}
}

See page 19.

HTH,
Richard

Re: [Linuxptp-users] ptp inducing as many hops as possible

[Mace, K. P. <Kyl...@gd...>]

That is very good news. What do you mean by "gain peaking"?

Thank you,
Kyle Mace

-----Original Message-----
From: Richard Cochran [mailto:ric...@gm...] 
Sent: Thursday, April 20, 2017 4:27 PM
To: Mace, Kyle P.
Cc: lin...@li...
Subject: Re: [Linuxptp-users] ptp inducing as many hops as possible

On Thu, Apr 20, 2017 at 07:08:52PM +0000, Mace, Kyle P. wrote:
> I am in need to test ptp with as many 'hops' as possible. I have 
> multiple systems with dual Ethernet ports. I would like to have a 
> setup something like this: grandmaster -> BC 1 port 1 -> BC 1 port 2
> -> BC 2 port 1 -> BC 2 port 2 -> BC 3 port 1 -> BC 3 port 3 ->
> etc.. Anyone have any ideas if this is possible?

The linuxptp stack supports this.

Remember that all the ports of the BC must share the same clock (that is, the same PHC).

Also, long chains of BC are prone to "gain peaking".

HTH,
Richard

Re: [Linuxptp-users] LinuxPTP on Zynq with DP83640

[Wolfgang H. <wh...@xi...>]

Richard Cochran wrote:
> On Tue, Apr 18, 2017 at 12:54:23PM -0700, Wolfgang Hennig wrote:
>> I'm working on a project where a Xilinx Zynq uses a DP83640 (eval board) as
>> the Ethernet PHY. I'm running Ubuntu/Linaro 15 on the Zynq's ARM processor
>> and successfully modified kernel options and compiled/installed LinuxPTP to
>> synchronize time over the network. Now I would like to use the DP83640's
>> configurable reference clock (output on a GPIO pin) to clock logic in the
>> FPGA section of the Zynq. To do that I would need to read/write the
>> DP83640's internal registers, but I don't know how.
>>
>> It seems that there is some functionality in setting the registers in
>> dp83640.c. In an earlier post in this list:
> We don't have any support for enabling the clock output.  You can hack
> this in using something like this:
>
> 	ext_write(0, phydev, PAGE6, PTP_COC, coc);
That worked pretty well.
I put it in dp838640_config_init (as a first test).
>
>> I should add that I did not (yet) find where/how to modify the xemacps
>> driver per https://lwn.net/Articles/392150/
>>>     1. Before mdio_register, add
>>> 		bus->locktype = MDIOBUS_ATOMIC_RW;
>>>     2. In the .ndo_start_xmit function, add
>>> 		skb_tx_timestamp()
>>>     3. In the NAPI poll function, add
>>> 		skb_rx_timestamp()
> That info is obsolete.  All you need is a MAC driver that uses phylib
> and calls skb_tx_timestamp().
>
>> and so the HW timestamping might have been provided by the Zynq's build in
>> TSU instead of the DP83640. How would you tell?
> If your Zynq driver implements time stamping or PHC, then you should
> disable or remove that code.
There was definitely an issue there. Three changes in/for the xemacps 
driver were required:
1) turn off option CONFIG_XILINX_PS_EMAC_HWTSTAMP for the kernel compile
2) in ~line 2800 of xilinx_emacps.c, function xemacps_ioctl, add the 
#else section below so that it responds with the PHY's HW timestamping 
capabilities when queried by ptp4l. (Interestingly, ethtool -T always 
reports the PHY's capabilities)
     switch (cmd) {
     case SIOCGMIIPHY:
     case SIOCGMIIREG:
     case SIOCSMIIREG:
         return phy_mii_ioctl(phydev, rq, cmd);
#ifdef CONFIG_XILINX_PS_EMAC_HWTSTAMP
     case SIOCSHWTSTAMP:
         return xemacps_hwtstamp_ioctl(ndev, rq, cmd);
#else
case SIOCSHWTSTAMP:
         return phy_mii_ioctl(phydev, rq, cmd);
#endif
3) I added skb_tx_timestamp(skb) in line 2256 of xilinx_emacps.c, before 
the for loop over nr_frags. Seems to work, didn't experiment with other 
locations.



I now get offsets from master of about 100ns through our local network 
and <10ns when connecting 2 boards back to back. The DP83640's CLK_OUT 
signals are pretty well synchronized (looking at an oscilloscope I see 
the edges drifting by about 10ns relative to each other).


Thank you very much for the hints, Richard, that was very helpful.

Wolfgang
>
> BTW you can check sysfs to see the PHC device names:
>
> What:		/sys/class/ptp/ptpN/clock_name
> Description:
> 		This file contains the name of the PTP hardware clock
> 		as a human readable string. The purpose of this
> 		attribute is to provide the user with a "friendly
> 		name" and to help distinguish PHY based devices from
> 		MAC based ones. The string does not necessarily have
> 		to be any kind of unique id.
>
>
>> Does it matter where in the
>> function one adds skb_tx_timestamp()?
> See the comment in include/linux/skbuff.h.
>
> Thanks,
> Richard

-- 
Wolfgang Hennig, Ph.D.
XIA LLC
31057 Genstar Rd.
Hayward CA 94544

Re: [Linuxptp-users] High frequency

[Collins, C. L. <Cri...@gd...>]

Another strange bit is without PTP or any other time protocol running, on many of the systems the system clock is drifting rapidly after syncing it to hardware clock manually. Within twenty minutes the system clock is 7 minutes ahead of the hardware clock. Is there way to see what is affecting the system clock? There are about 8 machines acting this way, another 3 with seemingly the same hardware, OS, and software on the same switch that are not exhibiting the behavior.

The following is 24 hours after syncing the System clock to the hardware clock manually 
date;hwclock
Thu Apr 20 18:58:01 CDT 2017
Thu 20 Apr 2017 04:38:20 PM CDT  -0.310003 seconds
 
ethtool -i  ens2
driver: mlx4_en
version: 3.1-1.0.4 (30 Sep 2015)
firmware-version: 2.35.5100
bus-info: 0000:83:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes


ethtool -T ens2
Time stamping parameters for ens2:
Capabilities:
        hardware-transmit     (SOF_TIMESTAMPING_TX_HARDWARE)
        software-transmit     (SOF_TIMESTAMPING_TX_SOFTWARE)
        hardware-receive      (SOF_TIMESTAMPING_RX_HARDWARE)
        software-receive      (SOF_TIMESTAMPING_RX_SOFTWARE)
        software-system-clock (SOF_TIMESTAMPING_SOFTWARE)
        hardware-raw-clock    (SOF_TIMESTAMPING_RAW_HARDWARE)
PTP Hardware Clock: 2
Hardware Transmit Timestamp Modes:
        off                   (HWTSTAMP_TX_OFF)
        on                    (HWTSTAMP_TX_ON)
Hardware Receive Filter Modes:
        none                  (HWTSTAMP_FILTER_NONE)
        all                   (HWTSTAMP_FILTER_ALL)



-----Original Message-----
From: Richard Cochran [mailto:ric...@gm...] 
Sent: Thursday, April 20, 2017 5:23 PM
To: Collins, Cris L.
Cc: lin...@li...
Subject: Re: [Linuxptp-users] High frequency

On Wed, Apr 19, 2017 at 06:52:41PM +0000, Collins, Cris L. wrote:
> We have a dozen very similar RHEL 7.2 systems configured with Linuxptp 1.6 and the same NIC driver version on each. Some of them look great, some show a frequency of "-100000000" and an offset of ~ -2333333 and some say Clockcheck: clock jumped forward or running faster than expected. Everyone is on the same switch.

Sounds like a HW bug.
 
> What are some things we can do to debug the problem machines?

Which HW are you using?

Thanks,
Richard

Re: [Linuxptp-users] ptp inducing as many hops as possible

[Richard C. <ric...@gm...>]

On Thu, Apr 20, 2017 at 07:08:52PM +0000, Mace, Kyle P. wrote:
> I am in need to test ptp with as many 'hops' as possible. I have
> multiple systems with dual Ethernet ports. I would like to have a
> setup something like this: grandmaster -> BC 1 port 1 -> BC 1 port 2
> -> BC 2 port 1 -> BC 2 port 2 -> BC 3 port 1 -> BC 3 port 3 ->
> etc.. Anyone have any ideas if this is possible?

The linuxptp stack supports this.

Remember that all the ports of the BC must share the same clock (that
is, the same PHC).

Also, long chains of BC are prone to "gain peaking".

HTH,
Richard

Re: [Linuxptp-users] High frequency

[Richard C. <ric...@gm...>]

On Wed, Apr 19, 2017 at 06:52:41PM +0000, Collins, Cris L. wrote:
> We have a dozen very similar RHEL 7.2 systems configured with Linuxptp 1.6 and the same NIC driver version on each. Some of them look great, some show a frequency of "-100000000" and an offset of ~ -2333333 and some say Clockcheck: clock jumped forward or running faster than expected. Everyone is on the same switch.

Sounds like a HW bug.
 
> What are some things we can do to debug the problem machines?

Which HW are you using?

Thanks,
Richard

[Linuxptp-users] ptp inducing as many hops as possible

[Mace, K. P. <Kyl...@gd...>]

I am in need to test ptp with as many 'hops' as possible. I have multiple systems with dual Ethernet ports. I would like to have a setup something like this: grandmaster -> BC 1 port 1 -> BC 1 port 2 -> BC 2 port 1 -> BC 2 port 2 -> BC 3 port 1 -> BC 3 port 3 -> etc.. Anyone have any ideas if this is possible?

[Linuxptp-users] High frequency

[Collins, C. L. <Cri...@gd...>]

We have a dozen very similar RHEL 7.2 systems configured with Linuxptp 1.6 and the same NIC driver version on each. Some of them look great, some show a frequency of "-100000000" and an offset of ~ -2333333 and some say Clockcheck: clock jumped forward or running faster than expected. Everyone is on the same switch.

I can't see a program like chronyd ot NTP that would be adjusting the time. Is there a way to see what programs are adjusting the clock?

What are some things we can do to debug the problem machines?

Thank you for any direction that you may be able to provide.