Thread: [Linuxptp-users] ptp4l and Broadcom 5720 problem
PTP IEEE 1588 stack for Linux
Brought to you by:
rcochran
Menu
▾
▴
linuxptp-users
|
From: Jay S. <li...@sh...> - 2022-06-06 15:26:55
|
Hello all, after a bit of advice/help. I've got some CentOS 7 servers and I need to get them running ptp4l (from linuxptp) as clients, with the grandmaster being a Meinberg appliance. One with an Intel i350 works fine. Another with a Broadcom 5720 does not. Both servers are connected to the same network for their PTP. I suspect I know why the Broadcom doesn't work, but I was wondering if anyone could point me in the right direction on how to get it to do so. In short the i350's hardware timstamping capabilities say this: |# ethtool -T lan2 <snip> Hardware Receive Filter Modes: none (HWTSTAMP_FILTER_NONE) all (HWTSTAMP_FILTER_ALL) | But the Broadcom says this: |# ethtool -T lan2 <snip> Hardware Receive Filter Modes: none (HWTSTAMP_FILTER_NONE) ptpv1-l4-event (HWTSTAMP_FILTER_PTP_V1_L4_EVENT) ptpv2-l4-event (HWTSTAMP_FILTER_PTP_V2_L4_EVENT) ptpv2-l2-event (HWTSTAMP_FILTER_PTP_V2_L2_EVENT) | So I get the feeling that the Broadcom isn't picking up some of the messages coming back from the grandmaster. eg in the logs I can see: |Jun 3 10:12:37.000 ldalgc01 ptp4l[8290]: ptp4l[601302.197]: selected local clock e43d1a.fffe.acab65 as best master Jun 3 10:12:37.000 ldalgc01 ptp4l: [601302.197] selected local clock e43d1a.fffe.acab65 as best master Jun 3 10:12:38.000 ldalgc01 phc2sys: [601303.026] Waiting for ptp4l... Jun 3 10:12:39.000 ldalgc01 phc2sys: [601304.027] Waiting for ptp4l... | But a tcpdump shows that such messages are arriving: |10:12:39.381908 IP gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: UDP, length 54 10:12:39.398091 IP gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: UDP, length 54 10:12:39.404572 IP gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: UDP, length 54 | The Broadcom 5720 blurb says that it has Hardware support for IEEE 1588 timestamping <https://docs.broadcom.com/doc/5720-PB01-R>, so I guess it might need some tweaking of the ptp4l conf, but I'm not sure where to start. Current config (that works fine on the i350) is: |[global] # # Default Data Set # twoStepFlag 1 slaveOnly 1 priority1 128 priority2 128 domainNumber 0 #utc_offset 37 clockClass 248 clockAccuracy 0xFE offsetScaledLogVariance 0xFFFF free_running 0 freq_est_interval 1 dscp_event 0 dscp_general 0 dataset_comparison ieee1588 G.8275.defaultDS.localPriority 128 # # Port Data Set # logAnnounceInterval 1 logSyncInterval 0 logMinDelayReqInterval 0 logMinPdelayReqInterval 0 announceReceiptTimeout 3 syncReceiptTimeout 0 delayAsymmetry 0 fault_reset_interval 4 neighborPropDelayThresh 20000000 masterOnly 0 G.8275.portDS.localPriority 128 # # Run time options # assume_two_step 0 logging_level 6 path_trace_enabled 0 follow_up_info 0 hybrid_e2e 1 inhibit_multicast_service 0 net_sync_monitor 0 tc_spanning_tree 0 tx_timestamp_timeout 1 unicast_listen 0 unicast_master_table 0 unicast_req_duration 3600 use_syslog 1 verbose 1 summary_interval 0 kernel_leap 1 check_fup_sync 0 # # Servo Options # pi_proportional_const 0.0 pi_integral_const 0.0 pi_proportional_scale 0.0 pi_proportional_exponent -0.3 pi_proportional_norm_max 0.7 pi_integral_scale 0.0 pi_integral_exponent 0.4 pi_integral_norm_max 0.3 step_threshold 0.0 first_step_threshold 0.00002 max_frequency 900000000 clock_servo linreg sanity_freq_limit 200000000 ntpshm_segment 0 # # Transport options # transportSpecific 0x0 ptp_dst_mac 01:1B:19:00:00:00 p2p_dst_mac 01:80:C2:00:00:0E udp_ttl 32 udp6_scope 0x0E uds_address /var/run/ptp4l # # Default interface options # clock_type OC network_transport UDPv4 delay_mechanism E2E time_stamping hardware tsproc_mode filter delay_filter moving_median delay_filter_length 10 egressLatency 0 ingressLatency 0 boundary_clock_jbod 0 # # Clock description # productDescription ;; revisionData ;; manufacturerIdentity 00:00:00 userDescription ; timeSource 0xA0 Thanks in advance, J | |
|
From: Jay S. <li...@sh...> - 2022-06-06 15:48:41
|
On 06/06/2022 16:10, Jay Shardlow wrote: > Hello all, after a bit of advice/help. I've got some CentOS 7 servers > and I need to get them running ptp4l (from linuxptp) as clients, with > the grandmaster being a Meinberg appliance. One with an Intel i350 works > fine. Another with a Broadcom 5720 does not. Both servers are connected > to the same network for their PTP. I suspect I know why the Broadcom > doesn't work, but I was wondering if anyone could point me in the right > direction on how to get it to do so. > > In short the i350's hardware timstamping capabilities say this: > > |# ethtool -T lan2 <snip> Hardware Receive Filter Modes: > none (HWTSTAMP_FILTER_NONE) all > (HWTSTAMP_FILTER_ALL) | > > But the Broadcom says this: > > |# ethtool -T lan2 <snip> Hardware Receive Filter Modes: > none (HWTSTAMP_FILTER_NONE) ptpv1-l4-event > (HWTSTAMP_FILTER_PTP_V1_L4_EVENT) ptpv2-l4-event > (HWTSTAMP_FILTER_PTP_V2_L4_EVENT) ptpv2-l2-event > (HWTSTAMP_FILTER_PTP_V2_L2_EVENT) | > > So I get the feeling that the Broadcom isn't picking up some of the > messages coming back from the grandmaster. eg in the logs I can see: > > |Jun 3 10:12:37.000 ldalgc01 ptp4l[8290]: ptp4l[601302.197]: selected > local clock e43d1a.fffe.acab65 as best master Jun 3 10:12:37.000 > ldalgc01 ptp4l: [601302.197] selected local clock e43d1a.fffe.acab65 as > best master Jun 3 10:12:38.000 ldalgc01 phc2sys: [601303.026] Waiting > for ptp4l... Jun 3 10:12:39.000 ldalgc01 phc2sys: [601304.027] Waiting > for ptp4l... | > > But a tcpdump shows that such messages are arriving: > > |10:12:39.381908 IP gsdgmca01-ptp0.prod.instinet.com.ptp-general > > 224.0.1.129.ptp-general: UDP, length 54 10:12:39.398091 IP > gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: > UDP, length 54 10:12:39.404572 IP > gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: > UDP, length 54 | > > The Broadcom 5720 blurb says that it has Hardware support for IEEE 1588 > timestamping <https://docs.broadcom.com/doc/5720-PB01-R>, so I guess it > might need some tweaking of the ptp4l conf, but I'm not sure where to > start. Current config (that works fine on the i350) is: > > |[global] # # Default Data Set # twoStepFlag 1 > slaveOnly 1 priority1 128 priority2 > 128 domainNumber 0 #utc_offset 37 > clockClass 248 clockAccuracy 0xFE > offsetScaledLogVariance 0xFFFF free_running 0 > freq_est_interval 1 dscp_event 0 > dscp_general 0 dataset_comparison ieee1588 > G.8275.defaultDS.localPriority 128 # # Port Data Set # > logAnnounceInterval 1 logSyncInterval 0 > logMinDelayReqInterval 0 logMinPdelayReqInterval 0 > announceReceiptTimeout 3 syncReceiptTimeout 0 > delayAsymmetry 0 fault_reset_interval 4 > neighborPropDelayThresh 20000000 masterOnly 0 > G.8275.portDS.localPriority 128 # # Run time options # > assume_two_step 0 logging_level 6 > path_trace_enabled 0 follow_up_info 0 > hybrid_e2e 1 inhibit_multicast_service 0 > net_sync_monitor 0 tc_spanning_tree 0 > tx_timestamp_timeout 1 unicast_listen 0 > unicast_master_table 0 unicast_req_duration 3600 > use_syslog 1 verbose 1 > summary_interval 0 kernel_leap 1 > check_fup_sync 0 # # Servo Options # pi_proportional_const 0.0 > pi_integral_const 0.0 pi_proportional_scale 0.0 > pi_proportional_exponent -0.3 pi_proportional_norm_max 0.7 > pi_integral_scale 0.0 pi_integral_exponent 0.4 > pi_integral_norm_max 0.3 step_threshold 0.0 > first_step_threshold 0.00002 max_frequency 900000000 > clock_servo linreg sanity_freq_limit 200000000 > ntpshm_segment 0 # # Transport options # > transportSpecific 0x0 ptp_dst_mac 01:1B:19:00:00:00 > p2p_dst_mac 01:80:C2:00:00:0E udp_ttl 32 > udp6_scope 0x0E uds_address /var/run/ptp4l # # > Default interface options # clock_type OC > network_transport UDPv4 delay_mechanism E2E > time_stamping hardware tsproc_mode filter > delay_filter moving_median delay_filter_length 10 > egressLatency 0 ingressLatency 0 boundary_clock_jbod > 0 # # Clock description # productDescription ;; > revisionData ;; manufacturerIdentity 00:00:00 > userDescription ; timeSource 0xA0 Thanks in > advance, J | > > > _______________________________________________ > Linuxptp-users mailing list > Lin...@li... > https://lists.sourceforge.net/lists/listinfo/linuxptp-users Apologies for the bad formatting, let me try some of that again with a bit more care with the cut/paste. In short Intel i350 is fine, Broadcom 5720 is not. The Broadcom is listed as having Hardware support for IEEE 1588 timestamping. But I'm not sure how to get it workimg (like the Intel) with ptp4l. Intel i350: # ethtool -T lan2 <snip> Hardware Receive Filter Modes: none (HWTSTAMP_FILTER_NONE) all (HWTSTAMP_FILTER_ALL) Broadcom 5720: # ethtool -T lan2 <snip> Hardware Receive Filter Modes: none (HWTSTAMP_FILTER_NONE) ptpv1-l4-event (HWTSTAMP_FILTER_PTP_V1_L4_EVENT) ptpv2-l4-event (HWTSTAMP_FILTER_PTP_V2_L4_EVENT) ptpv2-l2-event (HWTSTAMP_FILTER_PTP_V2_L2_EVENT) ptp4l logs: Jun 3 10:12:37.000 ldalgc01 ptp4l[8290]: ptp4l[601302.197]: selected local clock e43d1a.fffe.acab65 as best master Jun 3 10:12:37.000 ldalgc01 ptp4l: [601302.197] selected local clock e43d1a.fffe.acab65 as best master Jun 3 10:12:38.000 ldalgc01 phc2sys: [601303.026] Waiting for ptp4l... Jun 3 10:12:39.000 ldalgc01 phc2sys: [601304.027] Waiting for ptp4l... Jun 3 10:12:40.000 ldalgc01 phc2sys: [601305.029] Waiting for ptp4l... tcpdump snippet: 10:12:39.381908 IP gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: UDP, length 54 10:12:39.398091 IP gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: UDP, length 54 10:12:39.404572 IP gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: UDP, length 54 10:12:39.406287 IP gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: UDP, length 54 10:12:39.486361 IP gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: UDP, length 54 10:12:39.499305 IP gsdgmca01-ptp0.prod.instinet.com.ptp-general > 224.0.1.129.ptp-general: UDP, length 54 ptp4l.conf (which works OK on the Intel i350): [global] # # Default Data Set # twoStepFlag 1 #slaveOnly 0 slaveOnly 1 priority1 128 priority2 128 domainNumber 0 #utc_offset 37 clockClass 248 clockAccuracy 0xFE offsetScaledLogVariance 0xFFFF free_running 0 freq_est_interval 1 dscp_event 0 dscp_general 0 dataset_comparison ieee1588 G.8275.defaultDS.localPriority 128 # # Port Data Set # logAnnounceInterval 1 logSyncInterval 0 logMinDelayReqInterval 0 logMinPdelayReqInterval 0 announceReceiptTimeout 3 syncReceiptTimeout 0 delayAsymmetry 0 fault_reset_interval 4 neighborPropDelayThresh 20000000 masterOnly 0 G.8275.portDS.localPriority 128 # # Run time options # assume_two_step 0 logging_level 6 path_trace_enabled 0 follow_up_info 0 #hybrid_e2e 0 hybrid_e2e 1 inhibit_multicast_service 0 net_sync_monitor 0 tc_spanning_tree 0 tx_timestamp_timeout 1 unicast_listen 0 unicast_master_table 0 unicast_req_duration 3600 use_syslog 1 #verbose 0 verbose 1 summary_interval 0 kernel_leap 1 check_fup_sync 0 # # Servo Options # pi_proportional_const 0.0 pi_integral_const 0.0 pi_proportional_scale 0.0 pi_proportional_exponent -0.3 pi_proportional_norm_max 0.7 pi_integral_scale 0.0 pi_integral_exponent 0.4 pi_integral_norm_max 0.3 step_threshold 0.0 first_step_threshold 0.00002 max_frequency 900000000 #clock_servo pi clock_servo linreg sanity_freq_limit 200000000 ntpshm_segment 0 # # Transport options # transportSpecific 0x0 ptp_dst_mac 01:1B:19:00:00:00 p2p_dst_mac 01:80:C2:00:00:0E #udp_ttl 1 udp_ttl 32 udp6_scope 0x0E uds_address /var/run/ptp4l # # Default interface options # clock_type OC network_transport UDPv4 delay_mechanism E2E time_stamping hardware tsproc_mode filter delay_filter moving_median delay_filter_length 10 egressLatency 0 ingressLatency 0 boundary_clock_jbod 0 # # Clock description # productDescription ;; revisionData ;; manufacturerIdentity 00:00:00 userDescription ; timeSource 0xA0 Thanks and sorry again for the previous formatting. J |
|
From: Miroslav L. <mli...@re...> - 2022-06-07 08:17:53
|
On Mon, Jun 06, 2022 at 04:10:19PM +0100, Jay Shardlow wrote: > Hello all, after a bit of advice/help. I've got some CentOS 7 servers and I > need to get them running ptp4l (from linuxptp) as clients, with the > grandmaster being a Meinberg appliance. One with an Intel i350 works fine. > Another with a Broadcom 5720 does not. Both servers are connected to the > same network for their PTP. I suspect I know why the Broadcom doesn't work, > but I was wondering if anyone could point me in the right direction on how > to get it to do so. Do the servers have the same configuration? Usually it's a firewall issue. If the issue was with missing timestamps, ptp4l would print an error. In my experience, BCM5720 works fine with UDPv4 PTP. -- Miroslav Lichvar |
|
From: Jay S. <li...@sh...> - 2022-06-07 09:13:44
|
On 07/06/2022 09:17, Miroslav Lichvar wrote: > On Mon, Jun 06, 2022 at 04:10:19PM +0100, Jay Shardlow wrote: >> Hello all, after a bit of advice/help. I've got some CentOS 7 servers and I >> need to get them running ptp4l (from linuxptp) as clients, with the >> grandmaster being a Meinberg appliance. One with an Intel i350 works fine. >> Another with a Broadcom 5720 does not. Both servers are connected to the >> same network for their PTP. I suspect I know why the Broadcom doesn't work, >> but I was wondering if anyone could point me in the right direction on how >> to get it to do so. > > Do the servers have the same configuration? Usually it's a firewall > issue. If the issue was with missing timestamps, ptp4l would print an > error. In my experience, BCM5720 works fine with UDPv4 PTP. > They're not identical, but they're both CentOS 7.9 running linuxptp 2.0.2. No firewall on the Broadcom server. To my untrained eye the only real difference is the Hardware Recieve Filter Modes. Elsewhere in our environment we have combos of Solarflare/sfptpd, Intel i350/ptpd, Intel X520/ptpd which all work fine, though they either support HWTSTAMP_FILTER_ALL or at least HWTSTAMP_FILTER_PTP_V2_EVENT. Though I've literally just read that ptp4l prefers to have HWTSTAMP_FILTER_PTP_V2_EVENT, but will fall back to HWTSTAMP_FILTER_PTP_V2_L4_EVENT or HWTSTAMP_FILTER_PTP_V2_L4_EVENT (the latter two available on the Broadcom). At this point I'm slightly out of my depth and out of ideas. The Broadcom obviously does work (or they wouldn't advertise it as such), so must be something with my environemnt, but I have no idea what. J |
|
From: Miroslav L. <mli...@re...> - 2022-06-07 09:28:12
|
On Tue, Jun 07, 2022 at 10:13:31AM +0100, Jay Shardlow wrote: > At this point I'm slightly out of my depth and out of ideas. The Broadcom > obviously does work (or they wouldn't advertise it as such), so must be > something with my environemnt, but I have no idea what. I doubt it's HW related. You can try software timestamping (-S) to rule that out. In the ptp4l log there should be a message about the remote clock. If there is only the local clock, ptp4l is not getting the annoucement messages. That doesn't require any timestamping. You can verify ptp4l is receiving messages from network with strace like this: strace -erecvmsg ptp4l -m -q -i eth0 -s -- Miroslav Lichvar |
|
From: Jay S. <li...@sh...> - 2022-06-07 13:29:17
|
On 07/06/2022 10:28, Miroslav Lichvar wrote:
> On Tue, Jun 07, 2022 at 10:13:31AM +0100, Jay Shardlow wrote:
>> At this point I'm slightly out of my depth and out of ideas. The Broadcom
>> obviously does work (or they wouldn't advertise it as such), so must be
>> something with my environemnt, but I have no idea what.
>
> I doubt it's HW related. You can try software timestamping (-S) to
> rule that out. In the ptp4l log there should be a message about the
> remote clock. If there is only the local clock, ptp4l is not getting
> the annoucement messages. That doesn't require any timestamping.
>
> You can verify ptp4l is receiving messages from network with strace
> like this:
>
> strace -erecvmsg ptp4l -m -q -i eth0 -s
>
Thank you.
No joy on the software timestamping:
# ptp4l -i lan2 -s -S -m
ptp4l[961566.609]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[961566.609]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[961572.642]: selected local clock e43d1a.fffe.acab65 as best master
ptp4l[961579.274]: selected local clock e43d1a.fffe.acab65 as best master
ptp4l[961587.069]: selected local clock e43d1a.fffe.acab65 as best master
And for the strace, on the working Intel I can see lots of this with the
IP of our master:
recvmsg(14, {msg_name={sa_family=AF_INET, sin_port=htons(320),
sin_addr=inet_addr("IP_REDACTED")}, msg_namelen=128->16,
msg_iov=[{iov_base="\t\2\0006\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\354Fp\377\376\0\277\4\0\1\34\246"...,
iov_len=1500}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 54
But on the Broadcom I don't see any such message from the same IP. So
looks like ptp4l is not getting such annoucements as you suspected.
I've asked the networks team to check the switchport for Broadcom box is
set up the same as Intel box, which they have confirmed it is. Meanwhile
I've asked Meinberg for a bit of help/advice on the appliance config in
case it's something at that end.
J
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X