Thread: [Linuxptp-users] Issue with initial high jitter(Path Delay Variation)
PTP IEEE 1588 stack for Linux
Brought to you by:
rcochran
From: Sujatha G. <suj...@gm...> - 2016-02-15 08:46:22
|
Hi All, I am facing issue with ptp4l application where the SLAVE to MASTER Path Delay Variation is very high for initial few seconds and it is taking ~62 seconds for the slave clock to get stabilized. This issue is seen 4 out of 5 times when the system is executed with the same software. Our target is to have the slave clock get stabilized with in 5 seconds from s0 state and the jitter should be less than 100 ns. In Success case(Logs provided below) - slave clock is taking 186 - 173 = 13 seconds for stabilization - Jitter is with 100ns after stabilization In Failire case (Logs provided below) - slave clock is taking 340 - 278 = 62 seconds for stabilization - jitter is maintained below 100ns after stabilization but it is very very high compared to previous case during the stabilization stage The issues observed in Failure case logs are , - the Master time stamp and slave time stamp difference is very high - at s1, adjtime called to adjust the slave time, but after that the time difference is more - adjtime is called only once and after that the time adjustment is happening only using adjfreq - the maximum adjfreq can happen is 32767999, hence it is trying to adjust the frequency using 32767999 for 29 times and then gradually frequency value is decreased to adjust the slave time and for this it is taking 62 seconds - This behaviour is observed 4 out of 5 times and suddenly for 1 time the systems starts behaving correctly where I got the the success logs. - I have tried to set the pi_offset_const in default.cfg to 1.0 thinking that if the time difference between MASTER and SLAVE goes beyond 1 second adjtime might get called. But the value changed in default.cfg does not get reflected in the software. I have verified it in pi.c file by printing s->max_offset value , This value is always 0 - Based on the logs below, the time difference between MASTER and SLAVE is not going beyond 1 second so I experimented the pi_offset_const with 0.5 as well but could not see any difference. Can somebody help me to solve this issue. I have been struggling with this issue for a long time and currently exhausted with all the possible options.Any pointers on this issue could be a great help to me. Thank you all in advance. *Success Logs:* ptp4l[167.314]: driver changed our HWTSTAMP options ptp4l[167.320]: tx_type 1 not 1 ptp4l[167.324]: rx_filter 1 not 12 ptp4l[167.329]: port 1: INITIALIZING to LISTENING on INITIALIZE ptp4l[167.340]: port 0: INITIALIZING to LISTENING on INITIALIZE ptp4l[168.503]: port 1: new foreign master 001ec0.fffe.8614b5-1 ptp4l[172.512]: selected best master clock 001ec0.fffe.8614b5 ptp4l[172.527]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE ptp4l[173.676]: master offset -1474359190506508 s0 freq +0 path delay 1391 ptp4l[174.677]: master offset -1474359190525503 s1 freq -18994 path delay 3347 ptp4l[175.676]: master offset 1686 s2 freq -17308 path delay 3347 ptp4l[175.681]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED ptp4l[176.677]: master offset 3777 s2 freq -14712 path delay 1391 ptp4l[177.677]: master offset 1412 s2 freq -15943 path delay 1425 ptp4l[178.677]: master offset 208 s2 freq -16724 path delay 1425 ptp4l[179.677]: master offset -209 s2 freq -17078 path delay 1391 ptp4l[180.677]: master offset 228 s2 freq -16704 path delay 886 ptp4l[181.678]: master offset -210 s2 freq -17074 path delay 886 ptp4l[182.677]: master offset 222 s2 freq -16705 path delay 381 ptp4l[183.677]: master offset -293 s2 freq -17153 path delay 458 ptp4l[184.678]: master offset -370 s2 freq -17318 path delay 458 ptp4l[185.678]: master offset -171 s2 freq -17230 path delay 381 ptp4l[186.678]: master offset -38 s2 freq -17148 path delay 370 ptp4l[187.678]: master offset -92 s2 freq -17214 path delay 354 ptp4l[188.678]: master offset 40 s2 freq -17109 path delay 354 ptp4l[189.678]: master offset -31 s2 freq -17168 path delay 349 ptp4l[190.678]: master offset 103 s2 freq -17044 path delay 346 ptp4l[191.678]: master offset 26 s2 freq -17090 path delay 346 ptp4l[192.678]: master offset -35 s2 freq -17143 path delay 344 ptp4l[193.678]: master offset -102 s2 freq -17220 path delay 344 ptp4l[194.678]: master offset 11 s2 freq -17138 path delay 344 ptp4l[195.682]: master offset -51 s2 freq -17197 path delay 340 ptp4l[196.678]: master offset 62 s2 freq -17099 path delay 340 ptp4l[197.678]: master offset 14 s2 freq -17128 path delay 337 ptp4l[198.678]: master offset -66 s2 freq -17204 path delay 337 ptp4l[199.678]: master offset 72 s2 freq -17086 path delay 337 ptp4l[200.678]: master offset 5 s2 freq -17131 path delay 339 ptp4l[201.680]: master offset -54 s2 freq -17189 path delay 339 *Failure Logs:* ptp4l[270.981]: selected /dev/ptp0 as PTP clock ptp4l[271.014]: driver changed our HWTSTAMP options ptp4l[271.020]: tx_type 1 not 1 ptp4l[271.024]: rx_filter 1 not 12 ptp4l[271.028]: port 1: INITIALIZING to LISTENING on INITIALIZE ptp4l[271.040]: port 0: INITIALIZING to LISTENING on INITIALIZE ptp4l[272.323]: port 1: new foreign master 001ec0.fffe.8614b5-1 ptp4l[276.323]: selected best master clock 001ec0.fffe.8614b5 ptp4l[276.328]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE ptp4l[278.496]: master offset -1475113437970290 s0 freq +0 path delay 7176 ptp4l[279.496]: master offset -1475113437987402 s1 freq -17112 path delay 7176 ptp4l[280.496]: master offset -1000000284 s2 freq -32767999 path delay 7176 ptp4l[280.501]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED ptp4l[281.501]: master offset -967516976 s2 freq -32767999 path delay 5595 ptp4l[282.497]: master offset -932832166 s2 freq -32767999 path delay -1927882 ptp4l[283.497]: master offset -898147308 s2 freq -32767999 path delay -3861360 ptp4l[284.497]: master offset -865395833 s2 freq -32767999 path delay -3861360 ptp4l[285.497]: master offset -831350054 s2 freq -32767999 path delay -5155742 ptp4l[286.497]: master offset -799893170 s2 freq -32767999 path delay -3861360 ptp4l[287.497]: master offset -765847415 s2 freq -32767999 path delay -5155742 ptp4l[288.497]: master offset -733096073 s2 freq -32767999 path delay -5155742 ptp4l[289.497]: master offset -698230049 s2 freq -32767999 path delay -7270253 ptp4l[290.498]: master offset -665478434 s2 freq -32767999 path delay -7270253 ptp4l[291.498]: master offset -632727061 s2 freq -32767999 path delay -7270253 ptp4l[292.498]: master offset -598665729 s2 freq -32767999 path delay -8580243 ptp4l[293.498]: master offset -564280251 s2 freq -32767999 path delay -10214326 ptp4l[294.498]: master offset -531529599 s2 freq -32767999 path delay -10213455 ptp4l[295.498]: master offset -497634061 s2 freq -32767999 path delay -11357800 ptp4l[296.498]: master offset -464882705 s2 freq -32767999 path delay -11357800 ptp4l[297.498]: master offset -432131187 s2 freq -32767999 path delay -11357800 ptp4l[298.498]: master offset -399380745 s2 freq -32767999 path delay -11356929 ptp4l[299.498]: master offset -366628279 s2 freq -32767999 path delay -11357800 ptp4l[300.498]: master offset -333876738 s2 freq -32767999 path delay -11357800 ptp4l[301.498]: master offset -301125423 s2 freq -32767999 path delay -11357800 ptp4l[302.498]: master offset -268374981 s2 freq -32767999 path delay -11356929 ptp4l[303.498]: master offset -235623646 s2 freq -32767999 path delay -11356929 ptp4l[304.498]: master offset -202794969 s2 freq -32767999 path delay -11434176 ptp4l[305.498]: master offset -170043447 s2 freq -32767999 path delay -11434176 ptp4l[306.498]: master offset -137787552 s2 freq -32767999 path delay -10938890 ptp4l[307.502]: master offset -105035853 s2 freq -32767999 path delay -10938890 ptp4l[308.498]: master offset -72284835 s2 freq -32767999 path delay -10938890 ptp4l[309.498]: master offset -39533393 s2 freq -32767999 path delay -10938890 ptp4l[310.498]: master offset -6782000 s2 freq -6799112 path delay -10938890 ptp4l[311.498]: master offset 240469 s2 freq -1811243 path delay -10938890 ptp4l[312.498]: master offset 607326 s2 freq -1372245 path delay -9465521 ptp4l[313.502]: master offset -82486 s2 freq -1879859 path delay -7416620 ptp4l[314.499]: master offset 1773547 s2 freq -48572 path delay -7416620 ptp4l[315.499]: master offset -306599 s2 freq -1596654 path delay -5288386 ptp4l[316.498]: master offset 452990 s2 freq -929044 path delay -4482990 ptp4l[317.498]: master offset 1379304 s2 freq +133167 path delay -4482990 ptp4l[318.499]: master offset -1209889 s2 freq -2042235 path delay -2033615 ptp4l[319.499]: master offset -570670 s2 freq -1765983 path delay -667937 ptp4l[320.499]: master offset 942311 s2 freq -424203 path delay -429615 ptp4l[321.499]: master offset 1361863 s2 freq +278042 path delay -429615 ptp4l[322.499]: master offset 877075 s2 freq +201813 path delay -233274 ptp4l[323.499]: master offset 657429 s2 freq +245290 path delay -233274 ptp4l[324.499]: master offset 256206 s2 freq +41295 path delay -94051 ptp4l[325.499]: master offset 101446 s2 freq -36603 path delay 482 ptp4l[326.499]: master offset 116561 s2 freq +8946 path delay 3951 ptp4l[327.499]: master offset 91065 s2 freq +18418 path delay 3951 ptp4l[328.499]: master offset 53869 s2 freq +8542 path delay 5785 ptp4l[329.499]: master offset 26391 s2 freq -2775 path delay 7659 ptp4l[330.499]: master offset 16293 s2 freq -4956 path delay 3184 ptp4l[331.499]: master offset 2694 s2 freq -13667 path delay 4398 ptp4l[332.500]: master offset -982 s2 freq -16535 path delay 4398 ptp4l[333.500]: master offset 856 s2 freq -14992 path delay 1800 ptp4l[334.500]: master offset -614 s2 freq -16205 path delay 1024 ptp4l[335.500]: master offset -1311 s2 freq -17086 path delay 585 ptp4l[336.500]: master offset -1493 s2 freq -17661 path delay 585 ptp4l[337.500]: master offset -1094 s2 freq -17710 path delay 585 ptp4l[338.500]: master offset -684 s2 freq -17628 path delay 585 ptp4l[339.503]: master offset -178 s2 freq -17328 path delay 494 ptp4l[340.500]: master offset -45 s2 freq -17248 path delay 399 ptp4l[341.500]: master offset -16 s2 freq -17233 path delay 399 ptp4l[342.500]: master offset 32 s2 freq -17189 path delay 362 ptp4l[343.500]: master offset 112 s2 freq -17100 path delay 328 ptp4l[344.500]: master offset -42 s2 freq -17220 path delay 328 ptp4l[345.499]: master offset -26 s2 freq -17217 path delay 328 ptp4l[346.500]: master offset 12 s2 freq -17187 path delay 328 ptp4l[347.500]: master offset 42 s2 freq -17153 path delay 328 ptp4l[348.500]: master offset -138 s2 freq -17320 path delay 333 ptp4l[349.500]: master offset -102 s2 freq -17326 path delay 333 ptp4l[350.500]: master offset -73 s2 freq -17327 path delay 336 ptp4l[351.500]: master offset -29 s2 freq -17305 path delay 337 ptp4l[352.500]: master offset 7 s2 freq -17278 path delay 336 ptp4l[353.500]: master offset 39 s2 freq -17244 path delay 334 ptp4l[354.500]: master offset 73 s2 freq -17198 path delay 332 ptp4l[355.500]: master offset 104 s2 freq -17145 path delay 330 ptp4l[356.500]: master offset -41 s2 freq -17259 path delay 330 ptp4l[357.500]: master offset -5 s2 freq -17235 path delay 330 Best Regards, Sujatha |
From: Miroslav L. <mli...@re...> - 2016-02-15 10:06:58
|
On Mon, Feb 15, 2016 at 02:16:13PM +0530, Sujatha Guguloth wrote: > In Failire case (Logs provided below) > - slave clock is taking 340 - 278 = 62 seconds for stabilization > - jitter is maintained below 100ns after stabilization but it is very very > high compared to previous case during the stabilization stage It looks like the initial step is off by one second for some reason. It could be a driver issue. What HW and kernel version do you use? As a workaround you could try setting step_threshold to 0.1 for example and allow ptp4l to correct that one second error by stepping instead of slewing. > *Failure Logs:* > ptp4l[270.981]: selected /dev/ptp0 as PTP clock > ptp4l[271.014]: driver changed our HWTSTAMP options > ptp4l[271.020]: tx_type 1 not 1 > ptp4l[271.024]: rx_filter 1 not 12 > ptp4l[271.028]: port 1: INITIALIZING to LISTENING on INITIALIZE > ptp4l[271.040]: port 0: INITIALIZING to LISTENING on INITIALIZE > ptp4l[272.323]: port 1: new foreign master 001ec0.fffe.8614b5-1 > ptp4l[276.323]: selected best master clock 001ec0.fffe.8614b5 > ptp4l[276.328]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE > ptp4l[278.496]: master offset -1475113437970290 s0 freq +0 path delay > 7176 > ptp4l[279.496]: master offset -1475113437987402 s1 freq -17112 path delay > 7176 > ptp4l[280.496]: master offset -1000000284 s2 freq -32767999 path delay > 7176 > ptp4l[280.501]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED > ptp4l[281.501]: master offset -967516976 s2 freq -32767999 path delay -- Miroslav Lichvar |
From: Sujatha G. <suj...@gm...> - 2016-02-15 12:05:08
|
Hi Miroslav Lichvar, Thank you for quick replay. We are using Linux 3.19 version and the Hardware is a proprietary FPGA Hardware. I will try your suggestion. Thank you. Best Regards, Sujatha On Mon, Feb 15, 2016 at 3:36 PM, Miroslav Lichvar <mli...@re...> wrote: > On Mon, Feb 15, 2016 at 02:16:13PM +0530, Sujatha Guguloth wrote: > > In Failire case (Logs provided below) > > - slave clock is taking 340 - 278 = 62 seconds for stabilization > > - jitter is maintained below 100ns after stabilization but it is very > very > > high compared to previous case during the stabilization stage > > It looks like the initial step is off by one second for some reason. > It could be a driver issue. What HW and kernel version do you use? > > As a workaround you could try setting step_threshold to 0.1 for > example and allow ptp4l to correct that one second error by stepping > instead of slewing. > > > *Failure Logs:* > > ptp4l[270.981]: selected /dev/ptp0 as PTP clock > > ptp4l[271.014]: driver changed our HWTSTAMP options > > ptp4l[271.020]: tx_type 1 not 1 > > ptp4l[271.024]: rx_filter 1 not 12 > > ptp4l[271.028]: port 1: INITIALIZING to LISTENING on INITIALIZE > > ptp4l[271.040]: port 0: INITIALIZING to LISTENING on INITIALIZE > > ptp4l[272.323]: port 1: new foreign master 001ec0.fffe.8614b5-1 > > ptp4l[276.323]: selected best master clock 001ec0.fffe.8614b5 > > ptp4l[276.328]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE > > ptp4l[278.496]: master offset -1475113437970290 s0 freq +0 path > delay > > 7176 > > ptp4l[279.496]: master offset -1475113437987402 s1 freq -17112 path > delay > > 7176 > > ptp4l[280.496]: master offset -1000000284 s2 freq -32767999 path delay > > 7176 > > ptp4l[280.501]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED > > ptp4l[281.501]: master offset -967516976 s2 freq -32767999 path delay > > -- > Miroslav Lichvar > |
From: Sujatha G. <suj...@gm...> - 2016-02-18 10:33:17
|
Hi Richard, to avoid the confusion, I am putting both the functions below here completely. Please have a look. static int xlnx_ptp_adjfreq(struct ptp_clock_info *ptp, s32 ppb) { struct xlnx_ptp_timer *timer = container_of(ptp, struct xlnx_ptp_timer, ptp_clock_info); int neg_adj = 0; u64 freq = 0; u32 diff = 0; u32 incval = 0; /* External clock source Frequency is 200MHz */ /* adjust the RTC Increment Control register by (1/200MHZ = 5ns) */ /* resolution of the increment register is 1/1048576 */ /* hence 5 * 1048576 = 0x500000 - value to be written to the register */ incval = 0x500000; if (ppb < 0) { neg_adj = 1; ppb = -ppb; } freq = incval; freq *= ppb; diff = div_u64(freq, 1000000000ULL); incval = neg_adj ? (incval - diff) : (incval + diff); /* Write the adjusted value to the RTC Increment Value Control Register */ /* Writing the adjustment value to this register intern adjusts the nanoseconds register value by the hw */ *out_be32((timer->baseaddr + XTIMER1588_RTC_INCREMENT), incval);* return 0; } static int xlnx_ptp_adjtime(struct ptp_clock_info *ptp, s64 delta) { unsigned long flags = 0; struct xlnx_ptp_timer *timer = container_of(ptp, struct xlnx_ptp_timer, ptp_clock_info); struct timespec offset, *then = ns_to_timespec(delta);* spin_lock_irqsave(&timer->reg_lock, flags); /* Read the seconds and nano seconds from the RTC Seconds and RTC Nano Seconds Registers */ xlnx_rtc_offset_read(timer, &offset); /* set the current offset values */ *offset = then;* /* Write the seconds and nano seconds to the RTC Seconds and RTC Nano Seconds Registers */ xlnx_rtc_offset_write(timer, (const struct timespec *)*&offset*); spin_unlock_irqrestore(&timer->reg_lock, flags); return 0; } Best Regards, Sujatha On Thu, Feb 18, 2016 at 3:33 PM, Sujatha Guguloth <suj...@gm...> wrote: > Hi Richard, > > I am writing the incval to the register in adjfreq after the calculation.I > had written that in the comments and actual code is below. > out_be32((timer->baseaddr + XTIMER1588_RTC_INCREMENT), incval); > > > In adjtime: delta is passed to ns_to_timespec > then = ns_to_timespec(delta) > offset = then; /* then is copied to offset */ > xlnx_rtc_offset_write(timer, (const struct timespec *)&offset); /* This > offset is written to the register */ > > > Best Regards, > Sujatha > > > On Thu, Feb 18, 2016 at 2:59 PM, Richard Cochran <ric...@gm... > > wrote: > >> Wow, your code is totally broken... >> >> On Thu, Feb 18, 2016 at 12:04:22PM +0530, Sujatha Guguloth wrote: >> > static int xlnx_ptp_adjfreq(struct ptp_clock_info *ptp, s32 ppb) >> > { >> > struct xlnx_ptp_timer *timer = container_of(ptp, struct >> > xlnx_ptp_timer, >> > ptp_clock_info); >> > >> > int neg_adj = 0; >> > u64 freq = 0; >> > u32 diff = 0; >> > u32 incval = 0; >> > >> > /* External clock source Frequency is 200MHz which is driving >> > the timer */ >> > /* adjust the RTC Increment Control register by (1/200MHZ = >> 5ns) */ >> > /* resolution of the increment register is 1/1048576 >> */ >> > /* hence 5 * 1048576 = 0x500000 - value to be written to the >> > register */ >> > incval = 0x500000; >> > >> > if (ppb < 0) { >> > neg_adj = 1; >> > ppb = -ppb; >> > } >> > >> > freq = incval; >> > freq *= ppb; >> > diff = div_u64(freq, 1000000000ULL); >> > >> > incval = neg_adj ? (incval - diff) : (incval + diff); >> > >> > /* Write the adjusted value to the *RTC Increment Value Control >> > Register* */ >> > /* Writing the adjustment value to this register intern adjusts >> the >> > nanoseconds register value by the hw */ >> >> You calculate 'incval' and then do nothing with it. >> >> > return 0; >> > } >> > >> > >> > static int xlnx_ptp_adjtime(struct ptp_clock_info *ptp, s64 delta) >> > { >> > unsigned long flags = 0; >> > struct xlnx_ptp_timer *timer = container_of(ptp, struct >> > xlnx_ptp_timer, >> > ptp_clock_info); >> > struct timespec offset, then = ns_to_timespec(delta); >> > >> > spin_lock_irqsave(&timer->reg_lock, flags); >> > >> > /* Read the seconds and nano seconds from the RTC Seconds and >> RTC >> > Nano Seconds Registers */ >> > xlnx_rtc_offset_read(timer, &offset); >> > >> > /* set the current offset values */ >> > offset = then; >> >> You are supposed to add the delta. >> >> > /* Write the seconds and nano seconds to the *RTC Seconds* and >> *RTC >> > Nano Seconds* Registers */ >> > xlnx_rtc_offset_write(timer, (const struct timespec *)&offset); >> > >> > spin_unlock_irqrestore(&timer->reg_lock, flags); >> > return 0; >> > } >> >> Good luck, >> Richard >> > > |
From: Richard C. <ric...@gm...> - 2016-02-18 13:15:01
|
On Thu, Feb 18, 2016 at 04:03:09PM +0530, Sujatha Guguloth wrote: > /* set the current offset values */ > *offset = then;* This is still wrong. The delta value is, well, a delta. You must add it to the current offset. I hate to say it, but this list about the linuxptp software, not about proprietary driver development. We pointed you in the right direction, saying that the problem lies in the driver and/or fpga. You will have to fix it yourself. Sorry, Richard |
From: Sujatha G. <suj...@gm...> - 2016-02-19 06:49:50
|
Thank you for the support Richard & all. On Thu, Feb 18, 2016 at 6:44 PM, Richard Cochran <ric...@gm...> wrote: > On Thu, Feb 18, 2016 at 04:03:09PM +0530, Sujatha Guguloth wrote: > > /* set the current offset values */ > > *offset = then;* > > This is still wrong. The delta value is, well, a delta. You must add > it to the current offset. > > I hate to say it, but this list about the linuxptp software, not about > proprietary driver development. We pointed you in the right > direction, saying that the problem lies in the driver and/or fpga. > You will have to fix it yourself. > > Sorry, > Richard > |
From: Sujatha G. <suj...@gm...> - 2016-02-16 05:12:35
|
Hi Miroslav Lichvar, I tried your suggestion by setting the pi_offset_const(set_threshold) to 0.1 but, with that the slave clock is not getting stabilized and and entering into s2 state at all, it keeps shuffling between s0 and s1 states It throws the error as "clock jumped backward or running slower than expected!" some times and "clock jumped forward or running faster than expected!" some times Below are the the logs: ptp4l[345.444]: selected /dev/ptp0 as PTP clock ptp4l[345.484]: driver changed our HWTSTAMP options ptp4l[345.489]: tx_type 1 not 1 ptp4l[345.494]: rx_filter 1 not 12 ptp4l[345.498]: port 1: INITIALIZING to LISTENING on INITIALIZE ptp4l[345.510]: port 0: INITIALIZING to LISTENING on INITIALIZE ptp4l[346.649]: port 1: new foreign master 001ec0.fffe.8614b5-1 ptp4l[350.649]: selected best master clock 001ec0.fffe.8614b5 ptp4l[350.654]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE ptp4l[352.826]: master offset -1553865481389947 s0 freq +0 path delay 5128 ptp4l[353.827]: master offset -1553865481406512 s1 freq -16565 path delay 5128 ptp4l[354.827]: master offset -1000000463 s0 freq -16565 path delay 5128 ptp4l[355.827]: master offset -999998257 s0 freq -16565 path delay 2769 ptp4l[356.828]: master offset -999998398 s1 freq -16706 path delay 2769 ptp4l[357.822]: clockcheck: clock jumped backward or running slower than expected! ptp4l[357.832]: master offset -481406489 s0 freq -16706 path delay 2769 ptp4l[358.829]: master offset -481404091 s1 freq -14308 path delay 411 ptp4l[359.827]: master offset -1000000600 s0 freq -14308 path delay 411 ptp4l[360.822]: clockcheck: clock jumped backward or running slower than expected! ptp4l[360.832]: master offset -1000003077 s0 freq -14308 path delay 637 ptp4l[361.829]: master offset -1000005315 s1 freq -16546 path delay 637 ptp4l[362.822]: clockcheck: clock jumped forward or running faster than expected! ptp4l[362.832]: master offset 518595705 s0 freq -16546 path delay 637 ptp4l[363.829]: master offset 518595591 s1 freq -16660 path delay 591 ptp4l[364.827]: master offset -1000005266 s0 freq -16660 path delay 591 ptp4l[365.822]: clockcheck: clock jumped backward or running slower than expected! ptp4l[365.832]: master offset -1000005230 s0 freq -16660 path delay 591 ptp4l[366.829]: master offset -1000005072 s1 freq -16502 path delay 478 ptp4l[367.822]: clockcheck: clock jumped forward or running faster than expected! ptp4l[367.832]: master offset 518595452 s0 freq -16502 path delay 478 ptp4l[368.829]: master offset 518595382 s1 freq -16572 path delay 401 ptp4l[369.828]: master offset -1000005217 s0 freq -16572 path delay 401 ptp4l[370.822]: clockcheck: clock jumped backward or running slower than expected! ptp4l[370.832]: master offset -1000005381 s0 freq -16572 path delay 401 ptp4l[371.829]: master offset -1000005508 s1 freq -16699 path delay 380 ptp4l[372.822]: clockcheck: clock jumped forward or running faster than expected! ptp4l[372.833]: master offset 518595424 s0 freq -16699 path delay 380 ptp4l[373.829]: master offset 518595461 s1 freq -16662 path delay 380 ptp4l[374.828]: master offset -1000005458 s0 freq -16662 path delay 380 ptp4l[375.822]: clockcheck: clock jumped backward or running slower than expected! ptp4l[375.834]: master offset -1000005401 s0 freq -16662 path delay 365 ptp4l[376.829]: master offset -1000005341 s1 freq -16602 path delay 352 Best Regards, Sujatha On Mon, Feb 15, 2016 at 5:34 PM, Sujatha Guguloth <suj...@gm...> wrote: > Hi Miroslav Lichvar, > > Thank you for quick replay. > > We are using Linux 3.19 version and the Hardware is a proprietary FPGA > Hardware. > > I will try your suggestion. > > Thank you. > > Best Regards, > Sujatha > > On Mon, Feb 15, 2016 at 3:36 PM, Miroslav Lichvar <mli...@re...> > wrote: > >> On Mon, Feb 15, 2016 at 02:16:13PM +0530, Sujatha Guguloth wrote: >> > In Failire case (Logs provided below) >> > - slave clock is taking 340 - 278 = 62 seconds for stabilization >> > - jitter is maintained below 100ns after stabilization but it is very >> very >> > high compared to previous case during the stabilization stage >> >> It looks like the initial step is off by one second for some reason. >> It could be a driver issue. What HW and kernel version do you use? >> >> As a workaround you could try setting step_threshold to 0.1 for >> example and allow ptp4l to correct that one second error by stepping >> instead of slewing. >> >> > *Failure Logs:* >> > ptp4l[270.981]: selected /dev/ptp0 as PTP clock >> > ptp4l[271.014]: driver changed our HWTSTAMP options >> > ptp4l[271.020]: tx_type 1 not 1 >> > ptp4l[271.024]: rx_filter 1 not 12 >> > ptp4l[271.028]: port 1: INITIALIZING to LISTENING on INITIALIZE >> > ptp4l[271.040]: port 0: INITIALIZING to LISTENING on INITIALIZE >> > ptp4l[272.323]: port 1: new foreign master 001ec0.fffe.8614b5-1 >> > ptp4l[276.323]: selected best master clock 001ec0.fffe.8614b5 >> > ptp4l[276.328]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE >> > ptp4l[278.496]: master offset -1475113437970290 s0 freq +0 path >> delay >> > 7176 >> > ptp4l[279.496]: master offset -1475113437987402 s1 freq -17112 path >> delay >> > 7176 >> > ptp4l[280.496]: master offset -1000000284 s2 freq -32767999 path delay >> > 7176 >> > ptp4l[280.501]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED >> > ptp4l[281.501]: master offset -967516976 s2 freq -32767999 path delay >> >> -- >> Miroslav Lichvar >> > > |
From: Miroslav L. <mli...@re...> - 2016-02-16 10:46:15
|
On Tue, Feb 16, 2016 at 10:42:27AM +0530, Sujatha Guguloth wrote: > Hi Miroslav Lichvar, > > I tried your suggestion by setting the pi_offset_const(set_threshold) to > 0.1 > but, with that the slave clock is not getting stabilized and and entering > into s2 state at all, it keeps shuffling between s0 and s1 states > It throws the error as "clock jumped backward or running slower than > expected!" some times and "clock jumped forward or running faster than > expected!" some times I think that means the clock stepping is completely broken. The driver/HW needs to be fixed. -- Miroslav Lichvar |
From: Sujatha G. <suj...@gm...> - 2016-02-16 10:51:06
|
Hi Miroslav Lichvar, What aspects of driver/HW needs to be fixed. could you please elaborate more on this. Best Regards, Sujatha On Tue, Feb 16, 2016 at 4:16 PM, Miroslav Lichvar <mli...@re...> wrote: > On Tue, Feb 16, 2016 at 10:42:27AM +0530, Sujatha Guguloth wrote: > > Hi Miroslav Lichvar, > > > > I tried your suggestion by setting the pi_offset_const(set_threshold) to > > 0.1 > > but, with that the slave clock is not getting stabilized and and entering > > into s2 state at all, it keeps shuffling between s0 and s1 states > > It throws the error as "clock jumped backward or running slower than > > expected!" some times and "clock jumped forward or running faster than > > expected!" some times > > I think that means the clock stepping is completely broken. The > driver/HW needs to be fixed. > > -- > Miroslav Lichvar > |
From: Miroslav L. <mli...@re...> - 2016-02-16 11:05:45
|
On Tue, Feb 16, 2016 at 04:20:56PM +0530, Sujatha Guguloth wrote: > Hi Miroslav Lichvar, > > What aspects of driver/HW needs to be fixed. could you please elaborate > more on this. The ADJ_SETOFFSET mode needs to be fixed. When ptp4l tries to step the clock, it seems the actual step is up to one second off. Maybe some integer operation with sub-second fields overflows and the second field is not adjusted accordingly. Just a guess. -- Miroslav Lichvar |
From: Keller, J. E <jac...@in...> - 2016-02-16 19:57:03
|
Hi, On Tue, 2016-02-16 at 16:20 +0530, Sujatha Guguloth wrote: > Hi Miroslav Lichvar, > > What aspects of driver/HW needs to be fixed. could you please > elaborate more on this. > > > Best Regards, > Sujatha At this point, we really can't offer more advice unless the source is available. The driver is definitely doing something funky as evidenced above, and unless the driver is available we can't really comment. I suggest using the provided testptp tool provided in the Linux Kernel documentation to do some sanity checks on all your ptp code paths for adjtime, settime, gettime, adjfreq, etc. Verify these are doing what you expect before you continue your debugging with ptp4l. Regards, Jake |
From: Sujatha G. <suj...@gm...> - 2016-02-17 04:08:07
|
Thank you Jocob and Lichavar for your quick replies. I will try with testptp. Best Regards, Sujatha On Wed, Feb 17, 2016 at 1:26 AM, Keller, Jacob E <jac...@in...> wrote: > Hi, > > On Tue, 2016-02-16 at 16:20 +0530, Sujatha Guguloth wrote: > > Hi Miroslav Lichvar, > > > > What aspects of driver/HW needs to be fixed. could you please > > elaborate more on this. > > > > > > Best Regards, > > Sujatha > > At this point, we really can't offer more advice unless the source is > available. The driver is definitely doing something funky as evidenced > above, and unless the driver is available we can't really comment. > > I suggest using the provided testptp tool provided in the Linux Kernel > documentation to do some sanity checks on all your ptp code paths for > adjtime, settime, gettime, adjfreq, etc. Verify these are doing what > you expect before you continue your debugging with ptp4l. > > Regards, > Jake |
From: Sujatha G. <suj...@gm...> - 2016-02-18 06:34:30
|
Hi Jacob, I tried to test the driver with testptp and it seems to be working as expected. I was able to settime, gettime, adjtime, adjfreq got the expected values. The setup I am using is explained below. MASTER [Linux Ubuntu System with Oregano NIC card for Hardware Time Stamping] < --> SLAVE [Xilinx FPGA Device which supports the HW time stamping ] Slave is getting its clock from an external clock source which is running at 200MHz Frequency. Xilinx Ethernet Driver which is used for HW timestamping, it is available on github @ https://github.com/Xilinx/linux-xlnx/blob/master/drivers/net/ethernet/xilinx/xilinx_axienet_main.c settime, gettime, adjtime, adjfreq functionality is developed in a separate timer driver which is not available on the github but this driver is pretty straight forward. When debugging with PTP4l application, In the normal scenario if step_threshold is set to 0.0, I have observed that - during s1 state *adjtime *gets called with ADJ_SETOFFSET | ADJ_NANO modes to adjust the clock. - when slave moves to s2 state, upon receiving the sync and followup packets, *adjfreq *is called for every 1 second to adjust the frequency of the clock adjtime and adjfreq are implemented in timer driver as below. static int xlnx_ptp_adjfreq(struct ptp_clock_info *ptp, s32 ppb) { struct xlnx_ptp_timer *timer = container_of(ptp, struct xlnx_ptp_timer, ptp_clock_info); int neg_adj = 0; u64 freq = 0; u32 diff = 0; u32 incval = 0; /* External clock source Frequency is 200MHz which is driving the timer */ /* adjust the RTC Increment Control register by (1/200MHZ = 5ns) */ /* resolution of the increment register is 1/1048576 */ /* hence 5 * 1048576 = 0x500000 - value to be written to the register */ incval = 0x500000; if (ppb < 0) { neg_adj = 1; ppb = -ppb; } freq = incval; freq *= ppb; diff = div_u64(freq, 1000000000ULL); incval = neg_adj ? (incval - diff) : (incval + diff); /* Write the adjusted value to the *RTC Increment Value Control Register* */ /* Writing the adjustment value to this register intern adjusts the nanoseconds register value by the hw */ return 0; } static int xlnx_ptp_adjtime(struct ptp_clock_info *ptp, s64 delta) { unsigned long flags = 0; struct xlnx_ptp_timer *timer = container_of(ptp, struct xlnx_ptp_timer, ptp_clock_info); struct timespec offset, then = ns_to_timespec(delta); spin_lock_irqsave(&timer->reg_lock, flags); /* Read the seconds and nano seconds from the RTC Seconds and RTC Nano Seconds Registers */ xlnx_rtc_offset_read(timer, &offset); /* set the current offset values */ offset = then; /* Write the seconds and nano seconds to the *RTC Seconds* and *RTC Nano Seconds* Registers */ xlnx_rtc_offset_write(timer, (const struct timespec *)&offset); spin_unlock_irqrestore(&timer->reg_lock, flags); return 0; } could you please review these functions and let us know if there are any issues. Thank you. Best Regards, Sujatha On Wed, Feb 17, 2016 at 9:37 AM, Sujatha Guguloth <suj...@gm...> wrote: > Thank you Jocob and Lichavar for your quick replies. > > I will try with testptp. > > Best Regards, > Sujatha > > On Wed, Feb 17, 2016 at 1:26 AM, Keller, Jacob E <jac...@in... > > wrote: > >> Hi, >> >> On Tue, 2016-02-16 at 16:20 +0530, Sujatha Guguloth wrote: >> > Hi Miroslav Lichvar, >> > >> > What aspects of driver/HW needs to be fixed. could you please >> > elaborate more on this. >> > >> > >> > Best Regards, >> > Sujatha >> >> At this point, we really can't offer more advice unless the source is >> available. The driver is definitely doing something funky as evidenced >> above, and unless the driver is available we can't really comment. >> >> I suggest using the provided testptp tool provided in the Linux Kernel >> documentation to do some sanity checks on all your ptp code paths for >> adjtime, settime, gettime, adjfreq, etc. Verify these are doing what >> you expect before you continue your debugging with ptp4l. >> >> Regards, >> Jake > > > |
From: Richard C. <ric...@gm...> - 2016-02-18 09:29:20
|
Wow, your code is totally broken... On Thu, Feb 18, 2016 at 12:04:22PM +0530, Sujatha Guguloth wrote: > static int xlnx_ptp_adjfreq(struct ptp_clock_info *ptp, s32 ppb) > { > struct xlnx_ptp_timer *timer = container_of(ptp, struct > xlnx_ptp_timer, > ptp_clock_info); > > int neg_adj = 0; > u64 freq = 0; > u32 diff = 0; > u32 incval = 0; > > /* External clock source Frequency is 200MHz which is driving > the timer */ > /* adjust the RTC Increment Control register by (1/200MHZ = 5ns) */ > /* resolution of the increment register is 1/1048576 */ > /* hence 5 * 1048576 = 0x500000 - value to be written to the > register */ > incval = 0x500000; > > if (ppb < 0) { > neg_adj = 1; > ppb = -ppb; > } > > freq = incval; > freq *= ppb; > diff = div_u64(freq, 1000000000ULL); > > incval = neg_adj ? (incval - diff) : (incval + diff); > > /* Write the adjusted value to the *RTC Increment Value Control > Register* */ > /* Writing the adjustment value to this register intern adjusts the > nanoseconds register value by the hw */ You calculate 'incval' and then do nothing with it. > return 0; > } > > > static int xlnx_ptp_adjtime(struct ptp_clock_info *ptp, s64 delta) > { > unsigned long flags = 0; > struct xlnx_ptp_timer *timer = container_of(ptp, struct > xlnx_ptp_timer, > ptp_clock_info); > struct timespec offset, then = ns_to_timespec(delta); > > spin_lock_irqsave(&timer->reg_lock, flags); > > /* Read the seconds and nano seconds from the RTC Seconds and RTC > Nano Seconds Registers */ > xlnx_rtc_offset_read(timer, &offset); > > /* set the current offset values */ > offset = then; You are supposed to add the delta. > /* Write the seconds and nano seconds to the *RTC Seconds* and *RTC > Nano Seconds* Registers */ > xlnx_rtc_offset_write(timer, (const struct timespec *)&offset); > > spin_unlock_irqrestore(&timer->reg_lock, flags); > return 0; > } Good luck, Richard |
From: Sujatha G. <suj...@gm...> - 2016-02-18 10:03:46
|
Hi Richard, I am writing the incval to the register in adjfreq after the calculation.I had written that in the comments and actual code is below. out_be32((timer->baseaddr + XTIMER1588_RTC_INCREMENT), incval); In adjtime: delta is passed to ns_to_timespec then = ns_to_timespec(delta) offset = then; /* then is copied to offset */ xlnx_rtc_offset_write(timer, (const struct timespec *)&offset); /* This offset is written to the register */ Best Regards, Sujatha On Thu, Feb 18, 2016 at 2:59 PM, Richard Cochran <ric...@gm...> wrote: > Wow, your code is totally broken... > > On Thu, Feb 18, 2016 at 12:04:22PM +0530, Sujatha Guguloth wrote: > > static int xlnx_ptp_adjfreq(struct ptp_clock_info *ptp, s32 ppb) > > { > > struct xlnx_ptp_timer *timer = container_of(ptp, struct > > xlnx_ptp_timer, > > ptp_clock_info); > > > > int neg_adj = 0; > > u64 freq = 0; > > u32 diff = 0; > > u32 incval = 0; > > > > /* External clock source Frequency is 200MHz which is driving > > the timer */ > > /* adjust the RTC Increment Control register by (1/200MHZ = > 5ns) */ > > /* resolution of the increment register is 1/1048576 > */ > > /* hence 5 * 1048576 = 0x500000 - value to be written to the > > register */ > > incval = 0x500000; > > > > if (ppb < 0) { > > neg_adj = 1; > > ppb = -ppb; > > } > > > > freq = incval; > > freq *= ppb; > > diff = div_u64(freq, 1000000000ULL); > > > > incval = neg_adj ? (incval - diff) : (incval + diff); > > > > /* Write the adjusted value to the *RTC Increment Value Control > > Register* */ > > /* Writing the adjustment value to this register intern adjusts > the > > nanoseconds register value by the hw */ > > You calculate 'incval' and then do nothing with it. > > > return 0; > > } > > > > > > static int xlnx_ptp_adjtime(struct ptp_clock_info *ptp, s64 delta) > > { > > unsigned long flags = 0; > > struct xlnx_ptp_timer *timer = container_of(ptp, struct > > xlnx_ptp_timer, > > ptp_clock_info); > > struct timespec offset, then = ns_to_timespec(delta); > > > > spin_lock_irqsave(&timer->reg_lock, flags); > > > > /* Read the seconds and nano seconds from the RTC Seconds and RTC > > Nano Seconds Registers */ > > xlnx_rtc_offset_read(timer, &offset); > > > > /* set the current offset values */ > > offset = then; > > You are supposed to add the delta. > > > /* Write the seconds and nano seconds to the *RTC Seconds* and > *RTC > > Nano Seconds* Registers */ > > xlnx_rtc_offset_write(timer, (const struct timespec *)&offset); > > > > spin_unlock_irqrestore(&timer->reg_lock, flags); > > return 0; > > } > > Good luck, > Richard > |