Re: [Linuxptp-users] Fw: “Resource temporarily unavailable” errors during flood ping test

[Keller, J. E <jac...@in...>]

> -----Original Message-----
> From: Richard Cochran [mailto:ric...@gm...]
> Sent: Tuesday, October 30, 2012 1:48 PM
> To: Keller, Jacob E
> Cc: lin...@li...
> Subject: Re: [Linuxptp-users] Fw: “Resource temporarily unavailable”
> errors during flood ping test
> 
> On Tue, Oct 30, 2012 at 06:23:53PM +0100, Richard Cochran wrote:
> > On Tue, Oct 30, 2012 at 09:54:30AM -0700, Jacob Keller wrote:
> > >
> > > I believe the true correct answer is to completely re-architect the
> > > tx_hwtstamp to be asynchronous, so that it just waits until it
> receives
> > > the timestamp for a complete sequence of events. That design is
> > > significantly more difficult to write though.
> >
> > But even if we did that way, it would not really be a better
> > solution. Think about your own Intel cards. They would end up missing
> > Tx time stamps and possibly mixing them up due to the hardware
> > limitation of having a Tx time stamp FIFO of depth one.
> 
> This may be the wrong list, but this reminds me of an issue with the
> Intel hardware that I have been meaning to ask you about.  The igb
> driver has always had the following comment regarding transmit time
> stamps:
> 
>  * If we were asked to do hardware stamping and such a time stamp is
>  * available, then it must have been for this skb here because we only
>  * allow only one such packet into the queue.
> 
> This statement wasn't actually true up until recently, when Matthew
> Vick added some code that enforced the one packet limit.
> 
> If I am not mistaken, the ixgb also would need some kind of guard
> against the case when a user program sends two or more event packets
> in a row, would it not?


Short answer: that limit is enforced by the hardware (it disables time stamping as long as the RXTSTMP register is locked), except in the mode that puts time stamp directly into packet buffer.


Long answer:

That comment actually refers to hardware design for the 82576 device. Basically, a packet is time stamped and the register stores RXHWTSTMP and sets the bit in the descriptor plus the bit in TSYNCRXCTL.

No more than one packet will have the bit set in the descriptor, because time stamping is disabled when there is a valid stamp in the RXHWTSTAMP registers, so that packet must match the timestamp in the registers.

There was some queuing code but this actually turns out to be bogus and did nothing of value, and I've petitioned to have it removed.

for 10Gbe, I added the ptp_match function to prevent the case where a time stamped packet is dropped.

The one-per-queue basically occurs because hardware design timestamps the packet, puts timestamp in registers, and indicates which packet got time stamped. There's no need for more correlation because the descriptor indicates which packet got time stamped, and as long as you don't read the RXTSTMP registers they remain locked and hardware won't timestamp another packet until you unlock the RXTSTMP registers. The ptp_match is necessary in the very rare case that a time stamped ptp packet never reaches the driver. (it will find the next ptp packet that should have been time stamped according to the timestamp mode, and then clear timestamps so that the error case causing timestamps to stop forever is avoided)

For the 82580 part timestamps are stored in the packet buffer avoiding the issue entirely.

So, the only guard necessary is the ptp_match function to prevent that condition. If there is a timestamp in the registers, hardware doesn't timestamp again until the user reads the timestamp value out. Two rapid event packets in succession will cause the first arrived to be time stamped and the second to not be time stamped.

> 
> Thanks,
> Richard