Re: [Linuxptp-users] Fw: “Resource temporarily unavailable” errors during flood ping test

[Keller, J. E <jac...@in...>]

> -----Original Message-----
> From: Jonatan Walck [mailto:jw...@ne...]
> Sent: Wednesday, October 31, 2012 12:30 AM
> To: lin...@li...
> Subject: Re: [Linuxptp-users] Fw: “Resource temporarily unavailable”
> errors during flood ping test
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 10/30/2012 10:23 PM, Keller, Jacob E wrote:
> >
> >
> >> -----Original Message----- From: Richard Cochran
> >> [mailto:ric...@gm...] Sent: Tuesday, October 30, 2012
> >> 1:48 PM To: Keller, Jacob E Cc:
> >> lin...@li... Subject: Re:
> >> [Linuxptp-users] Fw: “Resource temporarily unavailable” errors
> >> during flood ping test
> >>
> >> On Tue, Oct 30, 2012 at 06:23:53PM +0100, Richard Cochran wrote:
> >>> On Tue, Oct 30, 2012 at 09:54:30AM -0700, Jacob Keller wrote:
> >>>>
> >>>> I believe the true correct answer is to completely
> >>>> re-architect the tx_hwtstamp to be asynchronous, so that it
> >>>> just waits until it
> >> receives
> >>>> the timestamp for a complete sequence of events. That design
> >>>> is significantly more difficult to write though.
> >>>
> >>> But even if we did that way, it would not really be a better
> >>> solution. Think about your own Intel cards. They would end up
> >>> missing Tx time stamps and possibly mixing them up due to the
> >>> hardware limitation of having a Tx time stamp FIFO of depth
> >>> one.
> >>
> >> This may be the wrong list, but this reminds me of an issue with
> >> the Intel hardware that I have been meaning to ask you about.
> >> The igb driver has always had the following comment regarding
> >> transmit time stamps:
> >>
> >> * If we were asked to do hardware stamping and such a time stamp
> >> is * available, then it must have been for this skb here because
> >> we only * allow only one such packet into the queue.
> >>
> >> This statement wasn't actually true up until recently, when
> >> Matthew Vick added some code that enforced the one packet limit.
> >>
> >> If I am not mistaken, the ixgb also would need some kind of
> >> guard against the case when a user program sends two or more
> >> event packets in a row, would it not?
> >
> >
> > Short answer: that limit is enforced by the hardware (it disables
> > time stamping as long as the RXTSTMP register is locked), except in
> > the mode that puts time stamp directly into packet buffer.
> >
> >
> > Long answer:
> >
> > That comment actually refers to hardware design for the 82576
> > device. Basically, a packet is time stamped and the register stores
> > RXHWTSTMP and sets the bit in the descriptor plus the bit in
> > TSYNCRXCTL.
> >
> > No more than one packet will have the bit set in the descriptor,
> > because time stamping is disabled when there is a valid stamp in
> > the RXHWTSTAMP registers, so that packet must match the timestamp
> > in the registers.
> >
> > There was some queuing code but this actually turns out to be bogus
> > and did nothing of value, and I've petitioned to have it removed.
> >
> > for 10Gbe, I added the ptp_match function to prevent the case where
> > a time stamped packet is dropped.
> >
> > The one-per-queue basically occurs because hardware design
> > timestamps the packet, puts timestamp in registers, and indicates
> > which packet got time stamped. There's no need for more correlation
> > because the descriptor indicates which packet got time stamped, and
> > as long as you don't read the RXTSTMP registers they remain locked
> > and hardware won't timestamp another packet until you unlock the
> > RXTSTMP registers. The ptp_match is necessary in the very rare case
> > that a time stamped ptp packet never reaches the driver. (it will
> > find the next ptp packet that should have been time stamped
> > according to the timestamp mode, and then clear timestamps so that
> > the error case causing timestamps to stop forever is avoided)
> >
> > For the 82580 part timestamps are stored in the packet buffer
> > avoiding the issue entirely.
> >
> > So, the only guard necessary is the ptp_match function to prevent
> > that condition. If there is a timestamp in the registers, hardware
> > doesn't timestamp again until the user reads the timestamp value
> > out. Two rapid event packets in succession will cause the first
> > arrived to be time stamped and the second to not be time stamped.
> >
> >>
> >> Thanks, Richard
> >
> 
> Now that we are on the subject, have I understood correctly that
> 82599(ES) suffers from the same hardware design as the 82576? That is,
> timestamps in a seperate register rather than the possibility to be
> strapped on to each packet in the queue?
> 
> Are there any 10GE cards with packet buffer timestamping?


At this time, there are no 10Gb cards with per-packet timestamping in the buffer.

- Jake