After reading forums and bug tracker, I got a bit confused with the current state of hardware time-stamping support in PTPd 2.3.0 / 2.3.1. The more so that PTPd made a tremendous progress recently, thus earlier messages may not be actual anymore.
Ptpd is unable to lookup your NIC's timestamping capabilities so gives up instead of defaulting to standard software timestamping. This has been fixed in subversion since.
What is your precision target? If it's 100us, you're super safe. If it's 10us, you're OK. If it's 5us, read on, if it's less than that, read on and consider hardware assistance.
The sad truth is that if you're running ptpd with no hardware assistance whatsoever, you cannot trust the offset from master and one-way delay statistics logged by it.
If you are running hardware (NIC) that gives you ingress and egress timestamping and plug that into ptpd, you can get into the sub-microsecond precision (and accuracy) region.
On a microcontroller that does nothing but ptpd you will easily get sub-micro precision, and similar accuracy if you have hardware timestamping.
Yes, it (hardware timestamping) is in the plan, but, if you want to try something experimental, you can check out the v2_bpf branch and see if the pcap code works for you. This has only been tested on FreeBSD but it ought to work fine on other OSs.
So, what is the current state? Is hardware timestamping support still implemented in experimental branch only? Does it run on selected operating systems only? Does it require additional system configuration, or kernel and libpcap recompiling?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am working on a PTP implementation on a BeagleBone which has a HW Support. Can you give me a hint if there is a HW time stamp implementation on a similar ARM core ?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
HW timestamping is done by the MAC or the PHY - so it's not a question of the ARM core but the whole SoC. You'd need to check which MAC chip your system has / what PHY it's using ad if it's supported by the Linux PHC API. If it is, then once ptpd with HW support is released, it will run on it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for your answer. I think my question wasn't well formulated. I understand that the time stamp is done by the core itselfs (actually by the CPTS). Also, as I understand, the PTPd deamon receives the PTP TCP packet and then uses the internal time to get the time stamp. My first idea (coming from my HW design background) would be to just read the time information not from the internal timer but from the MAC itself (EVENT_FIFO) instead. But thanks for the info to check the PHC API. I write a Thesis about PTP Synchronisation and I wanted to include the HW time stamping, but I think this is out of reach with PTPd. I just stumbled over your answer that it was already implemented on some specific HW, which made me curious.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
First of all, there are no TCP packets but UDP. Hardware timestaming is work in progress in PTPd project but you can try another PTP implementation - linuxptp which has hardware timestamping already implemented.
Ideas you mention are already well known and implemented in many MACs an PHYs. Many of them are already covered by Linux PHC and linuxptp as Wojciech told.
No need to reinvent the wheel, just look around.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the answers. For you this is all clear and easy, but I have to dig in quite unknown matter :) So it is always appreciated to get some advice where to search.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
There is no hardware timestamping support as part of PTPd code base at this stage but I'm working on it, at least for Linux. There have been many experiments but nothing supportable has been merged back with ptpd trunk.
In the thread where I mention "PTP can't look up timestamping capabilities", I mean that PTPD is looking up software timestamping capabilities for the given NIC.
Many people have been modifying PTPd to use custom hardware timestamping solutions using NIC specific APIs for years, and hence all those references. Those are not official PTP releases but different users' own code. Many users and companies have used ptpd as a basis for their projects / products, but usually using NIC-specific or FPGA-specific code so nothing generic. There is now the PHC API in the Linux kernel so this is what I intend to implement for ptpd for Linux, however the idea is to introduce a generic transport API where it should be possible to add support for any hardware using as little code modifications as possible.
The bpf branch George talks about, has already been merged into ptpd and that is the libpcap + ethernet code - still no true hardware timestamping.
The addition of hardware timestamping introduces another problem (for traditional servers/hosts) - hardware timestamping alone, only makes sense in GM operation. In slave operation you need to run two servos: 1: sync NIC clock with PTP, and 2: sync OS clock to NIC clock. The linuxptp project achieves this with two daemons, ptp4l and phc2sys.
I have some test code which makes it possible to do all of this in ptpd, but this is not going to get into the PTPd codebase that quick. We do hope to have h/w timestamping support of some kind in ptpd this year. This is a major functionality ptpd is currently missing, and will allow the project to really get off the ground.
As to what's required - firs of all, your NIC MAC/PHY chip has to support PTP, that is be able to recognise and timestamp PTP packets on hardware level. As far as I'm aware, Linux is currently the only OS providing a common API to handle this. Kernel requirement is that you need a 2.6.30+ kernel (realistically 3.x), with SO_TIMESTAMPING support for ethtool and sockets. Red Hat have backported latest versions of this code and drivers into RHEL6.5/CentOS 6.5 (kernel 2.6.32).
The Linuxptp project by Richard Cochran (he wrote the kernel PTP API and wrote the linuxptp daemon) has an up to date driver support matrix on their website: http://linuxptp.sf.net
I hope this answers your question :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I know about PTP4L and already gave it a try, but found it too tied to Linux (no I see why), less transparent, poorly documented, without public support. For example, man pages say that ptp4l controls timestamping mode itself, but when I launched hwstamp_ctl in parallel it indicated that no hardware-specific mode was enabled; the only thing I managed to do with this tool is to break ptp4l down: it couldn't communicate anymore until the system was rebooted, meanwhile ptpd was able to take its place and start working. ptp4l doesn't even daemonize itself, which makes me wonder whether it's really intended for day-to-day productive use.
As to hardware and software requirements. Unless I'm mistaken, the first one is easy: if ethtool -T prints a plenty of HW stuff, then you're fine.
Getting Linux 2.6.32 or newer is not a problem usually, perhaps with necessary options already compiled in. Unless other modifications are required, one doesn't need to recompile the kernel, right? (Because that isn't possible sometimes.)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Linuxptp states this openly - it's intended for Linux only and for SO_TIMESTAMPING only. Commercial distributions are usually slow to adapt cutting edge technology - it took RHEL over three years to start fully supporting this API.
The PHC API is part of the Linux kernel so unless your distribution disabled it in their kernel, you should be able to use it out of the box. Debian, Ubuntu, RHEL/CentOS/Fedora all enable it. If something doesn't work, this is likely to be a problem with the driver and not necessarily ptp4l. This API doesn't currently support things like bonded interfaces, bridges or (unconfirmed) VLAN subinterfaces. Even if you enable timestamping on the undelying base interface, I'm not sure if the timestamps make it to the bond/br/vlan. You are correct - if you see HW RX timestamping and (very important for delay request and master mode) HW TX timestamping, ptp4l should work.
Some of my observations about ptp4l are similar to yours, however the issue you're describing is not always ptp4l's fault, driver support still leaves a lot to wish for. When PTPd gains support for this API, we'll probably going to run into some of the same issues with some drivers. PTPd aims to provide clear and descriptive logging, easy troubleshooting and offer a cross-platform, robust, feature-rich PTP protocol implementation fit for use in mission-critical production environments. We still have a long way to go but we're getting there. PTPd documentation is pretty much non-existent, but we try to make up for that with complete and up to date man pages and a rich built-in help system, and hopefully clear log output.
For less common platforms and legacy systems not supporting hardware timestamping, PTPd is the only free alternative to commercial products like TimeKeeper. We also intend to abstract the PTP protocol implementation into a library which can be reused for other purposes. Because of the BSD licencing, it is also somewhat attractive to vendors who wish to "quietly" use it for their products.
To all reading this - please correct me if some of the statements I'm making regarding LinuxPTP and the PHC API are untrue or need clarification.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I got a version/branch that it seems to support hardware timestamp https://github.com/audioscience/ptpd2
I see related code, like fd = open("/dev/ptp0", o_RDWR);
and the offset from master is really more smaller than software timestamp(without clkid, clock_device, open ptpClockDevice in the new version)
why the lastest version do not implement it?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You will find more than one unofficial fork with hardware timestamping support. It is quite obvious that with HW support the performance is better. That is the sole purpose of hardware assistance. You will also find that many of those ports were developed for specific purposes, sometimes for specific hardware and some were developed as research projects. You are free to use them and get back to PTPd when it gains this functionality.Some vendors used PTPd as the basis for their products - such as Solarflare's sfptpd intended for use with Solarflare network cards.
PTPd had many other missing functionalities and architectural problems that had to be resolved before moving towards hardware support (still has many) - which would end up with wasted effort if HW support was rushed too early.
It is not a hard task to bolt on hardware support (namely Linux PHC API) to PTPd to create an OS-dependent, inflexible solution. Many have done it and it works for them. PTPd is a cross-platform project aiming for APIs and more generic approaches. In that spirit, work is underway to introduce generic transport and clock control mechanisms, where the Linux PHC API will be supported from the start, but through an abstraction layer. We are hoping to see some effects later this year or early next year.
I hope this answers your questions.
Regards,
Wojciech
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi Wojciech,
Thanks for your reply.
Where can I get the unofficail fork or patch that support PHC and compliant with version 2.3.1?
I am interesting in the new one but need to support hardware timestamp.
I need the accuracy of offset from master in nanosecond. us is not enough.
May I get trail version to try it?
Thanks.
Last edit: CK 2015-09-07
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I said that you are free to use any of the versions or ports you found yourself, such as the one from Audioscience. There are no 2.3.1 releases with hardware patches as of now. There are no trial versions. We are a public, open project - as soon as usable code is available, it will be published. Right now it isn't, so it isn't. PTPd uses SourceForge and is soon moving to GitHub - you can download any code that's published but you will not find 2.3.1 with hardware support. You can look through the patches and discussion sections on this site, some people have sent patches for h/w support for version 2.3. With some work you can adapt it to 2.3.1 yourself if you wish. Or you can use LinuxPTP (ptp4l) which does support hardware timestamping.
Regards,
Wojciech
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
After reading forums and bug tracker, I got a bit confused with the current state of hardware time-stamping support in PTPd 2.3.0 / 2.3.1. The more so that PTPd made a tremendous progress recently, thus earlier messages may not be actual anymore.
2014-02-12, Wojciech Owczarek, https://sourceforge.net/p/ptpd/bugs/66/#b92e
2014-02-03, Wojciech Owczarek, https://sourceforge.net/p/ptpd/discussion/469208/thread/dc12f0d4/#9b23
2013-02-11, Wojciech Owczarek, https://sourceforge.net/p/ptpd/discussion/469208/thread/b9c3da73/#5021
2013-11-01, JohnC, https://sourceforge.net/p/ptpd/discussion/469208/thread/d3d25992/#5331
2013-08-08, Wojciech Owczarek, https://sourceforge.net/p/ptpd/discussion/469208/thread/35d099a6/#0f11/d947
2013-06-25, Jan Breuer , https://sourceforge.net/p/ptpd/discussion/469208/thread/0972358c/#0822
2012-09-21, George Neville-Neil, https://sourceforge.net/p/ptpd/discussion/469207/thread/cd08e18a/#eeb0
So, what is the current state? Is hardware timestamping support still implemented in experimental branch only? Does it run on selected operating systems only? Does it require additional system configuration, or kernel and libpcap recompiling?
I am working on a PTP implementation on a BeagleBone which has a HW Support. Can you give me a hint if there is a HW time stamp implementation on a similar ARM core ?
Franz,
HW timestamping is done by the MAC or the PHY - so it's not a question of the ARM core but the whole SoC. You'd need to check which MAC chip your system has / what PHY it's using ad if it's supported by the Linux PHC API. If it is, then once ptpd with HW support is released, it will run on it.
Thanks for your answer. I think my question wasn't well formulated. I understand that the time stamp is done by the core itselfs (actually by the CPTS). Also, as I understand, the PTPd deamon receives the PTP TCP packet and then uses the internal time to get the time stamp. My first idea (coming from my HW design background) would be to just read the time information not from the internal timer but from the MAC itself (EVENT_FIFO) instead. But thanks for the info to check the PHC API. I write a Thesis about PTP Synchronisation and I wanted to include the HW time stamping, but I think this is out of reach with PTPd. I just stumbled over your answer that it was already implemented on some specific HW, which made me curious.
First of all, there are no TCP packets but UDP. Hardware timestaming is work in progress in PTPd project but you can try another PTP implementation - linuxptp which has hardware timestamping already implemented.
Ideas you mention are already well known and implemented in many MACs an PHYs. Many of them are already covered by Linux PHC and linuxptp as Wojciech told.
No need to reinvent the wheel, just look around.
Thanks for the answers. For you this is all clear and easy, but I have to dig in quite unknown matter :) So it is always appreciated to get some advice where to search.
Anton,
There is no hardware timestamping support as part of PTPd code base at this stage but I'm working on it, at least for Linux. There have been many experiments but nothing supportable has been merged back with ptpd trunk.
In the thread where I mention "PTP can't look up timestamping capabilities", I mean that PTPD is looking up software timestamping capabilities for the given NIC.
Many people have been modifying PTPd to use custom hardware timestamping solutions using NIC specific APIs for years, and hence all those references. Those are not official PTP releases but different users' own code. Many users and companies have used ptpd as a basis for their projects / products, but usually using NIC-specific or FPGA-specific code so nothing generic. There is now the PHC API in the Linux kernel so this is what I intend to implement for ptpd for Linux, however the idea is to introduce a generic transport API where it should be possible to add support for any hardware using as little code modifications as possible.
The bpf branch George talks about, has already been merged into ptpd and that is the libpcap + ethernet code - still no true hardware timestamping.
The addition of hardware timestamping introduces another problem (for traditional servers/hosts) - hardware timestamping alone, only makes sense in GM operation. In slave operation you need to run two servos: 1: sync NIC clock with PTP, and 2: sync OS clock to NIC clock. The linuxptp project achieves this with two daemons, ptp4l and phc2sys.
I have some test code which makes it possible to do all of this in ptpd, but this is not going to get into the PTPd codebase that quick. We do hope to have h/w timestamping support of some kind in ptpd this year. This is a major functionality ptpd is currently missing, and will allow the project to really get off the ground.
As to what's required - firs of all, your NIC MAC/PHY chip has to support PTP, that is be able to recognise and timestamp PTP packets on hardware level. As far as I'm aware, Linux is currently the only OS providing a common API to handle this. Kernel requirement is that you need a 2.6.30+ kernel (realistically 3.x), with SO_TIMESTAMPING support for ethtool and sockets. Red Hat have backported latest versions of this code and drivers into RHEL6.5/CentOS 6.5 (kernel 2.6.32).
The Linuxptp project by Richard Cochran (he wrote the kernel PTP API and wrote the linuxptp daemon) has an up to date driver support matrix on their website: http://linuxptp.sf.net
I hope this answers your question :)
Thanks a lot, now it's much more clear.
I know about PTP4L and already gave it a try, but found it too tied to Linux (no I see why), less transparent, poorly documented, without public support. For example, man pages say that ptp4l controls timestamping mode itself, but when I launched hwstamp_ctl in parallel it indicated that no hardware-specific mode was enabled; the only thing I managed to do with this tool is to break ptp4l down: it couldn't communicate anymore until the system was rebooted, meanwhile ptpd was able to take its place and start working. ptp4l doesn't even daemonize itself, which makes me wonder whether it's really intended for day-to-day productive use.
As to hardware and software requirements. Unless I'm mistaken, the first one is easy: if ethtool -T prints a plenty of HW stuff, then you're fine.
Getting Linux 2.6.32 or newer is not a problem usually, perhaps with necessary options already compiled in. Unless other modifications are required, one doesn't need to recompile the kernel, right? (Because that isn't possible sometimes.)
Anton,
Linuxptp states this openly - it's intended for Linux only and for SO_TIMESTAMPING only. Commercial distributions are usually slow to adapt cutting edge technology - it took RHEL over three years to start fully supporting this API.
The PHC API is part of the Linux kernel so unless your distribution disabled it in their kernel, you should be able to use it out of the box. Debian, Ubuntu, RHEL/CentOS/Fedora all enable it. If something doesn't work, this is likely to be a problem with the driver and not necessarily ptp4l. This API doesn't currently support things like bonded interfaces, bridges or (unconfirmed) VLAN subinterfaces. Even if you enable timestamping on the undelying base interface, I'm not sure if the timestamps make it to the bond/br/vlan. You are correct - if you see HW RX timestamping and (very important for delay request and master mode) HW TX timestamping, ptp4l should work.
Some of my observations about ptp4l are similar to yours, however the issue you're describing is not always ptp4l's fault, driver support still leaves a lot to wish for. When PTPd gains support for this API, we'll probably going to run into some of the same issues with some drivers. PTPd aims to provide clear and descriptive logging, easy troubleshooting and offer a cross-platform, robust, feature-rich PTP protocol implementation fit for use in mission-critical production environments. We still have a long way to go but we're getting there. PTPd documentation is pretty much non-existent, but we try to make up for that with complete and up to date man pages and a rich built-in help system, and hopefully clear log output.
For less common platforms and legacy systems not supporting hardware timestamping, PTPd is the only free alternative to commercial products like TimeKeeper. We also intend to abstract the PTP protocol implementation into a library which can be reused for other purposes. Because of the BSD licencing, it is also somewhat attractive to vendors who wish to "quietly" use it for their products.
To all reading this - please correct me if some of the statements I'm making regarding LinuxPTP and the PHC API are untrue or need clarification.
Hi,
I got a version/branch that it seems to support hardware timestamp
https://github.com/audioscience/ptpd2
I see related code, like fd = open("/dev/ptp0", o_RDWR);
and the offset from master is really more smaller than software timestamp(without clkid, clock_device, open ptpClockDevice in the new version)
why the lastest version do not implement it?
Hi,
You will find more than one unofficial fork with hardware timestamping support. It is quite obvious that with HW support the performance is better. That is the sole purpose of hardware assistance. You will also find that many of those ports were developed for specific purposes, sometimes for specific hardware and some were developed as research projects. You are free to use them and get back to PTPd when it gains this functionality.Some vendors used PTPd as the basis for their products - such as Solarflare's sfptpd intended for use with Solarflare network cards.
PTPd had many other missing functionalities and architectural problems that had to be resolved before moving towards hardware support (still has many) - which would end up with wasted effort if HW support was rushed too early.
It is not a hard task to bolt on hardware support (namely Linux PHC API) to PTPd to create an OS-dependent, inflexible solution. Many have done it and it works for them. PTPd is a cross-platform project aiming for APIs and more generic approaches. In that spirit, work is underway to introduce generic transport and clock control mechanisms, where the Linux PHC API will be supported from the start, but through an abstraction layer. We are hoping to see some effects later this year or early next year.
I hope this answers your questions.
Regards,
Wojciech
Hi Wojciech,
Thanks for your reply.
Where can I get the unofficail fork or patch that support PHC and compliant with version 2.3.1?
I am interesting in the new one but need to support hardware timestamp.
I need the accuracy of offset from master in nanosecond. us is not enough.
May I get trail version to try it?
Thanks.
Last edit: CK 2015-09-07
can anybody here help me?
Thanks.
Hi,
I said that you are free to use any of the versions or ports you found yourself, such as the one from Audioscience. There are no 2.3.1 releases with hardware patches as of now. There are no trial versions. We are a public, open project - as soon as usable code is available, it will be published. Right now it isn't, so it isn't. PTPd uses SourceForge and is soon moving to GitHub - you can download any code that's published but you will not find 2.3.1 with hardware support. You can look through the patches and discussion sections on this site, some people have sent patches for h/w support for version 2.3. With some work you can adapt it to 2.3.1 yourself if you wish. Or you can use LinuxPTP (ptp4l) which does support hardware timestamping.
Regards,
Wojciech