linuxptp-users

[Linuxptp-users] phc2sys with ptp_kvm and mutiport mellanox NICs

[Hussamuddin N. <na...@ne...>]

Hello folks,

  I am using Mellanox Connect X5 dual port 100G card whose PTP clock 
devices (/dev/ptp0, /dev/ptp1) were synced using ptp4l and phc2sys via a 
PTP Appliance. We now install a KVM on the physical host and setup the 
Mellanox NIC as pass thru to the KVM. The KVM has the module ptp_kvm 
(/dev/ptp2) installed in it and i am able to successfully run phc2sys to 
sync /dev/ptp0 from /dev/ptp2 using the command

  phc2sys -O 35 -s /dev/ptp2 -c /dev/ptp0

NOTE: I am not using ptp4l inside the KVM since the ptp_kvm device gives 
the accurate time from the host that is already being synced using 
ptp4l/phc2sys on the host

But i am unable to provide an additional "-c" here to sync  the other 
ptp device in the KVM.

I know that is possible using the boundary clock config option in ptp4l  
when we have ptp4l and phc2sys running in automatic mode since we had 
that working on the host without the KVM.

So any suggestions on how do i do phc2sys for multiple destination devices ?

-- 
cheers,

Hussam
(Hussamuddin Nasir)

Netlab, GENI & FABRIC Operations Team

-------------------------------------------------------------------
Laboratory for Adv. Networking  Phone  : (859)218-0059
James F Hardymon Building       Fax    : (859)323-3740
301 Rose Street, Rm 237         E-mail : na...@ne...
Lexington, KY 40506-0495        Web    : http://www.netlab.uky.edu

                         University of Kentucky
                         **********************
-------------------------------------------------------------------
  

Re: [Linuxptp-users] phc2sys with ptp_kvm and mutiport mellanox NICs

[Jacob K. <jac...@in...>]

On 7/21/2020 3:24 PM, Hussamuddin Nasir via Linuxptp-users wrote:
> Hello folks,
> 
>   I am using Mellanox Connect X5 dual port 100G card whose PTP clock 
> devices (/dev/ptp0, /dev/ptp1) were synced using ptp4l and phc2sys via a 
> PTP Appliance. We now install a KVM on the physical host and setup the 
> Mellanox NIC as pass thru to the KVM. The KVM has the module ptp_kvm 
> (/dev/ptp2) installed in it and i am able to successfully run phc2sys to 
> sync /dev/ptp0 from /dev/ptp2 using the command
>
>   phc2sys -O 35 -s /dev/ptp2 -c /dev/ptp0
> 
> NOTE: I am not using ptp4l inside the KVM since the ptp_kvm device gives 
> the accurate time from the host that is already being synced using 
> ptp4l/phc2sys on the host

So you're passing one port of the device into the VM, but not the other?

> 
> But i am unable to provide an additional "-c" here to sync  the other 
> ptp device in the KVM.

Right, because the PTP device is created in the VM, not in the host.

> 
> I know that is possible using the boundary clock config option in ptp4l  
> when we have ptp4l and phc2sys running in automatic mode since we had 
> that working on the host without the KVM.
>> So any suggestions on how do i do phc2sys for multiple destination
devices ?
> 

The problem, I think, is that the two PTP clocks aren't both visible in
the same system. One is in the host, and one in the VM.

I don't know if this can be solved today. It seems like you'd be better
off doing a virtual device into the VM and leaving the whole MLX5 card
out in the host.

Thanks,
Jake

Re: [Linuxptp-users] phc2sys with ptp_kvm and mutiport mellanox NICs

[nasir <na...@ne...>]

No. I am passing in both ports to the KVM. The KVM sees three ptpt devices. One is using  the ptp_kvm module and the other two are the mellanox NICS. I am trying to sync the mellanox clocks using ptp_kvm as source using phc2sys but using multiple -c options results in phc2sys uaing the last ptp device in the command as its destination deviceSent from my Samsung Galaxy smartphone.
-------- Original message --------From: Jacob Keller <jac...@in...> Date: 7/21/20  7:28 PM  (GMT-05:00) To: lin...@li... Subject: Re: [Linuxptp-users] phc2sys with ptp_kvm and mutiport mellanox NICs On 7/21/2020 3:24 PM, Hussamuddin Nasir via Linuxptp-users wrote:> Hello folks,> >   I am using Mellanox Connect X5 dual port 100G card whose PTP clock > devices (/dev/ptp0, /dev/ptp1) were synced using ptp4l and phc2sys via a > PTP Appliance. We now install a KVM on the physical host and setup the > Mellanox NIC as pass thru to the KVM. The KVM has the module ptp_kvm > (/dev/ptp2) installed in it and i am able to successfully run phc2sys to > sync /dev/ptp0 from /dev/ptp2 using the command>>   phc2sys -O 35 -s /dev/ptp2 -c /dev/ptp0> > NOTE: I am not using ptp4l inside the KVM since the ptp_kvm device gives > the accurate time from the host that is already being synced using > ptp4l/phc2sys on the hostSo you're passing one port of the device into the VM, but not the other?> > But i am unable to provide an additional "-c" here to sync  the other > ptp device in the KVM.Right, because the PTP device is created in the VM, not in the host.> > I know that is possible using the boundary clock config option in ptp4l  > when we have ptp4l and phc2sys running in automatic mode since we had > that working on the host without the KVM.>> So any suggestions on how do i do phc2sys for multiple destinationdevices ?> The problem, I think, is that the two PTP clocks aren't both visible inthe same system. One is in the host, and one in the VM.I don't know if this can be solved today. It seems like you'd be betteroff doing a virtual device into the VM and leaving the whole MLX5 cardout in the host.Thanks,Jake_______________________________________________Linuxptp-users mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/linuxptp-users

Re: [Linuxptp-users] phc2sys with ptp_kvm and mutiport mellanox NICs

[Miroslav L. <mli...@re...>]

On Tue, Jul 21, 2020 at 08:43:10PM -0400, nasir via Linuxptp-users wrote:
> 
> No. I am passing in both ports to the KVM. The KVM sees three ptpt
> devices. One is using  the ptp_kvm module and the other two are the
> mellanox NICS. I am trying to sync the mellanox clocks using ptp_kvm
> as source using phc2sys but using multiple -c options results in
> phc2sys uaing the last ptp device in the command as its destination
> device

I'm not sure how multiple -c options is supposed to work, but I think
you could run multiple phc2sys instances, one for each clock that
needs to be synchronized.

-- 
Miroslav Lichvar

Re: [Linuxptp-users] phc2sys with ptp_kvm and mutiport mellanox NICs

[Hussamuddin N. <na...@ne...>]

Yes i can get it working with multiple instances of phc2sys, but was not 
sure that was the best way to move forward. Also does any one have 
experience using the ptp_kvm module (/dev/ptp_kvm) as a source PHC.

I was looking at the latest devel linuxptp code that has a new program 
called "ts2phc" that claims to sync multiple PHCs using a single source 
PHC, but when i try this command i get the error

> ~/linuxptp-code# ts2phc -s /dev/ptp_kvm -c enp7s0 -c enp8s0
> failed to arm slaves

Any suggestions there ?


Also when using phc2sys for this task when only one PHC is used as  
destination like

> /usr/sbin/phc2sys -O 0 -s /dev/ptp_kvm -c enp8s0
Is using the offset set to Zero correct ? the /dev/ptp_kvm gets its time 
from the host clock which is already being synced using a ptp4l (and a 
phc2sys) on the host running with HW Timestamping

-- 
cheers,

Hussam
(Hussamuddin Nasir)

Netlab & GENI Operations Team

-------------------------------------------------------------------
Laboratory for Adv. Networking  Phone  : (859)218-0059
James F Hardymon Building       Fax    : (859)323-3740
301 Rose Street, Rm 237         E-mail : na...@ne...
Lexington, KY 40506-0495        Web    : http://www.netlab.uky.edu

                         University of Kentucky
                         **********************
-------------------------------------------------------------------

On 7/22/20 1:45 AM, Miroslav Lichvar wrote:
> On Tue, Jul 21, 2020 at 08:43:10PM -0400, nasir via Linuxptp-users wrote:
>> No. I am passing in both ports to the KVM. The KVM sees three ptpt
>> devices. One is using  the ptp_kvm module and the other two are the
>> mellanox NICS. I am trying to sync the mellanox clocks using ptp_kvm
>> as source using phc2sys but using multiple -c options results in
>> phc2sys uaing the last ptp device in the command as its destination
>> device
> I'm not sure how multiple -c options is supposed to work, but I think
> you could run multiple phc2sys instances, one for each clock that
> needs to be synchronized.
>