astlinux-devel — The AstLinux development environment, compilation, etc

Re: [Astlinux-devel] Multiple peers with Wireguard

[David K. <da...@ke...>]

If we are thinking of tweaks to how wireguard is implemented in Astlinux I
would like to put in a word for supporting multiple wireguard interfaces.
So in addition to wg0 we can have wg1, wg2, etc.  Need not be unlimited but
any number greater than one would be good.  Principle use case would be for
me to separate the tunnel I use for WAN failover from tunnel(s) used for
anything else.

David

On Wed, Oct 10, 2018 at 11:20 AM Lonnie Abelbeck <li...@lo...>
wrote:

> Hi Michael,
>
> Given:
> pbx2 --WG-- pbx4 --WG-- pbx3
>
> Then ping pbx2 -> pbx3, a restart of wireguard (on pbx4) interrupts the
> ping for 17 seconds, exactly as your results.
>
> The advantage of using "wg set wg0 ...", for the active peers not changed,
> their flows are not effected.
>
> While this optimization is possible with WireGuard (unlike OpenVPN) it
> would involve some tricky coding ... not only changes using "wg set wg0
> ..." but the automatic routes created by Allowed IP's would need to be
> added/removed.  Then the question how the /mnt/kd/wireguard.script would be
> called for a "incremental" reload.
>
> This requires some thought...
>
> Lonnie
>
>
>
> > On Oct 9, 2018, at 11:18 PM, Michael Knill <
> mic...@ip...> wrote:
> >
> > Thanks Lonnie
> >
> > Yes restarting Wireguard caused a significant packet loss (ping from
> endpoint to server):
> > PING 172.29.253.1 (172.29.253.1): 56 data bytes
> > 64 bytes from 172.29.253.1: seq=0 ttl=64 time=13.475 ms
> > 64 bytes from 172.29.253.1: seq=1 ttl=64 time=13.277 ms
> > 64 bytes from 172.29.253.1: seq=2 ttl=64 time=13.039 ms
> > 64 bytes from 172.29.253.1: seq=3 ttl=64 time=12.660 ms
> > 64 bytes from 172.29.253.1: seq=4 ttl=64 time=13.301 ms
> > 64 bytes from 172.29.253.1: seq=5 ttl=64 time=15.457 ms
> > 64 bytes from 172.29.253.1: seq=6 ttl=64 time=13.471 ms
> > 64 bytes from 172.29.253.1: seq=7 ttl=64 time=13.273 ms
> > 64 bytes from 172.29.253.1: seq=8 ttl=64 time=12.644 ms
> > 64 bytes from 172.29.253.1: seq=9 ttl=64 time=13.139 ms
> > 64 bytes from 172.29.253.1: seq=10 ttl=64 time=12.931 ms
> > 64 bytes from 172.29.253.1: seq=27 ttl=64 time=13.851 ms
> > 64 bytes from 172.29.253.1: seq=28 ttl=64 time=12.701 ms
> > 64 bytes from 172.29.253.1: seq=29 ttl=64 time=12.899 ms
> > 64 bytes from 172.29.253.1: seq=30 ttl=64 time=13.522 ms
> > 64 bytes from 172.29.253.1: seq=31 ttl=64 time=13.188 ms
> > 64 bytes from 172.29.253.1: seq=32 ttl=64 time=12.696 ms
> > 64 bytes from 172.29.253.1: seq=33 ttl=64 time=13.680 ms
> > 64 bytes from 172.29.253.1: seq=34 ttl=64 time=16.112 ms
> > ^C
> > --- 172.29.253.1 ping statistics ---
> > 35 packets transmitted, 19 packets received, 45% packet loss
> > round-trip min/avg/max = 12.644/13.437/16.112 ms
> >
> > It would certainly be good to avoid this.
> >
> > Regards
> > Michael Knill
> >
> > On 10/10/18, 2:28 pm, "Lonnie Abelbeck" <li...@lo...>
> wrote:
> >
> >    Hi Michael, comments inline...
> >
> >> On Oct 8, 2018, at 9:20 PM, Michael Knill <
> mic...@ip...> wrote:
> >>
> >> Hi Devs
> >>
> >> I have a couple of questions regarding Wireguard in Astlinux:
> >> • Having to restart Wireguard to re-read the wg0.peer file is not
> particularly good for multiple clients as I assume it drops all the current
> connections? I tried wg setconf wg0 /mnt/kd/wireguard/peer/wg0.peer which
> re-reads the configuration but you still seem to need to restart to
> activate it! I did however try to add a peer this way ‘wg set wg0 peer
> <public key> endpoint <ip:port> allowed-ips <allowed subnet> and it came up
> immediately without a restart. Could we add this into the GUI somehow?
> >
> >    WireGuard is "stateless" so adding/deleting/editing configurations
> should not cause major problems, but some packets could be dropped in the
> process.
> >
> >    Various "wg" sub-commands are:
> >    --
> >    set: Change the current configuration, add peers, remove peers, or
> change peers
> >    setconf: Applies a configuration file to a WireGuard interface
> >    addconf: Appends a configuration file to a WireGuard interface
> >    --
> >
> >    Indeed "wg set wg0 ..." is quite powerful and can change your config
> in realtime with minimal disruption.
> >
> >    I will have to some research to completely answer your question.
> >
> >
> >> • Is there an option for a peer name in wg0.peer other than the
> nonsensical Public Key natively in Wireguard? If not, can we add something
> >
> >    Funny you should mention this, Michael Keuter and I gave this some
> major thought on how to do it ... I even submitted a patch to Jason (he
> rejected it).
> >
> >    The bottom line is any "label" should be part of the overall
> wireguard config and as a result be stored in the kernel, this gave Jason
> some pause with arbitrary label sizes.  Also the effect for VPN providers
> with 1000's of peers needs to be considered.
> >
> >    The other approach is (in user-space) to hack together some sort of
> special comment in the text wireguard config associated with the peer
> PublicKey in a separate database and merge them together with "wg show wg0"
> to create the label-ized output.  Hack'ish to say the least.
> >
> >    This feature has been a common request in the wireguard mailing list.
> >
> >
> >> • The Wireguard VPN status on the Status Tab will start to use up quite
> a bit of space with multiple clients as it uses 6 lines per user whereas
> OpenVPN only uses 1. Should we consider reformatting the output as the
> Status Tab is already quite large when you have a large number of users?
> >
> >    Interesting issue, I guess I have not run into that myself.
> >
> >    The command "wg show wg0 dump" is a more tabular/compact format,
> though for less then 10 the current method seems better.
> >
> >
> >> Looking forward to discussing this.
> >>
> >> Regards
> >> Michael Knill
> >
> >    Lonnie
> >
> >
> >
> >    _______________________________________________
> >    Astlinux-devel mailing list
> >    Ast...@li...
> >    https://lists.sourceforge.net/lists/listinfo/astlinux-devel
> >
> >
> >
> > _______________________________________________
> > Astlinux-devel mailing list
> > Ast...@li...
> > https://lists.sourceforge.net/lists/listinfo/astlinux-devel
>
>
>
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
>

Re: [Astlinux-devel] Multiple peers with Wireguard

[Lonnie A. <li...@lo...>]

Hi Michael,

Given:
pbx2 --WG-- pbx4 --WG-- pbx3

Then ping pbx2 -> pbx3, a restart of wireguard (on pbx4) interrupts the ping for 17 seconds, exactly as your results.

The advantage of using "wg set wg0 ...", for the active peers not changed, their flows are not effected.

While this optimization is possible with WireGuard (unlike OpenVPN) it would involve some tricky coding ... not only changes using "wg set wg0 ..." but the automatic routes created by Allowed IP's would need to be added/removed.  Then the question how the /mnt/kd/wireguard.script would be called for a "incremental" reload.

This requires some thought...

Lonnie



> On Oct 9, 2018, at 11:18 PM, Michael Knill <mic...@ip...> wrote:
> 
> Thanks Lonnie
> 
> Yes restarting Wireguard caused a significant packet loss (ping from endpoint to server):
> PING 172.29.253.1 (172.29.253.1): 56 data bytes
> 64 bytes from 172.29.253.1: seq=0 ttl=64 time=13.475 ms
> 64 bytes from 172.29.253.1: seq=1 ttl=64 time=13.277 ms
> 64 bytes from 172.29.253.1: seq=2 ttl=64 time=13.039 ms
> 64 bytes from 172.29.253.1: seq=3 ttl=64 time=12.660 ms
> 64 bytes from 172.29.253.1: seq=4 ttl=64 time=13.301 ms
> 64 bytes from 172.29.253.1: seq=5 ttl=64 time=15.457 ms
> 64 bytes from 172.29.253.1: seq=6 ttl=64 time=13.471 ms
> 64 bytes from 172.29.253.1: seq=7 ttl=64 time=13.273 ms
> 64 bytes from 172.29.253.1: seq=8 ttl=64 time=12.644 ms
> 64 bytes from 172.29.253.1: seq=9 ttl=64 time=13.139 ms
> 64 bytes from 172.29.253.1: seq=10 ttl=64 time=12.931 ms
> 64 bytes from 172.29.253.1: seq=27 ttl=64 time=13.851 ms
> 64 bytes from 172.29.253.1: seq=28 ttl=64 time=12.701 ms
> 64 bytes from 172.29.253.1: seq=29 ttl=64 time=12.899 ms
> 64 bytes from 172.29.253.1: seq=30 ttl=64 time=13.522 ms
> 64 bytes from 172.29.253.1: seq=31 ttl=64 time=13.188 ms
> 64 bytes from 172.29.253.1: seq=32 ttl=64 time=12.696 ms
> 64 bytes from 172.29.253.1: seq=33 ttl=64 time=13.680 ms
> 64 bytes from 172.29.253.1: seq=34 ttl=64 time=16.112 ms
> ^C
> --- 172.29.253.1 ping statistics ---
> 35 packets transmitted, 19 packets received, 45% packet loss
> round-trip min/avg/max = 12.644/13.437/16.112 ms
> 
> It would certainly be good to avoid this.
> 
> Regards
> Michael Knill
> 
> On 10/10/18, 2:28 pm, "Lonnie Abelbeck" <li...@lo...> wrote:
> 
>    Hi Michael, comments inline...
> 
>> On Oct 8, 2018, at 9:20 PM, Michael Knill <mic...@ip...> wrote:
>> 
>> Hi Devs
>> 
>> I have a couple of questions regarding Wireguard in Astlinux:
>> • Having to restart Wireguard to re-read the wg0.peer file is not particularly good for multiple clients as I assume it drops all the current connections? I tried wg setconf wg0 /mnt/kd/wireguard/peer/wg0.peer which re-reads the configuration but you still seem to need to restart to activate it! I did however try to add a peer this way ‘wg set wg0 peer <public key> endpoint <ip:port> allowed-ips <allowed subnet> and it came up immediately without a restart. Could we add this into the GUI somehow?
> 
>    WireGuard is "stateless" so adding/deleting/editing configurations should not cause major problems, but some packets could be dropped in the process.
> 
>    Various "wg" sub-commands are:
>    --
>    set: Change the current configuration, add peers, remove peers, or change peers
>    setconf: Applies a configuration file to a WireGuard interface
>    addconf: Appends a configuration file to a WireGuard interface
>    --
> 
>    Indeed "wg set wg0 ..." is quite powerful and can change your config in realtime with minimal disruption.
> 
>    I will have to some research to completely answer your question.
> 
> 
>> • Is there an option for a peer name in wg0.peer other than the nonsensical Public Key natively in Wireguard? If not, can we add something
> 
>    Funny you should mention this, Michael Keuter and I gave this some major thought on how to do it ... I even submitted a patch to Jason (he rejected it).
> 
>    The bottom line is any "label" should be part of the overall wireguard config and as a result be stored in the kernel, this gave Jason some pause with arbitrary label sizes.  Also the effect for VPN providers with 1000's of peers needs to be considered.
> 
>    The other approach is (in user-space) to hack together some sort of special comment in the text wireguard config associated with the peer PublicKey in a separate database and merge them together with "wg show wg0" to create the label-ized output.  Hack'ish to say the least.
> 
>    This feature has been a common request in the wireguard mailing list.
> 
> 
>> • The Wireguard VPN status on the Status Tab will start to use up quite a bit of space with multiple clients as it uses 6 lines per user whereas OpenVPN only uses 1. Should we consider reformatting the output as the Status Tab is already quite large when you have a large number of users?
> 
>    Interesting issue, I guess I have not run into that myself.
> 
>    The command "wg show wg0 dump" is a more tabular/compact format, though for less then 10 the current method seems better.
> 
> 
>> Looking forward to discussing this.
>> 
>> Regards
>> Michael Knill
> 
>    Lonnie
> 
> 
> 
>    _______________________________________________
>    Astlinux-devel mailing list
>    Ast...@li...
>    https://lists.sourceforge.net/lists/listinfo/astlinux-devel
> 
> 
> 
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel

Re: [Astlinux-devel] Multiple peers with Wireguard

[Michael K. <mic...@ip...>]

Maybe using the 'wg set' commands, we completely abstract ourselves away from the configuration file and use something like the firewall tab to configure peers. You could put the label in there as well.

Regards
Michael Knill

On 10/10/18, 3:19 pm, "Michael Knill" <mic...@ip...> wrote:

    Thanks Lonnie
    
    Yes restarting Wireguard caused a significant packet loss (ping from endpoint to server):
    PING 172.29.253.1 (172.29.253.1): 56 data bytes
    64 bytes from 172.29.253.1: seq=0 ttl=64 time=13.475 ms
    64 bytes from 172.29.253.1: seq=1 ttl=64 time=13.277 ms
    64 bytes from 172.29.253.1: seq=2 ttl=64 time=13.039 ms
    64 bytes from 172.29.253.1: seq=3 ttl=64 time=12.660 ms
    64 bytes from 172.29.253.1: seq=4 ttl=64 time=13.301 ms
    64 bytes from 172.29.253.1: seq=5 ttl=64 time=15.457 ms
    64 bytes from 172.29.253.1: seq=6 ttl=64 time=13.471 ms
    64 bytes from 172.29.253.1: seq=7 ttl=64 time=13.273 ms
    64 bytes from 172.29.253.1: seq=8 ttl=64 time=12.644 ms
    64 bytes from 172.29.253.1: seq=9 ttl=64 time=13.139 ms
    64 bytes from 172.29.253.1: seq=10 ttl=64 time=12.931 ms
    64 bytes from 172.29.253.1: seq=27 ttl=64 time=13.851 ms
    64 bytes from 172.29.253.1: seq=28 ttl=64 time=12.701 ms
    64 bytes from 172.29.253.1: seq=29 ttl=64 time=12.899 ms
    64 bytes from 172.29.253.1: seq=30 ttl=64 time=13.522 ms
    64 bytes from 172.29.253.1: seq=31 ttl=64 time=13.188 ms
    64 bytes from 172.29.253.1: seq=32 ttl=64 time=12.696 ms
    64 bytes from 172.29.253.1: seq=33 ttl=64 time=13.680 ms
    64 bytes from 172.29.253.1: seq=34 ttl=64 time=16.112 ms
    ^C
    --- 172.29.253.1 ping statistics ---
    35 packets transmitted, 19 packets received, 45% packet loss
    round-trip min/avg/max = 12.644/13.437/16.112 ms
    
    It would certainly be good to avoid this.
    
    Regards
    Michael Knill
    
    On 10/10/18, 2:28 pm, "Lonnie Abelbeck" <li...@lo...> wrote:
    
        Hi Michael, comments inline...
        
        > On Oct 8, 2018, at 9:20 PM, Michael Knill <mic...@ip...> wrote:
        > 
        > Hi Devs
        >  
        > I have a couple of questions regarding Wireguard in Astlinux:
        > • Having to restart Wireguard to re-read the wg0.peer file is not particularly good for multiple clients as I assume it drops all the current connections? I tried wg setconf wg0 /mnt/kd/wireguard/peer/wg0.peer which re-reads the configuration but you still seem to need to restart to activate it! I did however try to add a peer this way ‘wg set wg0 peer <public key> endpoint <ip:port> allowed-ips <allowed subnet> and it came up immediately without a restart. Could we add this into the GUI somehow?
        
        WireGuard is "stateless" so adding/deleting/editing configurations should not cause major problems, but some packets could be dropped in the process.
        
        Various "wg" sub-commands are:
        --
        set: Change the current configuration, add peers, remove peers, or change peers
        setconf: Applies a configuration file to a WireGuard interface
        addconf: Appends a configuration file to a WireGuard interface
        --
        
        Indeed "wg set wg0 ..." is quite powerful and can change your config in realtime with minimal disruption.
        
        I will have to some research to completely answer your question.
        
        
        > • Is there an option for a peer name in wg0.peer other than the nonsensical Public Key natively in Wireguard? If not, can we add something
        
        Funny you should mention this, Michael Keuter and I gave this some major thought on how to do it ... I even submitted a patch to Jason (he rejected it).
        
        The bottom line is any "label" should be part of the overall wireguard config and as a result be stored in the kernel, this gave Jason some pause with arbitrary label sizes.  Also the effect for VPN providers with 1000's of peers needs to be considered.
        
        The other approach is (in user-space) to hack together some sort of special comment in the text wireguard config associated with the peer PublicKey in a separate database and merge them together with "wg show wg0" to create the label-ized output.  Hack'ish to say the least.
        
        This feature has been a common request in the wireguard mailing list.
        
        
        > • The Wireguard VPN status on the Status Tab will start to use up quite a bit of space with multiple clients as it uses 6 lines per user whereas OpenVPN only uses 1. Should we consider reformatting the output as the Status Tab is already quite large when you have a large number of users?
        
        Interesting issue, I guess I have not run into that myself.
        
        The command "wg show wg0 dump" is a more tabular/compact format, though for less then 10 the current method seems better.
        
        
        > Looking forward to discussing this.
        >  
        > Regards
        > Michael Knill
        
        Lonnie
        
        
        
        _______________________________________________
        Astlinux-devel mailing list
        Ast...@li...
        https://lists.sourceforge.net/lists/listinfo/astlinux-devel
        
    
    
    _______________________________________________
    Astlinux-devel mailing list
    Ast...@li...
    https://lists.sourceforge.net/lists/listinfo/astlinux-devel
    

Re: [Astlinux-devel] Multiple peers with Wireguard

[Michael K. <mic...@ip...>]

Thanks Lonnie

Yes restarting Wireguard caused a significant packet loss (ping from endpoint to server):
PING 172.29.253.1 (172.29.253.1): 56 data bytes
64 bytes from 172.29.253.1: seq=0 ttl=64 time=13.475 ms
64 bytes from 172.29.253.1: seq=1 ttl=64 time=13.277 ms
64 bytes from 172.29.253.1: seq=2 ttl=64 time=13.039 ms
64 bytes from 172.29.253.1: seq=3 ttl=64 time=12.660 ms
64 bytes from 172.29.253.1: seq=4 ttl=64 time=13.301 ms
64 bytes from 172.29.253.1: seq=5 ttl=64 time=15.457 ms
64 bytes from 172.29.253.1: seq=6 ttl=64 time=13.471 ms
64 bytes from 172.29.253.1: seq=7 ttl=64 time=13.273 ms
64 bytes from 172.29.253.1: seq=8 ttl=64 time=12.644 ms
64 bytes from 172.29.253.1: seq=9 ttl=64 time=13.139 ms
64 bytes from 172.29.253.1: seq=10 ttl=64 time=12.931 ms
64 bytes from 172.29.253.1: seq=27 ttl=64 time=13.851 ms
64 bytes from 172.29.253.1: seq=28 ttl=64 time=12.701 ms
64 bytes from 172.29.253.1: seq=29 ttl=64 time=12.899 ms
64 bytes from 172.29.253.1: seq=30 ttl=64 time=13.522 ms
64 bytes from 172.29.253.1: seq=31 ttl=64 time=13.188 ms
64 bytes from 172.29.253.1: seq=32 ttl=64 time=12.696 ms
64 bytes from 172.29.253.1: seq=33 ttl=64 time=13.680 ms
64 bytes from 172.29.253.1: seq=34 ttl=64 time=16.112 ms
^C
--- 172.29.253.1 ping statistics ---
35 packets transmitted, 19 packets received, 45% packet loss
round-trip min/avg/max = 12.644/13.437/16.112 ms

It would certainly be good to avoid this.

Regards
Michael Knill

On 10/10/18, 2:28 pm, "Lonnie Abelbeck" <li...@lo...> wrote:

    Hi Michael, comments inline...
    
    > On Oct 8, 2018, at 9:20 PM, Michael Knill <mic...@ip...> wrote:
    > 
    > Hi Devs
    >  
    > I have a couple of questions regarding Wireguard in Astlinux:
    > • Having to restart Wireguard to re-read the wg0.peer file is not particularly good for multiple clients as I assume it drops all the current connections? I tried wg setconf wg0 /mnt/kd/wireguard/peer/wg0.peer which re-reads the configuration but you still seem to need to restart to activate it! I did however try to add a peer this way ‘wg set wg0 peer <public key> endpoint <ip:port> allowed-ips <allowed subnet> and it came up immediately without a restart. Could we add this into the GUI somehow?
    
    WireGuard is "stateless" so adding/deleting/editing configurations should not cause major problems, but some packets could be dropped in the process.
    
    Various "wg" sub-commands are:
    --
    set: Change the current configuration, add peers, remove peers, or change peers
    setconf: Applies a configuration file to a WireGuard interface
    addconf: Appends a configuration file to a WireGuard interface
    --
    
    Indeed "wg set wg0 ..." is quite powerful and can change your config in realtime with minimal disruption.
    
    I will have to some research to completely answer your question.
    
    
    > • Is there an option for a peer name in wg0.peer other than the nonsensical Public Key natively in Wireguard? If not, can we add something
    
    Funny you should mention this, Michael Keuter and I gave this some major thought on how to do it ... I even submitted a patch to Jason (he rejected it).
    
    The bottom line is any "label" should be part of the overall wireguard config and as a result be stored in the kernel, this gave Jason some pause with arbitrary label sizes.  Also the effect for VPN providers with 1000's of peers needs to be considered.
    
    The other approach is (in user-space) to hack together some sort of special comment in the text wireguard config associated with the peer PublicKey in a separate database and merge them together with "wg show wg0" to create the label-ized output.  Hack'ish to say the least.
    
    This feature has been a common request in the wireguard mailing list.
    
    
    > • The Wireguard VPN status on the Status Tab will start to use up quite a bit of space with multiple clients as it uses 6 lines per user whereas OpenVPN only uses 1. Should we consider reformatting the output as the Status Tab is already quite large when you have a large number of users?
    
    Interesting issue, I guess I have not run into that myself.
    
    The command "wg show wg0 dump" is a more tabular/compact format, though for less then 10 the current method seems better.
    
    
    > Looking forward to discussing this.
    >  
    > Regards
    > Michael Knill
    
    Lonnie
    
    
    
    _______________________________________________
    Astlinux-devel mailing list
    Ast...@li...
    https://lists.sourceforge.net/lists/listinfo/astlinux-devel
    

Re: [Astlinux-devel] Multiple peers with Wireguard

[Lonnie A. <li...@lo...>]

Hi Michael, comments inline...

> On Oct 8, 2018, at 9:20 PM, Michael Knill <mic...@ip...> wrote:
> 
> Hi Devs
>  
> I have a couple of questions regarding Wireguard in Astlinux:
> • Having to restart Wireguard to re-read the wg0.peer file is not particularly good for multiple clients as I assume it drops all the current connections? I tried wg setconf wg0 /mnt/kd/wireguard/peer/wg0.peer which re-reads the configuration but you still seem to need to restart to activate it! I did however try to add a peer this way ‘wg set wg0 peer <public key> endpoint <ip:port> allowed-ips <allowed subnet> and it came up immediately without a restart. Could we add this into the GUI somehow?

WireGuard is "stateless" so adding/deleting/editing configurations should not cause major problems, but some packets could be dropped in the process.

Various "wg" sub-commands are:
--
set: Change the current configuration, add peers, remove peers, or change peers
setconf: Applies a configuration file to a WireGuard interface
addconf: Appends a configuration file to a WireGuard interface
--

Indeed "wg set wg0 ..." is quite powerful and can change your config in realtime with minimal disruption.

I will have to some research to completely answer your question.


> • Is there an option for a peer name in wg0.peer other than the nonsensical Public Key natively in Wireguard? If not, can we add something

Funny you should mention this, Michael Keuter and I gave this some major thought on how to do it ... I even submitted a patch to Jason (he rejected it).

The bottom line is any "label" should be part of the overall wireguard config and as a result be stored in the kernel, this gave Jason some pause with arbitrary label sizes.  Also the effect for VPN providers with 1000's of peers needs to be considered.

The other approach is (in user-space) to hack together some sort of special comment in the text wireguard config associated with the peer PublicKey in a separate database and merge them together with "wg show wg0" to create the label-ized output.  Hack'ish to say the least.

This feature has been a common request in the wireguard mailing list.


> • The Wireguard VPN status on the Status Tab will start to use up quite a bit of space with multiple clients as it uses 6 lines per user whereas OpenVPN only uses 1. Should we consider reformatting the output as the Status Tab is already quite large when you have a large number of users?

Interesting issue, I guess I have not run into that myself.

The command "wg show wg0 dump" is a more tabular/compact format, though for less then 10 the current method seems better.


> Looking forward to discussing this.
>  
> Regards
> Michael Knill

Lonnie

[Astlinux-devel] Multiple peers with Wireguard

[Michael K. <mic...@ip...>]

Hi Devs

I have a couple of questions regarding Wireguard in Astlinux:

  1.  Having to restart Wireguard to re-read the wg0.peer file is not particularly good for multiple clients as I assume it drops all the current connections? I tried wg setconf wg0 /mnt/kd/wireguard/peer/wg0.peer which re-reads the configuration but you still seem to need to restart to activate it! I did however try to add a peer this way ‘wg set wg0 peer <public key> endpoint <ip:port> allowed-ips <allowed subnet> and it came up immediately without a restart. Could we add this into the GUI somehow?
  2.  Is there an option for a peer name in wg0.peer other than the nonsensical Public Key natively in Wireguard? If not, can we add something
  3.  The Wireguard VPN status on the Status Tab will start to use up quite a bit of space with multiple clients as it uses 6 lines per user whereas OpenVPN only uses 1. Should we consider reformatting the output as the Status Tab is already quite large when you have a large number of users?
Looking forward to discussing this.

Regards
Michael Knill

[Astlinux-devel] Announcing AstLinux Release: 1.3.4

[Lonnie A. <li...@lo...>]

Announcing AstLinux Release: 1.3.4

More Info: AstLinux Project
https://www.astlinux-project.org/

AstLinux 1.3.4 Highlights:
* Asterisk Versions: 11.25.3, 13.23.1
* Upgrade to Linux Kernel 3.16.57, including the RUNNIX bootloader, security and bug fixes
* genx86_64-vm board type, add support for virtio-blk as a bootable disk driver, also added to RUNNIX
* genx86_64-vm board type, add support for QEMU Guest Agent "qemu-ga"
* rng-tools, new package, Random Number Generator daemon "rngd", enabled by default
* keepalived, new package, VRRP High Availability daemon "keepalived"
* msmtpd, added SMTP localhost daemon to forward 127.0.0.1:25 to "sendmail", enabled by default
* WireGuard VPN, latest development snapshots during its incorporation into the Linux Kernel
* Package upgrades providing important security and bug fixes

Full ChangeLog:
https://raw.githubusercontent.com/astlinux-project/astlinux/1.3.4/docs/ChangeLog.txt

All users are encouraged to upgrade.

AstLinux Team

[Astlinux-devel] AstLinux Pre-Release: astlinux-1.3-3915-85f590

[Lonnie A. <li...@lo...>]

Announcing Pre-Release Version: astlinux-1.3-3915-85f590

If there are no issues, this version will be tagged as AstLinux 1.3.4 .

The AstLinux Team is regularly upgrading packages containing security and bug fixes as well as adding new features of our own.

-- Linux Kernel 3.16.57, security and bug fixes.

-- genx86_64-vm board type, add support for virtio-blk as a bootable disk driver, also added to RUNNIX.
  Tested via Proxmox and Vultr (hosted).

-- genx86_64-vm board type, add support for QEMU Guest Agent (qemu-ga 2.12.0).

-- rng-tools, new package, version 6.5, Random Number Generator (RNG) daemon
  Enabled by default to increase the available entropy for the kernel's "random" sources.
  Uses one of 3 sources, in order:
  1) /dev/hwrng (typically via virtio-rng for the genx86_64-vm board type)
  2) Intel RDRAND instruction on supported CPU's
  3) jitterentropy, Hardware RNG based on CPU timing jitter

-- keepalived, new package, version 2.0.7, VRRP High Availability daemon

-- msmtpd (msmtp 1.8.0), added SMTP localhost daemon to forward 127.0.0.1:25 to "sendmail", enabled by default.

-- Asterisk 13 version bump to 13.23.1

New Documentation Topics:
VRRP High Availability Daemon (keepalived) -- https://doc.astlinux-project.org/userdoc:tt_high_availability

Qotom Q530G6 Core i3-6100U Fanless Appliance -- https://doc.astlinux-project.org/userdoc:board_qotom_q530g6

Linode KVM -- https://doc.astlinux-project.org/userdoc:hosted_guest_vm_linode

Vultr KVM -- https://doc.astlinux-project.org/userdoc:hosted_guest_vm_vultr

Updated Documentation Topics:
WAN Failover -- https://doc.astlinux-project.org/userdoc:tt_wan_failover#example4g_lte_modem_failover

Tarsnap Online Backup -- https://doc.astlinux-project.org/userdoc:tt_tarsnap_online_backup#restore_to_a_new_install


The "AstLinux Pre-Release ChangeLog" and "Repository URL" entries can be found under the "Development" tab of the AstLinux Project web site ...

AstLinux Project -> Development
https://www.astlinux-project.org/dev.html

AstLinux Team

Re: [Astlinux-devel] Tarsnap restore

[Michael K. <mic...@ip...>]

Great thanks guys

Ps I wrote a restore script whereby you can restore all files, a single file, or just the database. Seems to work well, especially the database restore. Not sure if its useful for anyone.

tarsnap_restore <archive> [<fname>|db|all]

--------------
if [ -z $2 ]; then
		echo;echo "No archive name provided"; echo
	exit
fi
if [ -z $3 ]; then
		echo;echo "No file(s) to restore"; echo
	exit
else
	case $3 in
	"all")
		echo; echo "WARNING: This action will overwrite your current configuration files and database with the entire archive $2"
		echo "It will also stop and restart Asterisk cutting off any calls that may be in progress"
		read -p 'Do you know what you are doing? (y/N):' restore
		if [ "$restore" == "y" ] || [ "$restore" == "Y" ]; then
			echo; echo "Ok restoring all files now..."; echo
			/usr/bin/tarsnap -xvf "$2" -C /mnt/kd
			echo; echo "Restoring database..."
			echo "Shutting down Asterisk"
			service asterisk stop >/dev/null
			sleep 2
			sqlite3 /mnt/kd/astdb.sqlite3 'DROP TABLE IF EXISTS astdb;' >/dev/null
			sqlite3 /mnt/kd/astdb.sqlite3 < /mnt/kd/astdb_contents.dump >/dev/null
			echo "Starting Asterisk"
			service asterisk start >/dev/null
			sleep 2
			echo; echo "Restoration of archive $2 complete"; echo
		else
				echo; echo "Archive $2 was not restored"; echo
		fi
	;;
	"db")
		echo; echo "WARNING: This action will overwrite your current database with the database in archive $2"
		echo "It will also stop and restart Asterisk cutting off any calls that may be in progress"
		read -p 'Do you know what you are doing? (y/N):' restore
		if [ "$restore" == "y" ] || [ "$restore" == "Y" ]; then
			echo; echo "Ok restoring database now..."; echo
			/usr/bin/tarsnap -xvf "$2" -C /mnt/kd astdb_contents.dump
			echo "Shutting down Asterisk"
			service asterisk stop >/dev/null
			sleep 2
			sqlite3 /mnt/kd/astdb.sqlite3 'DROP TABLE IF EXISTS astdb;' >/dev/null
			sqlite3 /mnt/kd/astdb.sqlite3 < /mnt/kd/astdb_contents.dump >/dev/null
			echo "Starting Asterisk"
			service asterisk start >/dev/null
			sleep 2
			echo; echo "Restoration of database from archive $2 complete"; echo
		else
				echo; echo "Database was not restored"; echo
		fi
	;;
	*)
		echo; echo "WARNING: This action will overwrite the file $3 from the archive $2"
		read -p 'Do you know what you are doing? (y/N):' restore
		if [ "$restore" == "y" ] || [ "$restore" == "Y" ]; then
			echo "Ok restoring $3 now"
			/usr/bin/tarsnap -xvf "$2" -C /mnt/kd "$3"
			echo; echo "Restoration of $3 complete"; echo
		else
			echo; echo "$3 was not restored"; echo
		fi
	;;
	esac
fi
exit
-----------------------

Regards
Michael Knill

On 30/9/18, 12:45 am, "Michael Keuter" <li...@mk...> wrote:

    
    > Am 29.09.2018 um 09:12 schrieb Michael Knill <mic...@ip...>:
    > 
    > Hi Devs
    >  
    > Im thinking that we should mention in the Tarsnap Online Backup doco about what needs to be done if you are restoring to a new box .
    > I think the process is as follows:
    >  
    > 	• Build new Astlinux server
    > 	• Copy stored tarsnap.key to /mnt/kd/tarsnap
    > 	• tarsnap -xvf <archive> -C /mnt/kd to restore all files
    > 	• tarsnap –fsck to resync cache so you can backup again. If you don't you get:
    > tarsnap: Sequence number mismatch: Run --fsck
    > tarsnap: Error creating new archive
    > tarsnap: Error exit delayed from previous errors.
    
    I added the "tarsnap --fsck" part to our Wiki.
    And a link to the Action Script. 
    
    > It may be worth also mentioning the process for restoring the database.dump file:
    > e.g.
    > echo; echo "Restoring database..."
    > echo "Shutting down Asterisk"
    > service asterisk stop >/dev/null
    > sleep 2
    > sqlite3 /mnt/kd/astdb.sqlite3 'DROP TABLE IF EXISTS astdb;' >/dev/null
    > sqlite3 /mnt/kd/astdb.sqlite3 < /mnt/kd/database.dump >/dev/null
    > echo "Starting Asterisk"
    > service asterisk start >/dev/null
    > sleep 2
    >  
    > PS I am not a scripting expert so I probably have done some things the long way.
    >  
    > What do you think?
    >  
    > Regards
    > Michael Knill
    
    Michael
    
    http://www.mksolutions.info
    
    
    
    
    
    _______________________________________________
    Astlinux-devel mailing list
    Ast...@li...
    https://lists.sourceforge.net/lists/listinfo/astlinux-devel
    

Re: [Astlinux-devel] Tarsnap restore

[Lonnie A. <li...@lo...>]

> On Sep 29, 2018, at 9:27 AM, Michael Keuter <li...@mk...> wrote:
> 
> 
>> Am 29.09.2018 um 09:12 schrieb Michael Knill <mic...@ip...>:
>> 
>> Hi Devs
>> 
>> Im thinking that we should mention in the Tarsnap Online Backup doco about what needs to be done if you are restoring to a new box .
>> I think the process is as follows:
>> 
>> 	• Build new Astlinux server
>> 	• Copy stored tarsnap.key to /mnt/kd/tarsnap
>> 	• tarsnap -xvf <archive> -C /mnt/kd to restore all files
>> 	• tarsnap –fsck to resync cache so you can backup again. If you don't you get:
>> tarsnap: Sequence number mismatch: Run --fsck
>> tarsnap: Error creating new archive
>> tarsnap: Error exit delayed from previous errors.
> 
> I added the "tarsnap --fsck" part to our Wiki.
> And a link to the Action Script. 

I also added a new "Restore to a New Install" section:

https://doc.astlinux-project.org/userdoc:tt_tarsnap_online_backup#restore_to_a_new_install

Let me know if I missed anything.

Lonnie

Re: [Astlinux-devel] Tarsnap restore

[Michael K. <li...@mk...>]

> Am 29.09.2018 um 09:12 schrieb Michael Knill <mic...@ip...>:
> 
> Hi Devs
>  
> Im thinking that we should mention in the Tarsnap Online Backup doco about what needs to be done if you are restoring to a new box .
> I think the process is as follows:
>  
> 	• Build new Astlinux server
> 	• Copy stored tarsnap.key to /mnt/kd/tarsnap
> 	• tarsnap -xvf <archive> -C /mnt/kd to restore all files
> 	• tarsnap –fsck to resync cache so you can backup again. If you don't you get:
> tarsnap: Sequence number mismatch: Run --fsck
> tarsnap: Error creating new archive
> tarsnap: Error exit delayed from previous errors.

I added the "tarsnap --fsck" part to our Wiki.
And a link to the Action Script. 

> It may be worth also mentioning the process for restoring the database.dump file:
> e.g.
> echo; echo "Restoring database..."
> echo "Shutting down Asterisk"
> service asterisk stop >/dev/null
> sleep 2
> sqlite3 /mnt/kd/astdb.sqlite3 'DROP TABLE IF EXISTS astdb;' >/dev/null
> sqlite3 /mnt/kd/astdb.sqlite3 < /mnt/kd/database.dump >/dev/null
> echo "Starting Asterisk"
> service asterisk start >/dev/null
> sleep 2
>  
> PS I am not a scripting expert so I probably have done some things the long way.
>  
> What do you think?
>  
> Regards
> Michael Knill

Michael

http://www.mksolutions.info

[Astlinux-devel] Tarsnap restore

[Michael K. <mic...@ip...>]

Hi Devs

Im thinking that we should mention in the Tarsnap Online Backup doco about what needs to be done if you are restoring to a new box .
I think the process is as follows:


  1.  Build new Astlinux server
  2.  Copy stored tarsnap.key to /mnt/kd/tarsnap
  3.  tarsnap -xvf <archive> -C /mnt/kd to restore all files
  4.  tarsnap –fsck to resync cache so you can backup again. If you don't you get:
tarsnap: Sequence number mismatch: Run --fsck
tarsnap: Error creating new archive
tarsnap: Error exit delayed from previous errors.

It may be worth also mentioning the process for restoring the database.dump file:
e.g.
echo; echo "Restoring database..."
echo "Shutting down Asterisk"
service asterisk stop >/dev/null
sleep 2
sqlite3 /mnt/kd/astdb.sqlite3 'DROP TABLE IF EXISTS astdb;' >/dev/null
sqlite3 /mnt/kd/astdb.sqlite3 < /mnt/kd/database.dump >/dev/null
echo "Starting Asterisk"
service asterisk start >/dev/null
sleep 2

PS I am not a scripting expert so I probably have done some things the long way.

What do you think?

Regards
Michael Knill

[Astlinux-devel] AstLinux Pre-Release: astlinux-1.3-3888-523a33

[Lonnie A. <li...@lo...>]

Announcing Pre-Release Version: astlinux-1.3-3888-523a33

The AstLinux Team is regularly upgrading packages containing security and bug fixes as well as adding new features of our own.

-- Linux Kernel 3.16.57, security and bug fixes.

-- genx86_64-vm board type, add support for virtio-blk as a bootable disk driver, also added to RUNNIX.
   Tested via Proxmox and Vultr (hosted).

-- genx86_64-vm board type, add support for QEMU Guest Agent (qemu-ga 2.12.0).

-- rng-tools, new package, version 6.4, Random Number Generator (RNG) daemon
   Enabled by default to increase the available entropy for the kernel's "random" sources.
   Uses one of 3 sources, in order:
   1) /dev/hwrng (typically via virtio-rng for the genx86_64-vm board type)
   2) Intel RDRAND instruction on supported CPU's
   3) jitterentropy, Hardware RNG based on CPU timing jitter

-- keepalived, new package, version 2.0.7, VRRP High Availability daemon

-- msmtpd (msmtp 1.8.0), added SMTP localhost daemon to forward 127.0.0.1:25 to "sendmail", enabled by default.

-- Asterisk 13 version bump to 13.23.0

New Documentation Topics:
VRRP High Availability Daemon (keepalived) -- https://doc.astlinux-project.org/userdoc:tt_high_availability

Qotom Q530G6 Core i3-6100U Fanless Appliance -- https://doc.astlinux-project.org/userdoc:board_qotom_q530g6

Linode KVM -- https://doc.astlinux-project.org/userdoc:hosted_guest_vm_linode

Vultr KVM -- https://doc.astlinux-project.org/userdoc:hosted_guest_vm_vultr

Updated Documentation Topics:
WAN Failover -- https://doc.astlinux-project.org/userdoc:tt_wan_failover#example4g_lte_modem_failover

These pre-release images are for those who would like to take advantage of the AstLinux development before the next official release, as well as providing testing for the project.

The "AstLinux Pre-Release ChangeLog" and "Repository URL" entries can be found under the "Development" tab of the AstLinux Project web site ...

AstLinux Project -> Development
https://www.astlinux-project.org/dev.html

While these images are considered 'stable', the lack of testing will not make these images suitable for critical production systems.

If you should come across an issue, please report back here.

AstLinux Team

Re: [Astlinux-devel] Tarsnap Backups

[Michael K. <mic...@ip...>]

Cool thanks Lonnie. I understand.
I guess that I have set up things so that I need to know all files changed from a release reference file list and this is effectively what I back up.

Im looking forward to playing with it.

Regards
Michael Knill

On 7/9/18, 8:07 am, "Lonnie Abelbeck" <li...@lo...> wrote:

    I guess I'm saying rsync before tarsnap-backup seems redundant since tarsnap will find all the changed files in the Dirs/Files you specify.
    
    Lonnie
    
    
    > On Sep 6, 2018, at 4:55 PM, Michael Knill <mic...@ip...> wrote:
    > 
    > Thanks Lonnie
    > 
    > Yes I certainly hear what you are saying and it does sort of defeat the store only once purpose of deduplication but does it break it? 
    > Effectively all I am doing is presenting a list of included files to tarsnap!
    > 
    > Regards
    > Michael Knill
    > 
    > On 7/9/18, 4:49 am, "Lonnie Abelbeck" <li...@lo...> wrote:
    > 
    >    Hi Michael,
    > 
    >> So I assume that I can just replace 2) with tarsnap
    > 
    >    No, that would somewhat defeat tarsnap's "Deduplication" http://www.tarsnap.com feature, which saves you money.
    > 
    >    I would replace all 1-3 steps with tarsnap.  Each AstLinux box would have their own tarsnap.key .
    > 
    >    The web interface
    >    Network tab -> Tarsnap Backup: { Tarsnap Backup Options }
    > 
    >    defines the following variables
    >    --
    >    BACKUP_KD_DEFAULTS
    >    BACKUP_KD_INCLUDE_DIRS
    >    BACKUP_KD_INCLUDE_FILES
    >    BACKUP_ASTURW_DEFAULTS
    >    BACKUP_ASTURW_INCLUDE_DIRS
    >    BACKUP_ASTURW_INCLUDE_FILES
    >    BACKUP_PRUNE_AGE_DAYS
    >    BACKUP_NOTIFY
    >    BACKUP_NOTIFY_FROM
    >    --
    > 
    >    Files/Dirs automatically excluded via /etc/tarsnap.conf:
    >    --
    >    exclude *.bak
    >    exclude *.db
    >    exclude *.mdb
    >    exclude *.sqlite3
    >    exclude *.fossil
    >    exclude *.netset
    >    exclude *.rom
    >    exclude *.bin
    >    --
    > 
    >    For most users the defaults just work, and possibly add a few tweaks.
    > 
    >    If you want *everything* (minus the excluded suffixes), disable both "Backup [kd] Defaults" and "Backup [asturw] Defaults:" and specify "*" for all four Backup [kd|asturw] Dirs/Files entries.  Though I would doubt many users would do that.
    > 
    >    Be sure to perform some dry-run testing from the command line to help understand what is happening.
    >    --
    >    tarsnap-backup --help
    >    --
    > 
    > 
    >> PS the doco mentions that I can edit /etc/tarsnap.conf. Does that mean that /mnt/kd/tarsnap.conf will take precedence or do I need to edit this file?
    > 
    >    It was intended that most users will never need to edit the /etc/tarsnap.conf file, and as such it is not linked to /mnt/kd/tarsnap.conf .  Editing /etc/tarsnap.conf will use unionfs just as editing /etc/rc.modules does.
    > 
    >    Lonnie
    > 
    > 
    > 
    >> On Sep 5, 2018, at 9:02 PM, Michael Knill <mic...@ip...> wrote:
    >> 
    >> Hi Devs
    >> 
    >> I have decided that I will be changing my current backup strategy to Tarsnap for the following reasons:
    >> 	• I need to back up sensitive files securely
    >> 	• I need to be performing daily backups as I suspect a few of my customers will be doing their own basic administration soon
    >> 	• And it looks awesome from my preliminary tests
    >> 
    >> My current backup strategy is as follows:
    >> 	1) Find all changed files with rsync by comparing files in /mnt/kd and /mnt/kd/.ref and using an excluded directory/file list.
    >> 	2) Use tar to back up all the resulting files on the Astlinux server
    >> 	3) Move the backup file to the management server via SCP
    >> 
    >> So I assume that I can just replace 2) with tarsnap instead of tar and not do 3) but the problem is that I cant do this natively using the tarsnap-backup script.
    >> Will I need to modify/recreate it or is there another way?
    >> 
    >> PS the doco mentions that I can edit /etc/tarsnap.conf. Does that mean that /mnt/kd/tarsnap.conf will take precedence or do I need to edit this file?
    >> 
    >> Thanks
    >> 
    >> Regards
    >> Michael Knill
    >> ------------------------------------------------------------------------------
    >> Check out the vibrant tech community on one of the world's most
    >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
    >> Astlinux-devel mailing list
    >> Ast...@li...
    >> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
    > 
    > 
    >    ------------------------------------------------------------------------------
    >    Check out the vibrant tech community on one of the world's most
    >    engaging tech sites, Slashdot.org! http://sdm.link/slashdot
    >    _______________________________________________
    >    Astlinux-devel mailing list
    >    Ast...@li...
    >    https://lists.sourceforge.net/lists/listinfo/astlinux-devel
    > 
    > 
    > ------------------------------------------------------------------------------
    > Check out the vibrant tech community on one of the world's most
    > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
    > _______________________________________________
    > Astlinux-devel mailing list
    > Ast...@li...
    > https://lists.sourceforge.net/lists/listinfo/astlinux-devel
    
    
    ------------------------------------------------------------------------------
    Check out the vibrant tech community on one of the world's most
    engaging tech sites, Slashdot.org! http://sdm.link/slashdot
    _______________________________________________
    Astlinux-devel mailing list
    Ast...@li...
    https://lists.sourceforge.net/lists/listinfo/astlinux-devel
    

Re: [Astlinux-devel] Tarsnap Backups

[Lonnie A. <li...@lo...>]

I guess I'm saying rsync before tarsnap-backup seems redundant since tarsnap will find all the changed files in the Dirs/Files you specify.

Lonnie


> On Sep 6, 2018, at 4:55 PM, Michael Knill <mic...@ip...> wrote:
> 
> Thanks Lonnie
> 
> Yes I certainly hear what you are saying and it does sort of defeat the store only once purpose of deduplication but does it break it? 
> Effectively all I am doing is presenting a list of included files to tarsnap!
> 
> Regards
> Michael Knill
> 
> On 7/9/18, 4:49 am, "Lonnie Abelbeck" <li...@lo...> wrote:
> 
>    Hi Michael,
> 
>> So I assume that I can just replace 2) with tarsnap
> 
>    No, that would somewhat defeat tarsnap's "Deduplication" http://www.tarsnap.com feature, which saves you money.
> 
>    I would replace all 1-3 steps with tarsnap.  Each AstLinux box would have their own tarsnap.key .
> 
>    The web interface
>    Network tab -> Tarsnap Backup: { Tarsnap Backup Options }
> 
>    defines the following variables
>    --
>    BACKUP_KD_DEFAULTS
>    BACKUP_KD_INCLUDE_DIRS
>    BACKUP_KD_INCLUDE_FILES
>    BACKUP_ASTURW_DEFAULTS
>    BACKUP_ASTURW_INCLUDE_DIRS
>    BACKUP_ASTURW_INCLUDE_FILES
>    BACKUP_PRUNE_AGE_DAYS
>    BACKUP_NOTIFY
>    BACKUP_NOTIFY_FROM
>    --
> 
>    Files/Dirs automatically excluded via /etc/tarsnap.conf:
>    --
>    exclude *.bak
>    exclude *.db
>    exclude *.mdb
>    exclude *.sqlite3
>    exclude *.fossil
>    exclude *.netset
>    exclude *.rom
>    exclude *.bin
>    --
> 
>    For most users the defaults just work, and possibly add a few tweaks.
> 
>    If you want *everything* (minus the excluded suffixes), disable both "Backup [kd] Defaults" and "Backup [asturw] Defaults:" and specify "*" for all four Backup [kd|asturw] Dirs/Files entries.  Though I would doubt many users would do that.
> 
>    Be sure to perform some dry-run testing from the command line to help understand what is happening.
>    --
>    tarsnap-backup --help
>    --
> 
> 
>> PS the doco mentions that I can edit /etc/tarsnap.conf. Does that mean that /mnt/kd/tarsnap.conf will take precedence or do I need to edit this file?
> 
>    It was intended that most users will never need to edit the /etc/tarsnap.conf file, and as such it is not linked to /mnt/kd/tarsnap.conf .  Editing /etc/tarsnap.conf will use unionfs just as editing /etc/rc.modules does.
> 
>    Lonnie
> 
> 
> 
>> On Sep 5, 2018, at 9:02 PM, Michael Knill <mic...@ip...> wrote:
>> 
>> Hi Devs
>> 
>> I have decided that I will be changing my current backup strategy to Tarsnap for the following reasons:
>> 	• I need to back up sensitive files securely
>> 	• I need to be performing daily backups as I suspect a few of my customers will be doing their own basic administration soon
>> 	• And it looks awesome from my preliminary tests
>> 
>> My current backup strategy is as follows:
>> 	1) Find all changed files with rsync by comparing files in /mnt/kd and /mnt/kd/.ref and using an excluded directory/file list.
>> 	2) Use tar to back up all the resulting files on the Astlinux server
>> 	3) Move the backup file to the management server via SCP
>> 
>> So I assume that I can just replace 2) with tarsnap instead of tar and not do 3) but the problem is that I cant do this natively using the tarsnap-backup script.
>> Will I need to modify/recreate it or is there another way?
>> 
>> PS the doco mentions that I can edit /etc/tarsnap.conf. Does that mean that /mnt/kd/tarsnap.conf will take precedence or do I need to edit this file?
>> 
>> Thanks
>> 
>> Regards
>> Michael Knill
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
>> Astlinux-devel mailing list
>> Ast...@li...
>> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
> 
> 
>    ------------------------------------------------------------------------------
>    Check out the vibrant tech community on one of the world's most
>    engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>    _______________________________________________
>    Astlinux-devel mailing list
>    Ast...@li...
>    https://lists.sourceforge.net/lists/listinfo/astlinux-devel
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel

Re: [Astlinux-devel] Tarsnap Backups

[Michael K. <mic...@ip...>]

Thanks Lonnie

Yes I certainly hear what you are saying and it does sort of defeat the store only once purpose of deduplication but does it break it? 
Effectively all I am doing is presenting a list of included files to tarsnap!

Regards
Michael Knill

On 7/9/18, 4:49 am, "Lonnie Abelbeck" <li...@lo...> wrote:

    Hi Michael,
    
    > So I assume that I can just replace 2) with tarsnap
    
    No, that would somewhat defeat tarsnap's "Deduplication" http://www.tarsnap.com feature, which saves you money.
    
    I would replace all 1-3 steps with tarsnap.  Each AstLinux box would have their own tarsnap.key .
    
    The web interface
    Network tab -> Tarsnap Backup: { Tarsnap Backup Options }
    
    defines the following variables
    --
    BACKUP_KD_DEFAULTS
    BACKUP_KD_INCLUDE_DIRS
    BACKUP_KD_INCLUDE_FILES
    BACKUP_ASTURW_DEFAULTS
    BACKUP_ASTURW_INCLUDE_DIRS
    BACKUP_ASTURW_INCLUDE_FILES
    BACKUP_PRUNE_AGE_DAYS
    BACKUP_NOTIFY
    BACKUP_NOTIFY_FROM
    --
    
    Files/Dirs automatically excluded via /etc/tarsnap.conf:
    --
    exclude *.bak
    exclude *.db
    exclude *.mdb
    exclude *.sqlite3
    exclude *.fossil
    exclude *.netset
    exclude *.rom
    exclude *.bin
    --
    
    For most users the defaults just work, and possibly add a few tweaks.
    
    If you want *everything* (minus the excluded suffixes), disable both "Backup [kd] Defaults" and "Backup [asturw] Defaults:" and specify "*" for all four Backup [kd|asturw] Dirs/Files entries.  Though I would doubt many users would do that.
    
    Be sure to perform some dry-run testing from the command line to help understand what is happening.
    --
    tarsnap-backup --help
    --
    
    
    > PS the doco mentions that I can edit /etc/tarsnap.conf. Does that mean that /mnt/kd/tarsnap.conf will take precedence or do I need to edit this file?
    
    It was intended that most users will never need to edit the /etc/tarsnap.conf file, and as such it is not linked to /mnt/kd/tarsnap.conf .  Editing /etc/tarsnap.conf will use unionfs just as editing /etc/rc.modules does.
    
    Lonnie
    
    
    
    > On Sep 5, 2018, at 9:02 PM, Michael Knill <mic...@ip...> wrote:
    > 
    > Hi Devs
    >  
    > I have decided that I will be changing my current backup strategy to Tarsnap for the following reasons:
    > 	• I need to back up sensitive files securely
    > 	• I need to be performing daily backups as I suspect a few of my customers will be doing their own basic administration soon
    > 	• And it looks awesome from my preliminary tests
    >  
    > My current backup strategy is as follows:
    > 	1) Find all changed files with rsync by comparing files in /mnt/kd and /mnt/kd/.ref and using an excluded directory/file list.
    > 	2) Use tar to back up all the resulting files on the Astlinux server
    > 	3) Move the backup file to the management server via SCP
    >  
    > So I assume that I can just replace 2) with tarsnap instead of tar and not do 3) but the problem is that I cant do this natively using the tarsnap-backup script.
    > Will I need to modify/recreate it or is there another way?
    >  
    > PS the doco mentions that I can edit /etc/tarsnap.conf. Does that mean that /mnt/kd/tarsnap.conf will take precedence or do I need to edit this file?
    >  
    > Thanks
    >  
    > Regards
    > Michael Knill
    > ------------------------------------------------------------------------------
    > Check out the vibrant tech community on one of the world's most
    > engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
    > Astlinux-devel mailing list
    > Ast...@li...
    > https://lists.sourceforge.net/lists/listinfo/astlinux-devel
    
    
    ------------------------------------------------------------------------------
    Check out the vibrant tech community on one of the world's most
    engaging tech sites, Slashdot.org! http://sdm.link/slashdot
    _______________________________________________
    Astlinux-devel mailing list
    Ast...@li...
    https://lists.sourceforge.net/lists/listinfo/astlinux-devel
    

Re: [Astlinux-devel] Tarsnap Backups

[Lonnie A. <li...@lo...>]

Hi Michael,

> So I assume that I can just replace 2) with tarsnap

No, that would somewhat defeat tarsnap's "Deduplication" http://www.tarsnap.com feature, which saves you money.

I would replace all 1-3 steps with tarsnap.  Each AstLinux box would have their own tarsnap.key .

The web interface
Network tab -> Tarsnap Backup: { Tarsnap Backup Options }

defines the following variables
--
BACKUP_KD_DEFAULTS
BACKUP_KD_INCLUDE_DIRS
BACKUP_KD_INCLUDE_FILES
BACKUP_ASTURW_DEFAULTS
BACKUP_ASTURW_INCLUDE_DIRS
BACKUP_ASTURW_INCLUDE_FILES
BACKUP_PRUNE_AGE_DAYS
BACKUP_NOTIFY
BACKUP_NOTIFY_FROM
--

Files/Dirs automatically excluded via /etc/tarsnap.conf:
--
exclude *.bak
exclude *.db
exclude *.mdb
exclude *.sqlite3
exclude *.fossil
exclude *.netset
exclude *.rom
exclude *.bin
--

For most users the defaults just work, and possibly add a few tweaks.

If you want *everything* (minus the excluded suffixes), disable both "Backup [kd] Defaults" and "Backup [asturw] Defaults:" and specify "*" for all four Backup [kd|asturw] Dirs/Files entries.  Though I would doubt many users would do that.

Be sure to perform some dry-run testing from the command line to help understand what is happening.
--
tarsnap-backup --help
--


> PS the doco mentions that I can edit /etc/tarsnap.conf. Does that mean that /mnt/kd/tarsnap.conf will take precedence or do I need to edit this file?

It was intended that most users will never need to edit the /etc/tarsnap.conf file, and as such it is not linked to /mnt/kd/tarsnap.conf .  Editing /etc/tarsnap.conf will use unionfs just as editing /etc/rc.modules does.

Lonnie



> On Sep 5, 2018, at 9:02 PM, Michael Knill <mic...@ip...> wrote:
> 
> Hi Devs
>  
> I have decided that I will be changing my current backup strategy to Tarsnap for the following reasons:
> 	• I need to back up sensitive files securely
> 	• I need to be performing daily backups as I suspect a few of my customers will be doing their own basic administration soon
> 	• And it looks awesome from my preliminary tests
>  
> My current backup strategy is as follows:
> 	1) Find all changed files with rsync by comparing files in /mnt/kd and /mnt/kd/.ref and using an excluded directory/file list.
> 	2) Use tar to back up all the resulting files on the Astlinux server
> 	3) Move the backup file to the management server via SCP
>  
> So I assume that I can just replace 2) with tarsnap instead of tar and not do 3) but the problem is that I cant do this natively using the tarsnap-backup script.
> Will I need to modify/recreate it or is there another way?
>  
> PS the doco mentions that I can edit /etc/tarsnap.conf. Does that mean that /mnt/kd/tarsnap.conf will take precedence or do I need to edit this file?
>  
> Thanks
>  
> Regards
> Michael Knill
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel

Re: [Astlinux-devel] Comcast/Xfinity DOCSIS 3.1

[Lonnie A. <li...@lo...>]

DOCSIS 3.1 was enabled overnight at my location, the simple "mtr" is better than before...

For my COX DOCSIS 3.1 (to the default gateway address)
--
gw-lan ~ # mtr -wn z.y.x.w
Start: 2018-09-06T07:31:30-0500
HOST: gw-lan       Loss%   Snt   Last   Avg  Best  Wrst StDev
 1.|-- z.y.x.w     0.0%    10    6.0   6.5   6.0   7.1   0.5
--

Hardly a definitive test, but jitter is measurably lower than before.

Using  iperf3 to my Linode instance, 9-hops, 31.5 ms away...

gw-lan ~ # iperf3 -c host.tld -u
Connecting to host host.tld, port 5201
...
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  1.25 MBytes  1.05 Mbits/sec  0.000 ms  0/906 (0%)  sender
[  5]   0.00-10.03  sec  1.25 MBytes  1.05 Mbits/sec  1.324 ms  0/905 (0%)  receiver

gw-lan ~ # iperf3 -c host.tld -u -R
Connecting to host host.tld, port 5201
Reverse mode, remote host host.tld is sending
...
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.03  sec  1.26 MBytes  1.05 Mbits/sec  0.000 ms  0/909 (0%)  sender
[  5]   0.00-10.00  sec  1.25 MBytes  1.05 Mbits/sec  0.085 ms  0/907 (0%)  receiver

Using "mtr" to host.tld (my Linode instance) has a noticeably lower "StDev" than before, of course other factors can effect this, but the first hop has a noticeably lower ICMP standard deviation with DOCSIS 3.1 than before with DOCSIS 3.0.

For what it is worth.

Lonnie



> On Aug 6, 2018, at 9:20 AM, Lonnie Abelbeck <li...@lo...> wrote:
> 
> Should anyone make the D3.0 to D3.1 modem upgrade it would be interesting to measure any latency change.
> 
> On the edge AstLinux box ...
> 
> Use "ip r" to find the default "via" gateway address
> 
> use "mtr -wn z.y.x.w" to the gateway address z.y.x.w for both D3.0 and D3.1 and compare.
> 
> For my COX DOCSIS 3.0
> --
> gw-lan ~ # mtr -wn z.y.x.w
> Start: 2018-08-06T09:14:52-0500
> HOST: gw-lan       Loss%   Snt   Last   Avg  Best  Wrst StDev
>  1.|-- z.y.x.w     0.0%    10    8.2   9.1   6.8  19.0   3.7
> --
> 
> Lonnie
> 
> 
> 
>> On Aug 6, 2018, at 8:38 AM, David Kerr <Da...@Ke...> wrote:
>> 
>> Thanks Lonnie.  I think I will wait-and-see.  The modems are quite expensive right now.  I am on Xfinity Blast tier which is supposed to be 250Mbps down / 10Mbps up and my speedtest shows I get ~210 down and ~12 up.  I suspect that the 210 down is probably close to practical limit on my modem, although it is supposed to go faster that is probably in perfect lab-bench conditions and not real world.  So it is possible a DOCSIS 3.1 modem might do me some good.
>> 
>> On Sun, Aug 5, 2018 at 9:01 AM, Lonnie Abelbeck <li...@lo...> wrote:
>> Doing some research and reaching out to "dslreports" forums, it does appear that a DOCSIS 3.1 modem (with 3.1 infrastructure) does offer more robust IP transport ... somewhat less latency and jitter.  DOCSIS 3.1 uses a better error correction method, Low Density Parity Check (LDPC), which is implemented in DOCSIS 3.1 modem hardware.
>> 
>> The question is whether it is worth upgrading now, or wait for your ISP to give you an incentive to upgrade to a DOCSIS 3.1 modem.
>> 
>> I'll probably look at getting a DOCSIS 3.1 modem shortly after 3.1 is available in my area, keeping the same download speed plan.
>> 
>> Lonnie
>> 
>> 
>> 
>>> On Aug 4, 2018, at 10:04 AM, David Kerr <Da...@Ke...> wrote:
>>> 
>>> That is exactly the question I am asking myself.  Is it worth it.  I can understand why the ISP would want me to upgrade (spread customer load across many more channels) but it is not at all clear why I would want to upgrade.  I agree latency and jitter is more important than raw bandwidth, I have plenty of that (download at least).
>>> 
>>> David.  
>>> 
>>> On Sat, Aug 4, 2018 at 10:56 AM, Lonnie Abelbeck <li...@lo...> wrote:
>>> 
>>>> On Aug 4, 2018, at 9:08 AM, David Kerr <Da...@Ke...> wrote:
>>>> 
>>>> Comcast/Xfinity have been bugging me to upgrade my cable modem for the past few months.  Started with emails, now getting telephone calls, stating that my current modem cannot take advantage of their latest technology upgrades.  I'm assuming they have rolled out DOCSIS 3.1 into my area.  I'm not sure I am going to do it yet but looking at the options there are basically three to chose from, Netgear, Arris and Motorola.
>>>> 
>>>> Ok, so the point of this post... the Motorola claims to support up to 2Gbps download by channel bonding two 1Gbps ethernet ports.  Of course Comcast doesn't have anything like that speed support but it does prompt the question as to whether Astlinux could channel bond two ports into EXTIF.  And of course it would need to also bond two ports for INTIF that would need to connect to a switch that supported bonding.
>>>> 
>>>> Clearly not something needed anytime soon, but I am curious.
>>>> 
>>>> Thoughts?
>>>> David
>>> 
>>> My ISP (COX) will be turning on DOCSIS 3.1 any day now in my area, I'm not interested in DOCSIS 3.1's 1 Gbps down (only 30-ish Mbps up).  Currently I have a 50/10 Mbps business connection and other than running speed tests I can't think of a time I noticed where the complete data path was limited by 50 Mbps.  Over time this will slowly change and then 100 Mbps may be useful.
>>> 
>>> Personally, the quality of the connection ... low latency, low jitter, low downtime is far more important to me than down speed greater then 100 Mbps.
>>> 
>>> So, how does a DOCSIS 3.1 modem perform (latency and jitter) on DOCSIS 3.1 infrastructure compare to a 3.0 modem on the same 3.1 infrastructure ?  I have yet to find an answer to that question.
>>> 
>>> Also beware that some DOCSIS 3.1 modems use an Intel Puma 6 chipset with a lot of problems.
>>> 
>>> Also beware COX residential (and Comcast I think) currently have a default 1 TB data tier, so at 1 Gbps 1 TB of traffic occurs in 133 minutes.
>>> 
>>> So David, I don't think ethernet bonding is worth worrying about. :-)  10 Gbps NIC's will probably be ubiquitous when we need such a thing.
>>> 
>>> But, since a DOCSIS 3.1 modem will aggregate a lot more channels than a DOCSIS 3.0 modem (on 3.1 infrastructure) would a DOCSIS 3.1 modem be a useful upgrade for a plan speed of 50-300 Mbps ?
>>> 
>>> Lonnie
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
> 
> 

[Astlinux-devel] Tarsnap Backups

[Michael K. <mic...@ip...>]

Hi Devs

I have decided that I will be changing my current backup strategy to Tarsnap for the following reasons:

  1.  I need to back up sensitive files securely
  2.  I need to be performing daily backups as I suspect a few of my customers will be doing their own basic administration soon
  3.  And it looks awesome from my preliminary tests

My current backup strategy is as follows:

  1.  Find all changed files with rsync by comparing files in /mnt/kd and /mnt/kd/.ref and using an excluded directory/file list.
  2.  Use tar to back up all the resulting files on the Astlinux server
  3.  Move the backup file to the management server via SCP

So I assume that I can just replace 2) with tarsnap instead of tar and not do 3) but the problem is that I cant do this natively using the tarsnap-backup script.
Will I need to modify/recreate it or is there another way?

PS the doco mentions that I can edit /etc/tarsnap.conf. Does that mean that /mnt/kd/tarsnap.conf will take precedence or do I need to edit this file?

Thanks

Regards
Michael Knill

Re: [Astlinux-devel] Comcast/Xfinity DOCSIS 3.1

[Lonnie A. <li...@lo...>]

Should anyone make the D3.0 to D3.1 modem upgrade it would be interesting to measure any latency change.

On the edge AstLinux box ...

Use "ip r" to find the default "via" gateway address

use "mtr -wn z.y.x.w" to the gateway address z.y.x.w for both D3.0 and D3.1 and compare.

For my COX DOCSIS 3.0
--
gw-lan ~ # mtr -wn z.y.x.w
Start: 2018-08-06T09:14:52-0500
HOST: gw-lan       Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- z.y.x.w     0.0%    10    8.2   9.1   6.8  19.0   3.7
--

Lonnie



> On Aug 6, 2018, at 8:38 AM, David Kerr <Da...@Ke...> wrote:
> 
> Thanks Lonnie.  I think I will wait-and-see.  The modems are quite expensive right now.  I am on Xfinity Blast tier which is supposed to be 250Mbps down / 10Mbps up and my speedtest shows I get ~210 down and ~12 up.  I suspect that the 210 down is probably close to practical limit on my modem, although it is supposed to go faster that is probably in perfect lab-bench conditions and not real world.  So it is possible a DOCSIS 3.1 modem might do me some good.
> 
> On Sun, Aug 5, 2018 at 9:01 AM, Lonnie Abelbeck <li...@lo...> wrote:
> Doing some research and reaching out to "dslreports" forums, it does appear that a DOCSIS 3.1 modem (with 3.1 infrastructure) does offer more robust IP transport ... somewhat less latency and jitter.  DOCSIS 3.1 uses a better error correction method, Low Density Parity Check (LDPC), which is implemented in DOCSIS 3.1 modem hardware.
> 
> The question is whether it is worth upgrading now, or wait for your ISP to give you an incentive to upgrade to a DOCSIS 3.1 modem.
> 
> I'll probably look at getting a DOCSIS 3.1 modem shortly after 3.1 is available in my area, keeping the same download speed plan.
> 
> Lonnie
> 
> 
> 
> > On Aug 4, 2018, at 10:04 AM, David Kerr <Da...@Ke...> wrote:
> > 
> > That is exactly the question I am asking myself.  Is it worth it.  I can understand why the ISP would want me to upgrade (spread customer load across many more channels) but it is not at all clear why I would want to upgrade.  I agree latency and jitter is more important than raw bandwidth, I have plenty of that (download at least).
> > 
> > David.  
> > 
> > On Sat, Aug 4, 2018 at 10:56 AM, Lonnie Abelbeck <li...@lo...> wrote:
> > 
> > > On Aug 4, 2018, at 9:08 AM, David Kerr <Da...@Ke...> wrote:
> > > 
> > > Comcast/Xfinity have been bugging me to upgrade my cable modem for the past few months.  Started with emails, now getting telephone calls, stating that my current modem cannot take advantage of their latest technology upgrades.  I'm assuming they have rolled out DOCSIS 3.1 into my area.  I'm not sure I am going to do it yet but looking at the options there are basically three to chose from, Netgear, Arris and Motorola.
> > > 
> > > Ok, so the point of this post... the Motorola claims to support up to 2Gbps download by channel bonding two 1Gbps ethernet ports.  Of course Comcast doesn't have anything like that speed support but it does prompt the question as to whether Astlinux could channel bond two ports into EXTIF.  And of course it would need to also bond two ports for INTIF that would need to connect to a switch that supported bonding.
> > > 
> > > Clearly not something needed anytime soon, but I am curious.
> > > 
> > > Thoughts?
> > > David
> > 
> > My ISP (COX) will be turning on DOCSIS 3.1 any day now in my area, I'm not interested in DOCSIS 3.1's 1 Gbps down (only 30-ish Mbps up).  Currently I have a 50/10 Mbps business connection and other than running speed tests I can't think of a time I noticed where the complete data path was limited by 50 Mbps.  Over time this will slowly change and then 100 Mbps may be useful.
> > 
> > Personally, the quality of the connection ... low latency, low jitter, low downtime is far more important to me than down speed greater then 100 Mbps.
> > 
> > So, how does a DOCSIS 3.1 modem perform (latency and jitter) on DOCSIS 3.1 infrastructure compare to a 3.0 modem on the same 3.1 infrastructure ?  I have yet to find an answer to that question.
> > 
> > Also beware that some DOCSIS 3.1 modems use an Intel Puma 6 chipset with a lot of problems.
> > 
> > Also beware COX residential (and Comcast I think) currently have a default 1 TB data tier, so at 1 Gbps 1 TB of traffic occurs in 133 minutes.
> > 
> > So David, I don't think ethernet bonding is worth worrying about. :-)  10 Gbps NIC's will probably be ubiquitous when we need such a thing.
> > 
> > But, since a DOCSIS 3.1 modem will aggregate a lot more channels than a DOCSIS 3.0 modem (on 3.1 infrastructure) would a DOCSIS 3.1 modem be a useful upgrade for a plan speed of 50-300 Mbps ?
> > 
> > Lonnie

Re: [Astlinux-devel] Comcast/Xfinity DOCSIS 3.1

[David K. <da...@ke...>]

Thanks Lonnie.  I think I will wait-and-see.  The modems are quite
expensive right now.  I am on Xfinity Blast tier which is supposed to be
250Mbps down / 10Mbps up and my speedtest shows I get ~210 down and ~12
up.  I suspect that the 210 down is probably close to practical limit on my
modem, although it is supposed to go faster that is probably in perfect
lab-bench conditions and not real world.  So it is possible a DOCSIS 3.1
modem might do me some good.

On Sun, Aug 5, 2018 at 9:01 AM, Lonnie Abelbeck <li...@lo...>
wrote:

> Doing some research and reaching out to "dslreports" forums, it does
> appear that a DOCSIS 3.1 modem (with 3.1 infrastructure) does offer more
> robust IP transport ... somewhat less latency and jitter.  DOCSIS 3.1 uses
> a better error correction method, Low Density Parity Check (LDPC), which is
> implemented in DOCSIS 3.1 modem hardware.
>
> The question is whether it is worth upgrading now, or wait for your ISP to
> give you an incentive to upgrade to a DOCSIS 3.1 modem.
>
> I'll probably look at getting a DOCSIS 3.1 modem shortly after 3.1 is
> available in my area, keeping the same download speed plan.
>
> Lonnie
>
>
>
> > On Aug 4, 2018, at 10:04 AM, David Kerr <Da...@Ke...> wrote:
> >
> > That is exactly the question I am asking myself.  Is it worth it.  I can
> understand why the ISP would want me to upgrade (spread customer load
> across many more channels) but it is not at all clear why I would want to
> upgrade.  I agree latency and jitter is more important than raw bandwidth,
> I have plenty of that (download at least).
> >
> > David.
> >
> > On Sat, Aug 4, 2018 at 10:56 AM, Lonnie Abelbeck <
> li...@lo...> wrote:
> >
> > > On Aug 4, 2018, at 9:08 AM, David Kerr <Da...@Ke...> wrote:
> > >
> > > Comcast/Xfinity have been bugging me to upgrade my cable modem for the
> past few months.  Started with emails, now getting telephone calls, stating
> that my current modem cannot take advantage of their latest technology
> upgrades.  I'm assuming they have rolled out DOCSIS 3.1 into my area.  I'm
> not sure I am going to do it yet but looking at the options there are
> basically three to chose from, Netgear, Arris and Motorola.
> > >
> > > Ok, so the point of this post... the Motorola claims to support up to
> 2Gbps download by channel bonding two 1Gbps ethernet ports.  Of course
> Comcast doesn't have anything like that speed support but it does prompt
> the question as to whether Astlinux could channel bond two ports into
> EXTIF.  And of course it would need to also bond two ports for INTIF that
> would need to connect to a switch that supported bonding.
> > >
> > > Clearly not something needed anytime soon, but I am curious.
> > >
> > > Thoughts?
> > > David
> >
> > My ISP (COX) will be turning on DOCSIS 3.1 any day now in my area, I'm
> not interested in DOCSIS 3.1's 1 Gbps down (only 30-ish Mbps up).
> Currently I have a 50/10 Mbps business connection and other than running
> speed tests I can't think of a time I noticed where the complete data path
> was limited by 50 Mbps.  Over time this will slowly change and then 100
> Mbps may be useful.
> >
> > Personally, the quality of the connection ... low latency, low jitter,
> low downtime is far more important to me than down speed greater then 100
> Mbps.
> >
> > So, how does a DOCSIS 3.1 modem perform (latency and jitter) on DOCSIS
> 3.1 infrastructure compare to a 3.0 modem on the same 3.1 infrastructure ?
> I have yet to find an answer to that question.
> >
> > Also beware that some DOCSIS 3.1 modems use an Intel Puma 6 chipset with
> a lot of problems.
> >
> > Also beware COX residential (and Comcast I think) currently have a
> default 1 TB data tier, so at 1 Gbps 1 TB of traffic occurs in 133 minutes.
> >
> > So David, I don't think ethernet bonding is worth worrying about. :-)
> 10 Gbps NIC's will probably be ubiquitous when we need such a thing.
> >
> > But, since a DOCSIS 3.1 modem will aggregate a lot more channels than a
> DOCSIS 3.0 modem (on 3.1 infrastructure) would a DOCSIS 3.1 modem be a
> useful upgrade for a plan speed of 50-300 Mbps ?
> >
> > Lonnie
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
>

Re: [Astlinux-devel] Comcast/Xfinity DOCSIS 3.1

[Lonnie A. <li...@lo...>]

Doing some research and reaching out to "dslreports" forums, it does appear that a DOCSIS 3.1 modem (with 3.1 infrastructure) does offer more robust IP transport ... somewhat less latency and jitter.  DOCSIS 3.1 uses a better error correction method, Low Density Parity Check (LDPC), which is implemented in DOCSIS 3.1 modem hardware.

The question is whether it is worth upgrading now, or wait for your ISP to give you an incentive to upgrade to a DOCSIS 3.1 modem.

I'll probably look at getting a DOCSIS 3.1 modem shortly after 3.1 is available in my area, keeping the same download speed plan.

Lonnie



> On Aug 4, 2018, at 10:04 AM, David Kerr <Da...@Ke...> wrote:
> 
> That is exactly the question I am asking myself.  Is it worth it.  I can understand why the ISP would want me to upgrade (spread customer load across many more channels) but it is not at all clear why I would want to upgrade.  I agree latency and jitter is more important than raw bandwidth, I have plenty of that (download at least).
> 
> David.  
> 
> On Sat, Aug 4, 2018 at 10:56 AM, Lonnie Abelbeck <li...@lo...> wrote:
> 
> > On Aug 4, 2018, at 9:08 AM, David Kerr <Da...@Ke...> wrote:
> > 
> > Comcast/Xfinity have been bugging me to upgrade my cable modem for the past few months.  Started with emails, now getting telephone calls, stating that my current modem cannot take advantage of their latest technology upgrades.  I'm assuming they have rolled out DOCSIS 3.1 into my area.  I'm not sure I am going to do it yet but looking at the options there are basically three to chose from, Netgear, Arris and Motorola.
> > 
> > Ok, so the point of this post... the Motorola claims to support up to 2Gbps download by channel bonding two 1Gbps ethernet ports.  Of course Comcast doesn't have anything like that speed support but it does prompt the question as to whether Astlinux could channel bond two ports into EXTIF.  And of course it would need to also bond two ports for INTIF that would need to connect to a switch that supported bonding.
> > 
> > Clearly not something needed anytime soon, but I am curious.
> > 
> > Thoughts?
> > David
> 
> My ISP (COX) will be turning on DOCSIS 3.1 any day now in my area, I'm not interested in DOCSIS 3.1's 1 Gbps down (only 30-ish Mbps up).  Currently I have a 50/10 Mbps business connection and other than running speed tests I can't think of a time I noticed where the complete data path was limited by 50 Mbps.  Over time this will slowly change and then 100 Mbps may be useful.
> 
> Personally, the quality of the connection ... low latency, low jitter, low downtime is far more important to me than down speed greater then 100 Mbps.
> 
> So, how does a DOCSIS 3.1 modem perform (latency and jitter) on DOCSIS 3.1 infrastructure compare to a 3.0 modem on the same 3.1 infrastructure ?  I have yet to find an answer to that question.
> 
> Also beware that some DOCSIS 3.1 modems use an Intel Puma 6 chipset with a lot of problems.
> 
> Also beware COX residential (and Comcast I think) currently have a default 1 TB data tier, so at 1 Gbps 1 TB of traffic occurs in 133 minutes.
> 
> So David, I don't think ethernet bonding is worth worrying about. :-)  10 Gbps NIC's will probably be ubiquitous when we need such a thing.
> 
> But, since a DOCSIS 3.1 modem will aggregate a lot more channels than a DOCSIS 3.0 modem (on 3.1 infrastructure) would a DOCSIS 3.1 modem be a useful upgrade for a plan speed of 50-300 Mbps ?
> 
> Lonnie

Re: [Astlinux-devel] Comcast/Xfinity DOCSIS 3.1

[David K. <da...@ke...>]

That is exactly the question I am asking myself.  Is it worth it.  I can
understand why the ISP would want me to upgrade (spread customer load
across many more channels) but it is not at all clear why I would want to
upgrade.  I agree latency and jitter is more important than raw bandwidth,
I have plenty of that (download at least).

David.

On Sat, Aug 4, 2018 at 10:56 AM, Lonnie Abelbeck <li...@lo...>
wrote:

>
> > On Aug 4, 2018, at 9:08 AM, David Kerr <Da...@Ke...> wrote:
> >
> > Comcast/Xfinity have been bugging me to upgrade my cable modem for the
> past few months.  Started with emails, now getting telephone calls, stating
> that my current modem cannot take advantage of their latest technology
> upgrades.  I'm assuming they have rolled out DOCSIS 3.1 into my area.  I'm
> not sure I am going to do it yet but looking at the options there are
> basically three to chose from, Netgear, Arris and Motorola.
> >
> > Ok, so the point of this post... the Motorola claims to support up to
> 2Gbps download by channel bonding two 1Gbps ethernet ports.  Of course
> Comcast doesn't have anything like that speed support but it does prompt
> the question as to whether Astlinux could channel bond two ports into
> EXTIF.  And of course it would need to also bond two ports for INTIF that
> would need to connect to a switch that supported bonding.
> >
> > Clearly not something needed anytime soon, but I am curious.
> >
> > Thoughts?
> > David
>
> My ISP (COX) will be turning on DOCSIS 3.1 any day now in my area, I'm not
> interested in DOCSIS 3.1's 1 Gbps down (only 30-ish Mbps up).  Currently I
> have a 50/10 Mbps business connection and other than running speed tests I
> can't think of a time I noticed where the complete data path was limited by
> 50 Mbps.  Over time this will slowly change and then 100 Mbps may be useful.
>
> Personally, the quality of the connection ... low latency, low jitter, low
> downtime is far more important to me than down speed greater then 100 Mbps.
>
> So, how does a DOCSIS 3.1 modem perform (latency and jitter) on DOCSIS 3.1
> infrastructure compare to a 3.0 modem on the same 3.1 infrastructure ?  I
> have yet to find an answer to that question.
>
> Also beware that some DOCSIS 3.1 modems use an Intel Puma 6 chipset with a
> lot of problems.
>
> Also beware COX residential (and Comcast I think) currently have a default
> 1 TB data tier, so at 1 Gbps 1 TB of traffic occurs in 133 minutes.
>
> So David, I don't think ethernet bonding is worth worrying about. :-)  10
> Gbps NIC's will probably be ubiquitous when we need such a thing.
>
> But, since a DOCSIS 3.1 modem will aggregate a lot more channels than a
> DOCSIS 3.0 modem (on 3.1 infrastructure) would a DOCSIS 3.1 modem be a
> useful upgrade for a plan speed of 50-300 Mbps ?
>
> Lonnie
>
>
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
>

Re: [Astlinux-devel] Comcast/Xfinity DOCSIS 3.1

[Lonnie A. <li...@lo...>]

> On Aug 4, 2018, at 9:08 AM, David Kerr <Da...@Ke...> wrote:
> 
> Comcast/Xfinity have been bugging me to upgrade my cable modem for the past few months.  Started with emails, now getting telephone calls, stating that my current modem cannot take advantage of their latest technology upgrades.  I'm assuming they have rolled out DOCSIS 3.1 into my area.  I'm not sure I am going to do it yet but looking at the options there are basically three to chose from, Netgear, Arris and Motorola.
> 
> Ok, so the point of this post... the Motorola claims to support up to 2Gbps download by channel bonding two 1Gbps ethernet ports.  Of course Comcast doesn't have anything like that speed support but it does prompt the question as to whether Astlinux could channel bond two ports into EXTIF.  And of course it would need to also bond two ports for INTIF that would need to connect to a switch that supported bonding.
> 
> Clearly not something needed anytime soon, but I am curious.
> 
> Thoughts?
> David

My ISP (COX) will be turning on DOCSIS 3.1 any day now in my area, I'm not interested in DOCSIS 3.1's 1 Gbps down (only 30-ish Mbps up).  Currently I have a 50/10 Mbps business connection and other than running speed tests I can't think of a time I noticed where the complete data path was limited by 50 Mbps.  Over time this will slowly change and then 100 Mbps may be useful.

Personally, the quality of the connection ... low latency, low jitter, low downtime is far more important to me than down speed greater then 100 Mbps.

So, how does a DOCSIS 3.1 modem perform (latency and jitter) on DOCSIS 3.1 infrastructure compare to a 3.0 modem on the same 3.1 infrastructure ?  I have yet to find an answer to that question.

Also beware that some DOCSIS 3.1 modems use an Intel Puma 6 chipset with a lot of problems.

Also beware COX residential (and Comcast I think) currently have a default 1 TB data tier, so at 1 Gbps 1 TB of traffic occurs in 133 minutes.

So David, I don't think ethernet bonding is worth worrying about. :-)  10 Gbps NIC's will probably be ubiquitous when we need such a thing.

But, since a DOCSIS 3.1 modem will aggregate a lot more channels than a DOCSIS 3.0 modem (on 3.1 infrastructure) would a DOCSIS 3.1 modem be a useful upgrade for a plan speed of 50-300 Mbps ?

Lonnie

[Astlinux-devel] Comcast/Xfinity DOCSIS 3.1

[David K. <da...@ke...>]

Comcast/Xfinity have been bugging me to upgrade my cable modem for the past
few months.  Started with emails, now getting telephone calls, stating that
my current modem cannot take advantage of their latest technology
upgrades.  I'm assuming they have rolled out DOCSIS 3.1 into my area.  I'm
not sure I am going to do it yet but looking at the options there are
basically three to chose from, Netgear, Arris and Motorola.

Ok, so the point of this post... the Motorola claims to support up to 2Gbps
download by channel bonding two 1Gbps ethernet ports.  Of course Comcast
doesn't have anything like that speed support but it does prompt the
question as to whether Astlinux could channel bond two ports into EXTIF.
And of course it would need to also bond two ports for INTIF that would
need to connect to a switch that supported bonding.

Clearly not something needed anytime soon, but I am curious.

Thoughts?
David