I have raised as feature request #3499

I have done some work on simple solution as outlined in earlier email and draft (ie not fully tested) code can be viewed Note Data Migration code for repl_limit is missing and I have not tried to amend code for new API's that ixsystems are developing

I have been thinking through more complex 'alternative solution' which creates a new app for TimeProfile which can then be used for Snapshots and replication.  Whilst this would technically be a more flexible solution I think it is overkill as existing solution probably covers 70% of business requirements and this expansion another 20%

Comments appreciated.



From: Jonathon Royle
Sent: 22 November 2013 15:56
To: Josh Paetzel
Subject: RE: [Freenas-devel] Replication bandwidth limit development proposal - for comment


Thanks for quick feedback and offer of assistance.

Is your recommendation to do my outline plan or the Alternative (more flexible/complex option)?   Either way I will progress over next few days

In current solution I have set replication traffic set with lower priority in router but this is not an easy option for user.  FYI I am pretty au fait with replication code (I am 80% through major change proposal following prior dialogue with Xin) so am aware of challenges.  One of the changes I am looking to add is the ability to limit how many replications get grabbed/queued by each iteration through the replication logic loop.  Happy to share this

Yep i will raise a ticket.



From: Josh Paetzel []
Sent: 22 November 2013 15:24
To: Jonathon Royle
Subject: Re: [Freenas-devel] Replication bandwidth limit development proposal - for comment

I like it and can help guide the needed data migrations in the database.

One thing to note:

Once a replication starts you can't change the throttling because it inlines the throttling in the shell command that does the replication. So in the current scheme if you start a large replication that overlaps it's timeslot it won't change.

It would be better to use the rate limiting in the firewall as that can be changed on the fly.

Can you open up a ticket for this?  I think we can flesh out the specs and split up the work a bit. It sounds like a great idea. 


Josh Paetzel
Director of IT, iXsystems
Servers for Open Source

On Nov 22, 2013, at 6:58 AM, Jonathon Royle <> wrote:

For our own purposes I am developing a change to replication which enable replication to have two bandwidth control parameters.

Business requirement.
  • Replication to be active 24/7 using a max bandwidth of X KBps
  • However during office hours limit replication bandwidth to Y KBps
If there is wider (ixsystems) interest then I will develop so that changeset can be merged into upstream.

Technical proposal for comment

Retain existing fields
repl_limit, repl_begin, repl_end 

Which will be used to control overall replication times and default bandwidth limits

Add new Fields, and expose in replication GUI (R/W)

timerepl_limit, timerepl_begin, timerepl_end

These will be used to control "office hour" times and replication usage. These will default to no time period eg 00:00-00:00 so by default not active. A timrepl_limit value of '0' will mean no replication, '-1' = no throttle

Business logic within will be be amended with an optimised version of

if (current time is between
repl_begin to repl_end AND repl_limit!=0 ) OR
(current time is between timerepl_begin to timerepl_end AND timerepl_limit!=0 ) then replicate

    if current time between
timerepl_begin and timerepl_end then
timerepl_limit >0 then send/receive with throttle of timerepl_limit
timerepl_limit =0 then no send/receive
send/receive with no throttle

repl_limit >0 then send/receive
with throttle of repl_limit
        if repl_limit =0 then no send/receive
        else send/receive with no throttle

ISSUE: Current value for
"no throttle" is '0' which is not that intuitive nor consistent with above proposal. In the timrepl proposal a bandwidth limit of '0' = do not replicate in "office hours"
PROPOSAL: Force a data change so that existing config of repl_limit = '0' is amended to repl_limit = '-1' Note '-1' appears to be currently accepted in DB schema but will possibly result in unhandled throttle error
QUESTION: Assuming there is an inbuilt routine for data change 9as tehre is for schema change) What is command/syntax please

NOTE: Code to be written to allow overlapping time time zones or not eg both of following supported
1) replicate 02:00 - 23:59 with no throttle except 08:00 - 18:00 when limit to 5Mbps. Primary use case
2) Replicate 00:00 - 06:00 no throttle and replicate 19:00 - 23:00 with 10Mbps limit. Alternative Use case


Allow many replication periods each with their own throttle limit (0 or -1 (default) = unlimited)
Need to create new replication/throttle table
Much more complex GUI as enter one time band and then click 'Add' to create another etc
I would need some guidance on using inbuilt tools to migrate data between tables
Overkill ?

Many thanks,


Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing
conversations that shape the rapidly evolving mobile landscape. Sign up now.
Freenas-devel mailing list