Menu
▾
▴
Re: [Exist-open] Full backup at 4 a.m.
|
From: Roy W. <gar...@ya...> - 2021-01-21 11:00:36
|
I don't know if it's relevant to your use case but I find backups are better controlled from cron. I create an xquery backup job and call it from crontab:/usr/bin/curl http://localhost:8008/exist/rest/db/backup.xqlYou could create full and incremental xquery jobs
Just a thought.
Regards,Roy
On Thursday, 21 January 2021, 10:24:17 GMT, Christian Achter <chr...@ge...> wrote:
Dear all,
thanks to your input I could solve my problem. As I also learnt something new about backups in eXist and I want to share this knowledge with you:
- For my backups that are created every 4 hours (see configuration below), I changed the value in <parameter name="max" value="6"/> from 6 to 5.
Fun fact: This caused my full backup to "wander through the day". On the first day the full backup was produced at 8 p.m., next day at 4 p.m., 3rd day at noon, 4th day at 8 a.m. and so on.
So the value="5" was less interpreted as "after 5 incremental backups create a full backup" (as described in the O'Reilly book and in the online documentation) but more like "every 5th backup will be a full backup". So every 20th hour, a full backup was created.
- So I changed back to <parameter name="max" value="6"/>. I still could not force to have the full backup at 4 a.m., at least not via the configuration in etc/conf.xml. The full backup was still created at midnight.
But I found this solution: At 3 a.m. I deleted the last full backup. So the newest backups were only around 7 or 8 or more incremental backup. Thus at 4 a.m., when the next backup was created, the system said: "Oh look, so many incremental backups - let's create a full backup now!"
Since then, my full backup is always created at 4 a.m.
- Additionally to 2. I found it interesting to see that the eXist backup functionality probably checks the backup files in the defined backup directory. I do not know if it checks the file names or the dates of creation supplied by the OS but eXist somehow knows how many incremental and full backups exist and how old they are. That is why deleting the last full backup forced eXist to create another full backup the next time backup scheduler was run.
I found that interesting, I hope you, too.
Thanks again for all the input and kind regards
Christian
Am 15.01.21 um 20:53 schrieb Christian Achter:
Hello Florian.
Thank you for getting back to me on this subject.
What you wrote totally makes sense to me. So I changed my code for the backup to:
<job type="system" name="backup"
class="org.exist.storage.ConsistencyCheckTask"
cron-trigger="0 0 0/4 * * ?">
<parameter name="output" value="/opt/server/ediarum/exist-db_5.2.0/backup"/>
<parameter name="backup" value="yes"/>
<parameter name="incremental" value="yes"/>
<parameter name="incremental-check" value="no"/>
<parameter name="max" value="5"/>
</job>
Actually, I only changed the value of parameter "max" from 6 to 5. This will create a full backup at midnight 12 a.m. (instead of 4 a.m.) but I am absolutely happy with that.
Just one thing from our experience while having this eXist server up and running for several months: The full backup was actually not "wandering" in time, by 4 hours every day; it is always created at 8 a.m. So maybe the "wandering" of the full backup only happens once, and the rule "0 0 0/4" might be more powerful than the parameter name="max" value="5". This might explain why the full backup "wandered" depending on the time that the eXist server was restarted.
I am excited to see the results tomorrow morning.
Thanks and kind regards
Christian
Am 15.01.21 um 18:32 schrieb Florian Schmitt:
Hi Christian,
in short: Based on the documentation, i don't see a way to make the
backup start on 0 (12am = midnight), with the full backup at 4am.
In your case, it seems that the combination of the increment of 4
together with the max attribute of 6 leads to strange results. Setting
max to 6 says: "Create one full backup after creating six incremental
backups". But with an increment of 4 for [0-23], starting on 0, you
get only five incremental backups per day. With max set to 6, the time
of the full backup will "roam" over the entire day:
On day 1, the backup run on 0 (12am = midnight) should be a full
backup, since the first run is always a full backup. With max set to
6, the following six (!) backups will be incremental, so the sixth
incremental backup is the first one of the following day (0 = 12am =
midnight), with the next full backup on 4am of day 2. The third full
backup again will take place after six incremental ones, thus on 8am
of day 3, and so on.
Thus, setting max to 5 should fix the problem of the different full
backup times, but you won't get a full backup at 4am with that trigger
definition; only the backup run on 0 (12am = midnight) will be a full
backup; the following five are incremental, the next one (again at 0 =
12am = midnight) is full, and so on. In that case, you may even drop
the max attribute completely; it doesn't have any special meaning in
this case. It won't have any effect since there are only five
incremental backups per day, with the first backup on next day as full
backup.
I assume that, if you want to have a full backup at 4am, you'll need
to make 4am as first backup run, so the cron expression would be "0 0
4/4 * * ?". But this trigger will create only five backups per day,
not six (no backup at 12am/midnight). Thus, i don't see a way to make
the backup trigger start on 0 (12am = midnight), creating six backups
per day, with the full backup on 4am.
Kind regards
Florian
Am 15.01.21 um 15:01 schrieb Christian Achter:
Dear all,
I have a backup scheduler running that creates 6 backups per day, 5
incremental and 1 full backup.
The code in my etc/conf.xml looks like this:
<job type="system" name="backup"
class="org.exist.storage.ConsistencyCheckTask"
cron-trigger="0 0 0/4 * * ?">
<parameter name="output"
value="/opt/server/ediarum/exist-db_5.2.0/backup"/>
<parameter name="backup" value="yes"/>
<parameter name="incremental" value="yes"/>
<parameter name="incremental-check" value="no"/>
<parameter name="max" value="6"/>
</job>
Here is my problem:
I want the full backup to be created at 4:00 a.m. I consulted the
eXist online documentation and the book and assumed that 0 0 0/4 would
be translated to: every 4 hours starting at midnight with the first
backup being a full backup.
But in reality, the full backup is created at 8:00 a.m. My experience
during several months showed that this time even differs. Sometimes,
the full backup was created at 4 p.m. oder at noon - maybe depending
in the time when the eXist-db ws started.
What am I doing wrong? How can I improve this scheduler to have a full
backup at 4 a.m. while keeping the 5 incremental backups during the day?
Thanks in advance and kind regards
Christian
_______________________________________________
Exist-open mailing list
Exi...@li...
https://lists.sourceforge.net/lists/listinfo/exist-open
_______________________________________________
Exist-open mailing list
Exi...@li...
https://lists.sourceforge.net/lists/listinfo/exist-open
_______________________________________________
Exist-open mailing list
Exi...@li...
https://lists.sourceforge.net/lists/listinfo/exist-open
_______________________________________________
Exist-open mailing list
Exi...@li...
https://lists.sourceforge.net/lists/listinfo/exist-open
|
