Menu
▾
▴
Re: [Exist-open] Full backup at 4 a.m.
|
From: Christian A. <chr...@ge...> - 2021-01-21 11:38:19
|
That's a good thought for my next eXist instance. For now I stick with "Never change a running system" ;-) Kind regards Christian Am 21.01.21 um 12:00 schrieb Roy Walter: > I don't know if it's relevant to your use case but I find backups are > better controlled from cron. I create an xquery backup job and call it > from crontab: > /usr/bin/curl http://localhost:8008/exist/rest/db/backup.xql > <http://localhost:8008/exist/rest/db/backup.xql> > You could create full and incremental xquery jobs > > Just a thought. > > Regards, > Roy > > On Thursday, 21 January 2021, 10:24:17 GMT, Christian Achter > <chr...@ge...> wrote: > > > Dear all, > > thanks to your input I could solve my problem. As I also learnt > something new about backups in eXist and I want to share this > knowledge with you: > > 1. For my backups that are created every 4 hours (see configuration > below), I changed the value in /<parameter name="max" value="6"/>/ > from 6 to 5. > *Fun fact:* This caused my full backup to "wander through the > day". On the first day the full backup was produced at 8 p.m., > next day at 4 p.m., 3rd day at noon, 4th day at 8 a.m. and so on. > So the value="5" was less interpreted as "after 5 incremental > backups create a full backup" (as described in the O'Reilly book > and in the online documentation) but more like "every 5th backup > will be a full backup". So every 20th hour, a full backup was created. > > 2. So I changed back to /<parameter name="max" value="6"/>/. I still > could not force to have the full backup at 4 a.m., at least not > via the configuration in etc/conf.xml. The full backup was still > created at midnight. > But I found this solution: At 3 a.m. I deleted the last full > backup. So the newest backups were only around 7 or 8 or more > incremental backup. Thus at 4 a.m., when the next backup was > created, the system said: "Oh look, so many incremental backups - > let's create a full backup now!" > Since then, my full backup is always created at 4 a.m. > > 3. Additionally to 2. I found it interesting to see that the eXist > backup functionality probably checks the backup files in the > defined backup directory. I do not know if it checks the file > names or the dates of creation supplied by the OS but eXist > somehow knows how many incremental and full backups exist and how > old they are. That is why deleting the last full backup forced > eXist to create another full backup the next time backup scheduler > was run. > > I found that interesting, I hope you, too. > > Thanks again for all the input and kind regards > Christian > > > Am 15.01.21 um 20:53 schrieb Christian Achter: >> Hello Florian. >> >> Thank you for getting back to me on this subject. >> >> What you wrote totally makes sense to me. So I changed my code for >> the backup to: >> >> <job type="system" name="backup" >> class="org.exist.storage.ConsistencyCheckTask" >> cron-trigger="0 0 0/4 * * ?"> >> <parameter name="output" >> value="/opt/server/ediarum/exist-db_5.2.0/backup"/> >> <parameter name="backup" value="yes"/> >> <parameter name="incremental" value="yes"/> >> <parameter name="incremental-check" value="no"/> >> <parameter name="max" value="5"/> >> </job> >> >> Actually, I only changed the value of parameter "max" from 6 to 5. >> This will create a full backup at midnight 12 a.m. (instead of 4 >> a.m.) but I am absolutely happy with that. >> >> Just one thing from our experience while having this eXist server up >> and running for several months: The full backup was actually not >> "wandering" in time, by 4 hours every day; it is always created at 8 >> a.m. So maybe the "wandering" of the full backup only happens once, >> and the rule "0 0 0/4" might be more powerful than the parameter >> name="max" value="5". This might explain why the full backup >> "wandered" depending on the time that the eXist server was restarted. >> >> I am excited to see the results tomorrow morning. >> >> Thanks and kind regards >> Christian >> >> >> Am 15.01.21 um 18:32 schrieb Florian Schmitt: >>> Hi Christian, >>> >>> in short: Based on the documentation, i don't see a way to make the >>> backup start on 0 (12am = midnight), with the full backup at 4am. >>> >>> In your case, it seems that the combination of the increment of 4 >>> together with the max attribute of 6 leads to strange results. Setting >>> max to 6 says: "Create one full backup after creating six incremental >>> backups". But with an increment of 4 for [0-23], starting on 0, you >>> get only five incremental backups per day. With max set to 6, the time >>> of the full backup will "roam" over the entire day: >>> >>> On day 1, the backup run on 0 (12am = midnight) should be a full >>> backup, since the first run is always a full backup. With max set to >>> 6, the following six (!) backups will be incremental, so the sixth >>> incremental backup is the first one of the following day (0 = 12am = >>> midnight), with the next full backup on 4am of day 2. The third full >>> backup again will take place after six incremental ones, thus on 8am >>> of day 3, and so on. >>> >>> Thus, setting max to 5 should fix the problem of the different full >>> backup times, but you won't get a full backup at 4am with that trigger >>> definition; only the backup run on 0 (12am = midnight) will be a full >>> backup; the following five are incremental, the next one (again at 0 = >>> 12am = midnight) is full, and so on. In that case, you may even drop >>> the max attribute completely; it doesn't have any special meaning in >>> this case. It won't have any effect since there are only five >>> incremental backups per day, with the first backup on next day as full >>> backup. >>> >>> >>> I assume that, if you want to have a full backup at 4am, you'll need >>> to make 4am as first backup run, so the cron expression would be "0 0 >>> 4/4 * * ?". But this trigger will create only five backups per day, >>> not six (no backup at 12am/midnight). Thus, i don't see a way to make >>> the backup trigger start on 0 (12am = midnight), creating six backups >>> per day, with the full backup on 4am. >>> >>> >>> Kind regards >>> Florian >>> >>> >>> Am 15.01.21 um 15:01 schrieb Christian Achter: >>>> Dear all, >>>> >>>> I have a backup scheduler running that creates 6 backups per day, 5 >>>> incremental and 1 full backup. >>>> >>>> The code in my etc/conf.xml looks like this: >>>> >>>> <job type="system" name="backup" >>>> class="org.exist.storage.ConsistencyCheckTask" >>>> cron-trigger="0 0 0/4 * * ?"> >>>> <parameter name="output" >>>> value="/opt/server/ediarum/exist-db_5.2.0/backup"/> >>>> <parameter name="backup" value="yes"/> >>>> <parameter name="incremental" value="yes"/> >>>> <parameter name="incremental-check" value="no"/> >>>> <parameter name="max" value="6"/> >>>> </job> >>>> >>>> Here is my problem: >>>> >>>> I want the full backup to be created at 4:00 a.m. I consulted the >>>> eXist online documentation and the book and assumed that 0 0 0/4 would >>>> be translated to: every 4 hours starting at midnight with the first >>>> backup being a full backup. >>>> >>>> But in reality, the full backup is created at 8:00 a.m. My experience >>>> during several months showed that this time even differs. Sometimes, >>>> the full backup was created at 4 p.m. oder at noon - maybe depending >>>> in the time when the eXist-db ws started. >>>> >>>> What am I doing wrong? How can I improve this scheduler to have a full >>>> backup at 4 a.m. while keeping the 5 incremental backups during the >>>> day? >>>> >>>> Thanks in advance and kind regards >>>> >>>> Christian >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Exist-open mailing list >>>> Exi...@li... >>>> <mailto:Exi...@li...> >>>> https://lists.sourceforge.net/lists/listinfo/exist-open >>>> <https://lists.sourceforge.net/lists/listinfo/exist-open> >>> >>> >>> _______________________________________________ >>> Exist-open mailing list >>> Exi...@li... >>> <mailto:Exi...@li...> >>> https://lists.sourceforge.net/lists/listinfo/exist-open >>> <https://lists.sourceforge.net/lists/listinfo/exist-open> >> >> >> _______________________________________________ >> Exist-open mailing list >> Exi...@li... >> <mailto:Exi...@li...> >> https://lists.sourceforge.net/lists/listinfo/exist-open >> <https://lists.sourceforge.net/lists/listinfo/exist-open> > _______________________________________________ > Exist-open mailing list > Exi...@li... <mailto:Exi...@li...> > https://lists.sourceforge.net/lists/listinfo/exist-open > <https://lists.sourceforge.net/lists/listinfo/exist-open> |
