rotate

letzter3
2011-11-26
2013-05-29
  • letzter3

    letzter3 - 2011-11-26

    Hello,

    using automysqlbackup on my debain-server. Should be version 2.5-6
    http://packages.debian.org/squeeze/automysqlbackup

    The Problem is, that daily backups are not rotate. The log says:

    AutoMySQLBackup VER 2.5
    http://sourceforge.net/projects/automysqlbackup/
    Backup of Database Server - ptLWL01
    ======================================================================
    Backup Start Time Do 24. Nov 01:00:02 CET 2011
    ======================================================================
    Daily Backup of Database ( ptlwl )
    Rotating last weeks Backup...
    No compression option set, check advanced settings
    ----------------------------------------------------------------------
    Backup End Do 24. Nov 01:00:19 CET 2011
    ======================================================================
    Total disk space used for backup storage..
    Size - Location
    1,9G /home/backup
    

    But the old backups are still there.
    I think, it's not the known old bug (with the gzip-compression).
    Here is the partly code from /usr/sbin/automysqlbackup

            # Weekly Backup
            if [ "$DNOW" = "$DOWEEKLY" ]; then
                    echo Weekly Backup of Database \( $DB \)
                    echo Rotating 5 weeks Backups...
                            if [ "$W" -le 05 ];then
                                    REMW=`expr 48 + $W`
                            elif [ "$W" -lt 15 ];then
                                    REMW=0`expr $W - 5`
                            else
                                    REMW=`expr $W - 5`
                            fi
                    eval rm -fv "$BACKUPDIR/weekly/$DB/${DB}_week.$REMW.*" 
                    echo
                            dbdump "$DB" "$BACKUPDIR/weekly/$DB/${DB}_week.$W.$DATE.sql"
                            compression "$BACKUPDIR/weekly/$DB/${DB}_week.$W.$DATE.sql"
                            BACKUPFILES="$BACKUPFILES $BACKUPDIR/weekly/$DB/${DB}_week.$W.$DATE.sql$SUFFIX"
                    echo ----------------------------------------------------------------------
            # Daily Backup
            else
                    echo Daily Backup of Database \( $DB \)
                    echo Rotating last weeks Backup...
                    eval rm -fv "$BACKUPDIR/daily/$DB/*.$DOW.sql.*" 
                    echo
                            dbdump "$DB" "$BACKUPDIR/daily/$DB/${DB}_$DATE.$DOW.sql"
                            compression "$BACKUPDIR/daily/$DB/${DB}_$DATE.$DOW.sql"
                            BACKUPFILES="$BACKUPFILES $BACKUPDIR/daily/$DB/${DB}_$DATE.$DOW.sql$SUFFIX"
                    echo ----------------------------------------------------------------------
            fi
            done
    echo Backup End `date`
    echo ======================================================================
    

    I've configurated in /etc/defaul/mysqlbackup.
    Using no compression. The post command compress and encode and transfer to external backup-space.
    Have no idea.

    Michael

     
  • PittaGurneyi

    PittaGurneyi - 2011-11-28

    Hi, I suggest you try version 3, since it was created, because the programming of 2.x was flawed. I won't help with 2.x errors, since the whole ordeal of rewriting the script nearly from scratch was the solution to all these problems. I hope you try the new version and can understand my point of view.

     
  • letzter3

    letzter3 - 2011-11-28

    I understand.
    Debian use v.2.5-6 as default in the Debian-Repositority and I can use the Debain-way for install and upgrade.
    Okay, will switch to 3.
    Thanks for your replay.

    Michael

     
  • letzter3

    letzter3 - 2012-01-07

    Okay, switched to version 3, but it seems, that the daily rotate is not working.
    Here is my config:
    (inactive removed for smaller size)

    root@ptLWL01:/etc/automysqlbackup# cat myserver.conf                                                                                                                                    
    #version=3.0_rc2                                                                                                                                                                        
    # Username to access the MySQL server e.g. dbuser
    CONFIG_mysql_dump_username='root'
    # Password to access the MySQL server e.g. password
    CONFIG_mysql_dump_password='halla4'
    # Host name (or IP address) of MySQL server e.g localhost
    CONFIG_mysql_dump_host='localhost'
    # "Friendly" host name of MySQL server to be used in email log
    # if unset or empty (default) will use CONFIG_mysql_dump_host instead
    CONFIG_mysql_dump_host_friendly='ptlwl01-mysql-Server'
    # Backup directory location e.g /backups
    CONFIG_backup_dir='/home/backup'
    # This is practically a moot point, since there is a fallback to the compression
    # functions without multicore support in the case that the multicore versions aren't
    # present in the system. Of course, if you have the latter installed, but don't want
    # to use them, just choose no here.
    # pigz -> gzip
    # pbzip2 -> bzip2
    CONFIG_multicore='no'
    # Databases to backup
    # List of databases for Daily/Weekly Backup e.g. ( 'DB1' 'DB2' 'DB3' ... )
    # set to (), i.e. empty, if you want to backup all databases
    CONFIG_db_names=('ptlwl')
    # You can use
    #declare -a MDBNAMES=( "${DBNAMES[@]}" 'added entry1' 'added entry2' ... )
    # INSTEAD to copy the contents of $DBNAMES and add further entries (optional).
    # List of databases for Monthly Backups.
    # set to (), i.e. empty, if you want to backup all databases
    CONFIG_db_month_names=('ptlwl')
    # List of DBNAMES to EXLUCDE if DBNAMES is empty, i.e. ().
    #CONFIG_db_exclude=( 'information_schema' )
    # Advanced Settings
    # Rotation Settings
    # Which day do you want monthly backups? (01 to 31)
    # If the chosen day is greater than the last day of the month, it will be done
    # on the last day of the month.
    # Set to 0 to disable monthly backups.
    CONFIG_do_monthly="31"
    # Which day do you want weekly backups? (1 to 7 where 1 is Monday)
    # Set to 0 to disable weekly backups.
    CONFIG_do_weekly="7"
    # Set rotation of daily backups. VALUE*24hours
    # If you want to keep only today's backups, you could choose 1, i.e. everything older than 24hours will be removed.
    CONFIG_rotation_daily=6
    # Set rotation for weekly backups. VALUE*24hours
    CONFIG_rotation_weekly=35
    # Set rotation for monthly backups. VALUE*24hours
    CONFIG_rotation_monthly=2
    # Server Connection Settings
    # Set the port for the mysql connection
    CONFIG_mysql_dump_port=3306
    # Backup dump settings
    # Include CREATE DATABASE in backup?
    CONFIG_mysql_dump_create_database='yes'
    # Separate backup directory and file for each DB? (yes or no)
    CONFIG_mysql_dump_use_separate_dirs='yes'
    # Choose Compression type. (gzip or bzip2)
    CONFIG_mysql_dump_compression='bzip2'
    # Store an additional copy of the latest backup to a standard
    # location so it can be downloaded by third party scripts.
    #CONFIG_mysql_dump_latest='no'
    # Remove all date and time information from the filenames in the latest folder.
    # Runs, if activated, once after the backups are completed. Practically it just finds all files in the latest folder
    # and removes the date and time information from the filenames (if present).
    #CONFIG_mysql_dump_latest_clean_filenames='no'
    # Notification setup
    # What would you like to be mailed to you?
    # - log   : send only log file
    # - files : send log file and sql files as attachments (see docs)
    # - stdout : will simply output the log to the screen if run manually.
    # - quiet : Only send logs if an error occurs to the MAILADDR.
    CONFIG_mailcontent='log'
    # Set the maximum allowed email size in k. (4000 = approx 5MB email [see docs])
    #CONFIG_mail_maxattsize=4000
    # Allow packing of files with tar and splitting it in pieces of CONFIG_mail_maxattsize.
    #CONFIG_mail_splitandtar='yes'
    # Use uuencode instead of mutt. WARNING: Not all email clients work well with uuencoded attachments.
    #CONFIG_mail_use_uuencoded_attachments='no'
    # Email Address to send mail to? (user@domain.com)
    CONFIG_mail_address='root'
    # Encryption
    # Do you wish to encrypt your backups using openssl?
    CONFIG_encrypt='yes'
    # Choose a password to encrypt the backups.
    CONFIG_encrypt_password='PASSWORD'
    # Other
    # dry-run, i.e. show what you are gonna do without actually doing it
    # inactive: =0 or commented out
    # active: uncommented AND =1
    CONFIG_dryrun=1
    

    If I see

    # Set rotation of daily backups. VALUE*24hours
    # If you want to keep only today's backups, you could choose 1, i.e. everything older than 24hours will be removed.
    CONFIG_rotation_daily=6
    

    I suggest, I will not have a databasedump older than 6 days.

    But there are databasedumps older 6 days!

    root@ptLWL01:/home/backup/daily/ptlwl# tree /home/backup/daily/ptlwl
    /home/backup/daily/ptlwl
    ├── daily_ptlwl_2011-12-31_00h49m_Saturday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-01_00h51m_Sunday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-02_00h40m_Monday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-03_00h52m_Tuesday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-04_00h35m_Wednesday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-05_00h41m_Thursday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-06_00h30m_Friday.sql.bz2.enc
    └── daily_ptlwl_2012-01-07_00h52m_Saturday.sql.bz2.enc
    0 directories, 8 files
    root@ptLWL01:/home/backup/daily/ptlwl#
    

    The logfile says:

    checking directory "/home/backup/daily" ... exists.
    checking directory "/home/backup/weekly" ... exists.
    checking directory "/home/backup/monthly" ... exists.
    checking directory "/home/backup/latest" ... exists.
    checking directory "/home/backup/tmp" ... exists.
    checking directory "/home/backup/fullschema" ... exists.
    checking directory "/home/backup/status" ... exists.
    # Testing for installed programs
    mysql ... found.
    mysqldump ... found.
    bzip2 ... found.
    mail ... found.
    # Parsing databases ... done.
    ======================================================================
    AutoMySQLBackup version 3.0
    http://sourceforge.net/projects/automysqlbackup/
    Backup of Database Server - ptlwl01-mysql-Server
    Databases - ptlwl
    Databases (monthly) - ptlwl
    ======================================================================
    ======================================================================
    Dump full schema.
    dry-running: mysqldump --user=root --password=PASSWORD --host=localhost --all-databases --routines --no-data --ssl --port=3306 | bzip2_compression > /home/backup/fullschema/fullschema_daily_2012-01-07_16h08m_Saturday.sql.bz2
    Rotating 0 month backups for
    dry-running: openssl enc -aes-256-cbc -e -in /home/backup/fullschema/fullschema_daily_2012-01-07_16h08m_Saturday.sql.bz2 -out /home/backup/fullschema/fullschema_daily_2012-01-07_16h08m_Saturday.sql.bz2.enc -pass pass:PASSWORD
    ======================================================================
    ======================================================================
    Dump status.
    dry-running: mysqlshow --user=root --password=PASSWORD --host=localhost --status --ssl --port=3306 | bzip2_compression > /home/backup/status/status_daily_2012-01-07_16h08m_Saturday.txt.bz2
    Rotating 0 month backups for
    dry-running: openssl enc -aes-256-cbc -e -in /home/backup/status/status_daily_2012-01-07_16h08m_Saturday.txt.bz2 -out /home/backup/status/status_daily_2012-01-07_16h08m_Saturday.txt.bz2.enc -pass pass:PASSWORD
    ======================================================================
    Backup Start Time Sat Jan  7 16:08:03 CET 2012
    ======================================================================
    Daily Backup ...
    Daily Backup of Database ( ptlwl )
    dry-running: mysqldump --user=root --password=PASSWORD --host=localhost --quote-names --opt --ssl --port=3306 --databases ptlwl | bzip2_compression > /home/backup/daily/ptlwl/daily_ptlwl_2012-01-07_16h08m_Saturday.sql.bz2
    ----------------------------------------------------------------------
    Backup End Time Sat Jan  7 16:08:03 CET 2012
    ======================================================================
    Total disk space used for backup storage...
    Size - Location
    2.7G /home/backup
    ======================================================================
    

    I have no idea, where is my / the error.

     
  • PittaGurneyi

    PittaGurneyi - 2012-01-08

    First of all, why are you removing monthly backups that are older than 2 days ?! If there is a sane reason, why some people are doing this, i have to make a quick adjustment to the script.

    Can you run a find command on your daily folder with
    find your_daily_folder_path -mtime +6 -printf '%f %T+\n'
    This should list all files older than 6 days with their last modification timestamp - that is what we are comparing against.

     
  • letzter3

    letzter3 - 2012-01-08

    First of all, why are you removing monthly backups that are older than 2 days ?! If there is a sane reason, why some people are doing this, i have to make a quick adjustment to the script.

    Ohh, is my error. There is no reason to do this. Will change this to 120 (4 months)

    But that would mean, that my monthly backup should rotate. It doesnt. There is a monthly backup

    root@ptLWL01:/home/backup/daily/ptlwl# find /home/backup/monthly/ptlwl -mtime +2 -printf '%f %T+\n'
    ptlwl 2011-12-31+00:58:10.0639264310
    monthly_ptlwl_2011-12-31_00h49m_December.sql.bz2.enc 2011-12-31+00:58:10.0639264310
    

    Here is the output from your command.

    root@ptLWL01:/home/backup/daily/ptlwl# find /home/backup/daily/ptlwl -mtime +6 -printf '%f %T+\n'
    daily_ptlwl_2012-01-01_00h51m_Sunday.sql.bz2.enc 2012-01-01+00:59:46.7479260560
    root@ptLWL01:/home/backup/daily/ptlwl#
    

    I'm a little bit confused. If I make a tree on my daily-backup-directory, I see one removed file. Take a look to my tree-command above, the daily_ptlwl_2011-12-31_00h49m_Saturday.sql.bz2.enc is done.

    root@ptLWL01:/home/backup/daily/ptlwl# tree /home/backup/daily/ptlwl
    /home/backup/daily/ptlwl
    ├── daily_ptlwl_2012-01-01_00h51m_Sunday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-02_00h40m_Monday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-03_00h52m_Tuesday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-04_00h35m_Wednesday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-05_00h41m_Thursday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-06_00h30m_Friday.sql.bz2.enc
    ├── daily_ptlwl_2012-01-07_00h52m_Saturday.sql.bz2.enc
    └── daily_ptlwl_2012-01-08_00h33m_Sunday.sql.bz2.enc
    0 directories, 8 files
    

    Michael

     
  • PittaGurneyi

    PittaGurneyi - 2012-01-08

    I am currently going over the script again, fixing a few issues and structuring it. There have approximately 2000 lines been added in the script since I started reworking the 2.5 version and not even half of that - the features these lines represent - were on my list when I started. Therefore it is time to get things straightened out, so that we can hopefully get to 3.0 final soon.
    From what I have seen so far, there is no reason, why the rotation should not work. What you have to remember is, that the mtime +6 option selects all files with a modification datetime stamp OLDER than 6*24 hours. Therefore, depending on when you run the script, you could probably get 8 kept backups. Perhaps you can take a look at man printf as well.

    Please test if backups are removed if you lower the rotation daily number.

    Have a nice day.

     
  • letzter3

    letzter3 - 2012-01-09

    datetime stamp OLDER than 6*24 hours. Therefore, depending on when you run the script, you could probably get 8 kept backups.

    Think, you are right and I think, rotation works fine, because today I cant see the backup from 1. Jan. 2012.

    Maybe the log could be more / exactly says, what happened. Maybe there is a missunderstanding by myside, because my weekly backup runs yesterday night….

    Thanks a lot for your supprt!

    Michael

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks