You can subscribe to this list here.
2002 |
Jan
|
Feb
|
Mar
|
Apr
(75) |
May
(2) |
Jun
(17) |
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
(2) |
Dec
|
---|---|---|---|---|---|---|---|---|---|---|---|---|
2003 |
Jan
(39) |
Feb
(22) |
Mar
(40) |
Apr
(12) |
May
(79) |
Jun
(11) |
Jul
(43) |
Aug
(25) |
Sep
(7) |
Oct
(62) |
Nov
(105) |
Dec
(114) |
2004 |
Jan
(109) |
Feb
(176) |
Mar
(208) |
Apr
(148) |
May
(91) |
Jun
(114) |
Jul
(47) |
Aug
(92) |
Sep
(82) |
Oct
(33) |
Nov
(221) |
Dec
(309) |
2005 |
Jan
(129) |
Feb
(244) |
Mar
(209) |
Apr
(194) |
May
(129) |
Jun
(267) |
Jul
(250) |
Aug
(422) |
Sep
(228) |
Oct
(141) |
Nov
(369) |
Dec
(176) |
2006 |
Jan
(184) |
Feb
(342) |
Mar
(296) |
Apr
(321) |
May
(225) |
Jun
(285) |
Jul
(264) |
Aug
(354) |
Sep
(279) |
Oct
(165) |
Nov
(221) |
Dec
(311) |
2007 |
Jan
(359) |
Feb
(194) |
Mar
(251) |
Apr
(188) |
May
(328) |
Jun
(263) |
Jul
(244) |
Aug
(339) |
Sep
(517) |
Oct
(156) |
Nov
(148) |
Dec
(158) |
2008 |
Jan
(158) |
Feb
(243) |
Mar
(180) |
Apr
(57) |
May
(156) |
Jun
(117) |
Jul
(189) |
Aug
(203) |
Sep
(250) |
Oct
(304) |
Nov
(130) |
Dec
(116) |
2009 |
Jan
(153) |
Feb
(123) |
Mar
(222) |
Apr
(171) |
May
(166) |
Jun
(127) |
Jul
(133) |
Aug
(102) |
Sep
(157) |
Oct
(191) |
Nov
(190) |
Dec
(229) |
2010 |
Jan
(207) |
Feb
(164) |
Mar
(125) |
Apr
(145) |
May
(139) |
Jun
(65) |
Jul
(97) |
Aug
(132) |
Sep
(87) |
Oct
(59) |
Nov
(81) |
Dec
(50) |
2011 |
Jan
(64) |
Feb
(41) |
Mar
(59) |
Apr
(42) |
May
(20) |
Jun
(39) |
Jul
(29) |
Aug
(59) |
Sep
(21) |
Oct
(66) |
Nov
(85) |
Dec
(45) |
2012 |
Jan
(25) |
Feb
(35) |
Mar
(41) |
Apr
(10) |
May
(26) |
Jun
(28) |
Jul
(32) |
Aug
(19) |
Sep
(31) |
Oct
(9) |
Nov
(21) |
Dec
(20) |
2013 |
Jan
(16) |
Feb
(23) |
Mar
(21) |
Apr
(16) |
May
(6) |
Jun
(2) |
Jul
(16) |
Aug
(13) |
Sep
(24) |
Oct
(28) |
Nov
(13) |
Dec
(20) |
2014 |
Jan
(11) |
Feb
(10) |
Mar
(51) |
Apr
(132) |
May
(9) |
Jun
(2) |
Jul
(4) |
Aug
(13) |
Sep
(3) |
Oct
(6) |
Nov
(42) |
Dec
(4) |
2015 |
Jan
(5) |
Feb
(11) |
Mar
(15) |
Apr
(15) |
May
(27) |
Jun
(3) |
Jul
(6) |
Aug
(10) |
Sep
(34) |
Oct
(29) |
Nov
(4) |
Dec
(1) |
2016 |
Jan
(29) |
Feb
(37) |
Mar
(3) |
Apr
|
May
(8) |
Jun
(8) |
Jul
(4) |
Aug
(4) |
Sep
(18) |
Oct
(13) |
Nov
(7) |
Dec
(3) |
2017 |
Jan
|
Feb
(10) |
Mar
(17) |
Apr
(4) |
May
(6) |
Jun
(35) |
Jul
(60) |
Aug
(57) |
Sep
(10) |
Oct
(33) |
Nov
(48) |
Dec
(2) |
2018 |
Jan
(5) |
Feb
(10) |
Mar
(2) |
Apr
(23) |
May
(12) |
Jun
(16) |
Jul
(10) |
Aug
(46) |
Sep
(1) |
Oct
(34) |
Nov
(12) |
Dec
(45) |
2019 |
Jan
(29) |
Feb
(22) |
Mar
(25) |
Apr
(13) |
May
(12) |
Jun
(8) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(2) |
Dec
(30) |
2020 |
Jan
(5) |
Feb
(46) |
Mar
(19) |
Apr
(5) |
May
(10) |
Jun
(53) |
Jul
(34) |
Aug
(7) |
Sep
(22) |
Oct
(7) |
Nov
(9) |
Dec
(58) |
2021 |
Jan
(23) |
Feb
(11) |
Mar
(43) |
Apr
(7) |
May
(29) |
Jun
(10) |
Jul
|
Aug
|
Sep
|
Oct
(21) |
Nov
|
Dec
(4) |
2022 |
Jan
|
Feb
|
Mar
(37) |
Apr
|
May
(3) |
Jun
(6) |
Jul
(4) |
Aug
(14) |
Sep
(3) |
Oct
(1) |
Nov
(2) |
Dec
|
2023 |
Jan
(20) |
Feb
(2) |
Mar
(1) |
Apr
|
May
(8) |
Jun
|
Jul
|
Aug
|
Sep
(3) |
Oct
(10) |
Nov
(5) |
Dec
(2) |
2024 |
Jan
(2) |
Feb
(2) |
Mar
(1) |
Apr
|
May
(41) |
Jun
|
Jul
(3) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Iyán M. V. <me...@iy...> - 2024-07-15 15:42:04
|
Hi Eric, On 15/07/2024 16:43, er...@ba... wrote: > Hello Iyán, > >> Le 15 juil. 2024 à 10:56, Iyán Méndez Veiga via Bacula-devel <bac...@li...> a écrit : >> >> Hello, >> >> I tried to create an issue directly in your GitLab instance, but an account is required and when I tried to register I never recieved the confirmation email. >> > > Thanks for the feedback, I will double check. > > >> There is a problem in current release 15.0.2 when the system supports zstd but not lzo. >> >> compress_len and real_compress_len are defined in an ifdef HAVE_LZO block (l.1327), but later used in an ifdef HAVE_ZSTD (l. 1385). This causes the following build error: >> >> restore.c: In function ‘bool decompress_data(JCR*, int32_t, char**, u_int32_t*)’: >> restore.c:1387:13: error: ‘compress_len’ was not declared in this scope; did you mean ‘comp_len’? >> 1387 | compress_len = jcr->compress_buf_size; >> | ^~~~~~~~~~~~ >> | comp_len >> restore.c:1388:13: error: ‘cbuf’ was not declared in this scope >> 1388 | cbuf = (const unsigned char*)*data + sizeof(comp_stream_header); >> | ^~~~ >> Compiling bxattr.c >> restore.c:1389:13: error: ‘real_compress_len’ was not declared in this scope >> 1389 | real_compress_len = *length - sizeof(comp_stream_header); >> | ^~~~~~~~~~~~~~~~~ >> make[1]: *** [Makefile:187: restore.o] Error 1 >> make[1]: *** Waiting for unfinished jobs.... >> make[1]: Leaving directory '/build/bacula-client/src/bacula-15.0.2/src/filed' >> >> I attach the full build logs. >> >> Another thing I noticed is that latest version button in sourceforge.net still points to 13.x instead of 15.x. Is 15.0.2 still considered a beta? If so, it was marked as Release in GitLab tags. >> > > No, 15.0.2 is the stable release, I thought I did the change, thanks for the heads up. I have pushed a fix in the git repository. > > https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/commit/c18d2ee1b1743facc057ee9e3ee9bfbd63299a64 Great, thanks! I applied the patch and it builds without issues. Also, I can see that sourceforge.net now shows the correct version. In any case, I replace this hacky command: curl -I https://sourceforge.net/projects/bacula/files/latest/download | grep -m 1 location | cut -d "/" -f 7 With simply checking the latest GitLab tag that starts with Release- ;) > > Thanks, > > Best Regards, > Eric > >> Best regards, >> Iyán >> >> -- >> Iyán Méndez Veiga >> GPG 0x422E3694311E5AC1 >> >> <build-logs-bacula-15.0.2.txt>_______________________________________________ >> Bacula-devel mailing list >> Bac...@li... >> https://lists.sourceforge.net/lists/listinfo/bacula-devel > Best, Iyán -- Iyán Méndez Veiga GPG 0x422E3694311E5AC1 |
From: <er...@ba...> - 2024-07-15 14:43:29
|
Hello Iyán, > Le 15 juil. 2024 à 10:56, Iyán Méndez Veiga via Bacula-devel <bac...@li...> a écrit : > > Hello, > > I tried to create an issue directly in your GitLab instance, but an account is required and when I tried to register I never recieved the confirmation email. > Thanks for the feedback, I will double check. > There is a problem in current release 15.0.2 when the system supports zstd but not lzo. > > compress_len and real_compress_len are defined in an ifdef HAVE_LZO block (l.1327), but later used in an ifdef HAVE_ZSTD (l. 1385). This causes the following build error: > > restore.c: In function ‘bool decompress_data(JCR*, int32_t, char**, u_int32_t*)’: > restore.c:1387:13: error: ‘compress_len’ was not declared in this scope; did you mean ‘comp_len’? > 1387 | compress_len = jcr->compress_buf_size; > | ^~~~~~~~~~~~ > | comp_len > restore.c:1388:13: error: ‘cbuf’ was not declared in this scope > 1388 | cbuf = (const unsigned char*)*data + sizeof(comp_stream_header); > | ^~~~ > Compiling bxattr.c > restore.c:1389:13: error: ‘real_compress_len’ was not declared in this scope > 1389 | real_compress_len = *length - sizeof(comp_stream_header); > | ^~~~~~~~~~~~~~~~~ > make[1]: *** [Makefile:187: restore.o] Error 1 > make[1]: *** Waiting for unfinished jobs.... > make[1]: Leaving directory '/build/bacula-client/src/bacula-15.0.2/src/filed' > > I attach the full build logs. > > Another thing I noticed is that latest version button in sourceforge.net still points to 13.x instead of 15.x. Is 15.0.2 still considered a beta? If so, it was marked as Release in GitLab tags. > No, 15.0.2 is the stable release, I thought I did the change, thanks for the heads up. I have pushed a fix in the git repository. https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/commit/c18d2ee1b1743facc057ee9e3ee9bfbd63299a64 Thanks, Best Regards, Eric > Best regards, > Iyán > > -- > Iyán Méndez Veiga > GPG 0x422E3694311E5AC1 > > <build-logs-bacula-15.0.2.txt>_______________________________________________ > Bacula-devel mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-devel |
From: Iyán M. V. <me...@iy...> - 2024-07-15 09:09:30
|
Hello, I tried to create an issue directly in your GitLab instance, but an account is required and when I tried to register I never recieved the confirmation email. There is a problem in current release 15.0.2 when the system supports zstd but not lzo. compress_len and real_compress_len are defined in an ifdef HAVE_LZO block (l.1327), but later used in an ifdef HAVE_ZSTD (l. 1385). This causes the following build error: restore.c: In function ‘bool decompress_data(JCR*, int32_t, char**, u_int32_t*)’: restore.c:1387:13: error: ‘compress_len’ was not declared in this scope; did you mean ‘comp_len’? 1387 | compress_len = jcr->compress_buf_size; | ^~~~~~~~~~~~ | comp_len restore.c:1388:13: error: ‘cbuf’ was not declared in this scope 1388 | cbuf = (const unsigned char*)*data + sizeof(comp_stream_header); | ^~~~ Compiling bxattr.c restore.c:1389:13: error: ‘real_compress_len’ was not declared in this scope 1389 | real_compress_len = *length - sizeof(comp_stream_header); | ^~~~~~~~~~~~~~~~~ make[1]: *** [Makefile:187: restore.o] Error 1 make[1]: *** Waiting for unfinished jobs.... make[1]: Leaving directory '/build/bacula-client/src/bacula-15.0.2/src/filed' I attach the full build logs. Another thing I noticed is that latest version button in sourceforge.net still points to 13.x instead of 15.x. Is 15.0.2 still considered a beta? If so, it was marked as Release in GitLab tags. Best regards, Iyán -- Iyán Méndez Veiga GPG 0x422E3694311E5AC1 |
From: Phil S. <ph...@ca...> - 2024-05-18 13:11:12
|
Further to the .bvfs_update RunScript issue— In addition to the 'RunScript() Unknown term code' error, adding .bvfs_update as a post-job RunScript seems to have *other* consequences, in the dbcheck script: Found 20789 orphaned File records. Deleting 20789 orphaned File records. Found 1 orphaned MetaEmail records. To prune orphaned Path entries, it is necessary to clear the BVFS Cache first with the bconsole ".bvfs_clear_cache yes" command. Checking for orphaned FileSet entries. This takes some time! It would seem to me that the necessity to clear the BVFS cache before running a dbcheck largely defeats the point of running .bvfs_update as a post-job script. Can anyone speak to this point? -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Ulrich L. <ulr...@ob...> - 2024-05-18 10:50:12
|
Hello Marcin, Thanks, this explains it pretty good. In the meantime I created a new datebase using make_mysql_tables and made a quick comparison of both schemas using mysqldump -d . I found some more (hopefully) minor differences (order of columns, int(11) vs. int(10) unsigned, missing indexes). Not sure if everything is related to bacula update scripts because we run bacula quite a long time and did a lot of bacula version updates, but also operating system and mysql/mariadb upgrades. Best regards Ulrich ________________________________ Von: Marcin Haba <gan...@gm...> Gesendet: Samstag, 18. Mai 2024 10:37 An: Ulrich Leodolter <ulr...@ob...> Cc: bac...@li... <bac...@li...> Betreff: Re: [Bacula-devel] database problem update after upgrade to 15.0.2 Hello Ulrich, I know this problem. I found and reported it a year ago. You can read about it here: https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/issues/2684 Best regards, Marcin Haba (gani) On Sat, 18 May 2024 at 10:18, Ulrich Leodolter <ulr...@ob...<mailto:ulr...@ob...>> wrote: hi, after upgrading to verions 15.0.2 our nightly update stats job failed: 18-May 06:15 vlbackup.obvsg.at-dir JobId 0: Fatal error: bdb.h:146 bdb.h:146 query INSERT INTO JobHisto (JobId, Job, Name, Type, Level,ClientId, JobStatus,SchedTime, StartTime, EndTime, RealEndTime, RealStartTime, JobTDate,VolSessionId, VolSessionTime, JobFiles, JobBytes, ReadBytes,JobErrors, JobMissingFiles, PoolId, FileSetId, PriorJobId, PriorJob, PurgedFiles, HasBase, HasCache, Reviewed, Comment, FileTable, isVirtualFull,CompressRatio,Rate,LastReadStorageId,WriteStorageId,LastReadDevice,WriteDevice,StatusInfo,Encrypted)SELECT JobId, Job, Name, Type, Level, ClientId, JobStatus,SchedTime, StartTime, EndTime, RealEndTime, RealStartTime, JobTDate,VolSessionId, VolSessionTime, JobFiles, JobBytes, ReadBytes,JobErrors, JobMissingFiles, PoolId, FileSetId, PriorJobId, PriorJob, PurgedFiles, HasBase, HasCache, Reviewed, Comment, FileTable, isVirtualFull,CompressRatio,Rate,LastReadStorageId,WriteStorageId,LastReadDevice,WriteDevice,StatusInfo,Encrypted FROM Job WHERE JobStatus IN ('T','W','f ,'A','E')AND NOT EXISTS (SELECT JobHisto.JobId FROM JobHisto WHERE JobHisto.Jobid=Job.JobId)AND JobTDate < 1715746502 failed: Unknown column 'FileTable' in 'field list' after upgrading to version 15.0.2 i have updated the database using the update_mysql_tables which is installed into scripts directory. the source code of this script is bacula/src/cats/update_mysql_tables.in<http://update_mysql_tables.in>, but there are also scripts to upgrade between specific versions in bacula/updatedb. it seems these update scripts are not in sync because only bacula/src/cats/update_mysql_tables.in<http://update_mysql_tables.in> contains the FileTable update: ALTER TABLE Job ADD COLUMN FileTable CHAR(20) default 'File'; ALTER TABLE JobHisto ADD COLUMN FileTable CHAR(20) default 'File'; there is no such alter FileTable in any of the bacula/updatedb scripts. it seems it is not a good idea to use scripts from bacula/updatedb after version upgrade because of missing changes, is that correct? i have now added the columns manually und update stats job runs with problems, but i need now the check the complete schema because i am unsure which scripts i used in the past. Best regards Ulrich _______________________________________________ Bacula-devel mailing list Bac...@li...<mailto:Bac...@li...> https://lists.sourceforge.net/lists/listinfo/bacula-devel -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus |
From: Marcin H. <gan...@gm...> - 2024-05-18 10:47:40
|
On Sat, 18 May 2024 at 11:11, Ulrich Leodolter <ulr...@ob...> wrote: > Hello Marcin, > > Thanks, this explains it pretty good. > > In the meantime I created a new datebase using make_mysql_tables and made > a quick comparison of both schemas using mysqldump -d . > > I found some more (hopefully) minor differences (order of columns, int(11) > vs. int(10) unsigned, missing indexes). Not sure if everything is related > to bacula update scripts because we run bacula quite a long time and did a > lot of bacula version updates, but also operating system and mysql/mariadb > upgrades. > Hello Ulrich, Great. Thanks for letting here know about it. It is useful. Best regards, Marcin Haba (gani) -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus |
From: Marcin H. <gan...@gm...> - 2024-05-18 08:38:04
|
Hello Ulrich, I know this problem. I found and reported it a year ago. You can read about it here: https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/issues/2684 Best regards, Marcin Haba (gani) On Sat, 18 May 2024 at 10:18, Ulrich Leodolter <ulr...@ob...> wrote: > hi, > > after upgrading to verions 15.0.2 our nightly update stats job failed: > > 18-May 06:15 vlbackup.obvsg.at-dir JobId 0: Fatal error: bdb.h:146 > bdb.h:146 query INSERT INTO JobHisto (JobId, Job, Name, Type, > Level,ClientId, JobStatus,SchedTime, StartTime, EndTime, RealEndTime, > RealStartTime, JobTDate,VolSessionId, VolSessionTime, JobFiles, JobBytes, > ReadBytes,JobErrors, JobMissingFiles, PoolId, FileSetId, PriorJobId, > PriorJob, PurgedFiles, HasBase, HasCache, Reviewed, Comment, FileTable, > isVirtualFull,CompressRatio,Rate,LastReadStorageId,WriteStorageId,LastReadDevice,WriteDevice,StatusInfo,Encrypted)SELECT > JobId, Job, Name, Type, Level, ClientId, JobStatus,SchedTime, StartTime, > EndTime, RealEndTime, RealStartTime, JobTDate,VolSessionId, VolSessionTime, > JobFiles, JobBytes, ReadBytes,JobErrors, JobMissingFiles, PoolId, > FileSetId, PriorJobId, PriorJob, PurgedFiles, HasBase, HasCache, Reviewed, > Comment, FileTable, > isVirtualFull,CompressRatio,Rate,LastReadStorageId,WriteStorageId,LastReadDevice,WriteDevice,StatusInfo,Encrypted > FROM Job WHERE JobStatus IN ('T','W','f > ,'A','E')AND NOT EXISTS (SELECT JobHisto.JobId FROM JobHisto WHERE > JobHisto.Jobid=Job.JobId)AND JobTDate < 1715746502 failed: > Unknown column 'FileTable' in 'field list' > > after upgrading to version 15.0.2 i have updated the database using the > update_mysql_tables which is installed into scripts directory. > > the source code of this script is bacula/src/cats/update_mysql_tables.in, > but there are also scripts to upgrade between specific versions in > bacula/updatedb. > > it seems these update scripts are not in sync because only > bacula/src/cats/update_mysql_tables.in contains the FileTable update: > > ALTER TABLE Job ADD COLUMN FileTable CHAR(20) default 'File'; > ALTER TABLE JobHisto ADD COLUMN FileTable CHAR(20) default 'File'; > > there is no such alter FileTable in any of the bacula/updatedb scripts. > > it seems it is not a good idea to use scripts from bacula/updatedb after > version upgrade because of missing changes, is that correct? > > i have now added the columns manually und update stats job runs with > problems, but i need now the check the complete schema because i am unsure > which scripts i used in the past. > > Best regards > Ulrich > _______________________________________________ > Bacula-devel mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-devel > -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus |
From: Ulrich L. <ulr...@ob...> - 2024-05-18 08:17:29
|
hi, after upgrading to verions 15.0.2 our nightly update stats job failed: 18-May 06:15 vlbackup.obvsg.at-dir JobId 0: Fatal error: bdb.h:146 bdb.h:146 query INSERT INTO JobHisto (JobId, Job, Name, Type, Level,ClientId, JobStatus,SchedTime, StartTime, EndTime, RealEndTime, RealStartTime, JobTDate,VolSessionId, VolSessionTime, JobFiles, JobBytes, ReadBytes,JobErrors, JobMissingFiles, PoolId, FileSetId, PriorJobId, PriorJob, PurgedFiles, HasBase, HasCache, Reviewed, Comment, FileTable, isVirtualFull,CompressRatio,Rate,LastReadStorageId,WriteStorageId,LastReadDevice,WriteDevice,StatusInfo,Encrypted)SELECT JobId, Job, Name, Type, Level, ClientId, JobStatus,SchedTime, StartTime, EndTime, RealEndTime, RealStartTime, JobTDate,VolSessionId, VolSessionTime, JobFiles, JobBytes, ReadBytes,JobErrors, JobMissingFiles, PoolId, FileSetId, PriorJobId, PriorJob, PurgedFiles, HasBase, HasCache, Reviewed, Comment, FileTable, isVirtualFull,CompressRatio,Rate,LastReadStorageId,WriteStorageId,LastReadDevice,WriteDevice,StatusInfo,Encrypted FROM Job WHERE JobStatus IN ('T','W','f ,'A','E')AND NOT EXISTS (SELECT JobHisto.JobId FROM JobHisto WHERE JobHisto.Jobid=Job.JobId)AND JobTDate < 1715746502 failed: Unknown column 'FileTable' in 'field list' after upgrading to version 15.0.2 i have updated the database using the update_mysql_tables which is installed into scripts directory. the source code of this script is bacula/src/cats/update_mysql_tables.in, but there are also scripts to upgrade between specific versions in bacula/updatedb. it seems these update scripts are not in sync because only bacula/src/cats/update_mysql_tables.in contains the FileTable update: ALTER TABLE Job ADD COLUMN FileTable CHAR(20) default 'File'; ALTER TABLE JobHisto ADD COLUMN FileTable CHAR(20) default 'File'; there is no such alter FileTable in any of the bacula/updatedb scripts. it seems it is not a good idea to use scripts from bacula/updatedb after version upgrade because of missing changes, is that correct? i have now added the columns manually und update stats job runs with problems, but i need now the check the complete schema because i am unsure which scripts i used in the past. Best regards Ulrich |
From: Phil S. <ph...@ca...> - 2024-05-16 17:40:06
|
On 5/11/24 11:32, Phil Stracchino wrote: > On 5/10/24 18:31, Phil Stracchino wrote: >> On 5/10/24 14:33, Phil Stracchino wrote: >>> On 5/9/24 21:07, Marcin Haba wrote: >>>> Runscript { >>>> Runs When = After >>>> Runs On Client = no >>>> Console = ".bvfs_update jobid = %i" >>>> } > > > Yeah, this is consistent. It appears to WORK, but sends me a > 'RunScript() Unknown term code' message for each and every backup job. Does anyone have insight into this 'unknown term code' alert? It seems like something that should be fixed. -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Eric B. <er...@ba...> - 2024-05-15 06:48:34
|
Hello Phil, On 5/14/24 19:33, Phil Stracchino wrote: > On 5/14/24 09:58, Eric Bollengier via Bacula-devel wrote: >> >> Yes, it can be added to Bacula, but the patch must define the special code >> in the mysql.c file and provide a default alternative for other engines >> (using the actual code). >> >> Best Regards, >> Eric > > > OK, I'm looking at mysql.c, but it's not immediately obvious to me how to > accomplish that. > > > Also, does anyone have any insight on the 'Runscript() Unknown term code' from > the .bvfs-update call? > A Console runscript doesn't have return code, thus, it's Unknown term code. You can use a command runscript and do "sh -c 'echo .bvfs_update | bconsole'" instead if you are really annoyed. A better solution would be to implement some kind of return code for each console command, but it's a pretty big task. Best Regards, Eric |
From: Phil S. <ph...@ca...> - 2024-05-14 17:33:44
|
On 5/14/24 09:58, Eric Bollengier via Bacula-devel wrote: > > Yes, it can be added to Bacula, but the patch must define the special code > in the mysql.c file and provide a default alternative for other engines > (using the actual code). > > Best Regards, > Eric OK, I'm looking at mysql.c, but it's not immediately obvious to me how to accomplish that. Also, does anyone have any insight on the 'Runscript() Unknown term code' from the .bvfs-update call? -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Eric B. <er...@ba...> - 2024-05-14 13:58:40
|
Hello, On 5/14/24 15:49, Martin Simmons wrote: >>>>>> On Mon, 13 May 2024 21:24:47 -0400, Phil Stracchino said: >> >> + "DELETE FROM PathVisibility " >> + "WHERE NOT EXISTS " >> + "(SELECT 1 FROM Job WHERE JobId=PathVisibility.JobId) LIMIT 100000"); > > DELETE ... LIMIT is a MySQL thing, not standard. Yes, it can be added to Bacula, but the patch must define the special code in the mysql.c file and provide a default alternative for other engines (using the actual code). Best Regards, Eric |
From: Martin S. <ma...@li...> - 2024-05-14 13:50:04
|
>>>>> On Mon, 13 May 2024 21:24:47 -0400, Phil Stracchino said: > > + "DELETE FROM PathVisibility " > + "WHERE NOT EXISTS " > + "(SELECT 1 FROM Job WHERE JobId=PathVisibility.JobId) LIMIT 100000"); DELETE ... LIMIT is a MySQL thing, not standard. __Martin |
From: Phil S. <ph...@ca...> - 2024-05-14 01:24:56
|
Fixed now. This is the CORRECTED patch to bvfs.c to make .bvfs_update stay under the 128K-row transaction limit for Galera 3. --- bacula-15.0.2/src/cats/bvfs.c.orig 2024-03-22 05:48:41.000000000 -0400 +++ bacula-15.0.2/src/cats/bvfs.c 2024-05-13 18:44:03.553492388 -0400 @@ -723,10 +723,11 @@ } void bvfs_update_cache(JCR *jcr, BDB *mdb) { uint32_t nb=0; + int dc = 0; db_list_ctx jobids_list; mdb->bdb_lock(); #ifdef xxx @@ -776,17 +777,22 @@ mdb->bdb_sql_query(mdb->cmd, db_list_handler, &jobids_list); bvfs_update_path_hierarchy_cache(jcr, mdb, jobids_list.list); - mdb->bdb_start_transaction(jcr); - Dmsg0(dbglevel, "Cleaning pathvisibility\n"); - Mmsg(mdb->cmd, - "DELETE FROM PathVisibility " - "WHERE NOT EXISTS " - "(SELECT 1 FROM Job WHERE JobId=PathVisibility.JobId)"); - nb = mdb->DeleteDB(jcr, mdb->cmd); + do { + mdb->bdb_start_transaction(jcr); + Dmsg0(dbglevel, "Cleaning pathvisibility\n"); + Mmsg(mdb->cmd, + "DELETE FROM PathVisibility " + "WHERE NOT EXISTS " + "(SELECT 1 FROM Job WHERE JobId=PathVisibility.JobId) LIMIT 100000"); + dc = mdb->DeleteDB(jcr, mdb->cmd); + nb += dc; + } + while (dc == 100000); + Dmsg1(dbglevel, "Affected row(s) = %d\n", nb); mdb->bdb_end_transaction(jcr); mdb->bdb_unlock(); } -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Phil S. <ph...@ca...> - 2024-05-13 22:44:59
|
On 5/13/24 18:38, Phil Stracchino wrote: > On 5/13/24 15:25, Phil Stracchino wrote: > hmm. evidently I got something wrong in my loop there. DOH!!!! Single-character typo... -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Phil S. <ph...@ca...> - 2024-05-13 22:38:39
|
On 5/13/24 15:25, Phil Stracchino wrote: hmm. evidently I got something wrong in my loop there. -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Phil S. <ph...@ca...> - 2024-05-13 19:26:03
|
Differential backup completed, bvfs-update is running now ... The DELETE FROM PathVisibility query is VERY, VERY slow, but bulk deletes in MySQL have always been slow. There is a technique that optimizes bulk delete speeds, but it adds significant complexity. It goes basically like this: CREATE TEMPORARY TABLE T (id INT UNSIGNED PRIMARY KEY); INSERT INTO T SELECT <row IDs you want to delete> FROM <target table>; REPEAT DELETE target FROM target JOIN T on target.id = T.id ORDER BY t.id DESC LIMIT 10000; UNTIL ROW_COUNT() < 10000 END REPEAT; DROP TEMPORARY TABLE T; -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Phil S. <ph...@ca...> - 2024-05-13 18:51:51
|
On 5/13/24 14:20, Phil Stracchino wrote: > If I were to try to *fix* this compatibility issue (and it would quite > possibly perform better this way anyway), what I would do there is add a > LIMIT 100000 and then loop on it until ROW_COUNT() < 100000. BEST > PRACTICE for Galera performance is to try to keep individual writes > below 10K rows, but an occasional 100K write isn't going to have much > impact. Here's a patch that SHOULD accomplish that: --- bacula-15.0.2/src/cats/bvfs.c.orig 2024-03-22 05:48:41.000000000 -0400 +++ bacula-15.0.2/src/cats/bvfs.c 2024-05-13 14:35:08.754557022 -0400 @@ -723,10 +723,11 @@ } void bvfs_update_cache(JCR *jcr, BDB *mdb) { uint32_t nb=0; + int dc = 0; db_list_ctx jobids_list; mdb->bdb_lock(); #ifdef xxx @@ -776,17 +777,23 @@ mdb->bdb_sql_query(mdb->cmd, db_list_handler, &jobids_list); bvfs_update_path_hierarchy_cache(jcr, mdb, jobids_list.list); - mdb->bdb_start_transaction(jcr); - Dmsg0(dbglevel, "Cleaning pathvisibility\n"); - Mmsg(mdb->cmd, - "DELETE FROM PathVisibility " - "WHERE NOT EXISTS " - "(SELECT 1 FROM Job WHERE JobId=PathVisibility.JobId)"); - nb = mdb->DeleteDB(jcr, mdb->cmd); + dc = -1; + do { + mdb->bdb_start_transaction(jcr); + Dmsg0(dbglevel, "Cleaning pathvisibility\n"); + Mmsg(mdb->cmd, + "DELETE FROM PathVisibility " + "WHERE NOT EXISTS " + "(SELECT 1 FROM Job WHERE JobId=PathVisibility.JobId) LIMIT 100000"); + dc = mdb->DeleteDB(jcr, mdb->cmd); + nb += dc; + } + while (dc = 100000); + Dmsg1(dbglevel, "Affected row(s) = %d\n", nb); mdb->bdb_end_transaction(jcr); mdb->bdb_unlock(); } I'm testing this patch now. -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Phil S. <ph...@ca...> - 2024-05-13 18:20:50
|
On 5/11/24 11:32, Phil Stracchino wrote: > On 5/10/24 18:31, Phil Stracchino wrote: >> On 5/10/24 14:33, Phil Stracchino wrote: >>> On 5/9/24 21:07, Marcin Haba wrote: >>>> Runscript { >>>> Runs When = After >>>> Runs On Client = no >>>> Console = ".bvfs_update jobid = %i" >>>> } > > > Yeah, this is consistent. It appears to WORK, but sends me a > 'RunScript() Unknown term code' message for each and every backup job. mmm, new development this morning, on a Differential job: 13-May 04:30 minbar-dir JobId 0: Using Catalog "Catalog" 13-May 04:30 minbar-dir JobId 0: Error: bdb.h:147 bdb.h:147 delete DELETE FROM PathVisibility WHERE NOT EXISTS (SELECT 1 FROM Job WHERE JobId=PathVisibility.JobId) failed: wsrep_max_ws_rows exceeded So .bvfs_update is generating too many rows in a single update here for a Galera cluster. (Galera3 has a hard limit of 4GB/128K rows written in any single replication update.) If I were to try to *fix* this compatibility issue (and it would quite possibly perform better this way anyway), what I would do there is add a LIMIT 100000 and then loop on it until ROW_COUNT() < 100000. BEST PRACTICE for Galera performance is to try to keep individual writes below 10K rows, but an occasional 100K write isn't going to have much impact. -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Phil S. <ph...@ca...> - 2024-05-11 15:32:22
|
On 5/10/24 18:31, Phil Stracchino wrote: > On 5/10/24 14:33, Phil Stracchino wrote: >> On 5/9/24 21:07, Marcin Haba wrote: >>> Runscript { >>> Runs When = After >>> Runs On Client = no >>> Console = ".bvfs_update jobid = %i" >>> } Yeah, this is consistent. It appears to WORK, but sends me a 'RunScript() Unknown term code' message for each and every backup job. -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Phil S. <ph...@ca...> - 2024-05-10 22:31:38
|
On 5/10/24 14:33, Phil Stracchino wrote: > On 5/9/24 21:07, Marcin Haba wrote: >> Runscript { >> Runs When = After >> Runs On Client = no >> Console = ".bvfs_update jobid = %i" >> } > > > Marcin, FYI I just tried this and it appears to have worked. > > MySQL 127.0.0.1> SELECT table_name, table_rows, data_length FROM > information_schema.tables WHERE table_schema = 'bacula'; > +----------------+------------+-------------+ > | table_name | table_rows | data_length | > +----------------+------------+-------------+ > | BaseFiles | 0 | 16384 | > ... > | PathHierarchy | 11406 | 1589248 | > | PathVisibility | 60408 | 6832128 | > > From a small incremental job that ran in a few seconds Hmm. I missed at the time that I did get an anomalous completion message: From: Bacula <ba...@ba...> To: ro...@ca... Subject: -RunScript- ( ) Unknown term code Date: Fri, 10 May 2024 14:41:39 -0400 X-Original-To: ro...@ba... 10-May 14:30 minbar-dir JobId 0: Using Catalog "Catalog" Nevertheless aside from that, the job appears to have completed successfully. 10-May 14:30 epsilon3-file JobId 631: Volume "INCR-20240510-04:30" previously written, moving to end of data. 10-May 14:30 epsilon3-file JobId 631: Ready to append to end of Volume "INCR-20240510-04:30" size=11,055,160,776 ... 10-May 14:30 epsilon3-file JobId 631: Elapsed time=00:00:21, Transfer rate=141.9 M Bytes/second 10-May 14:30 minbar-dir JobId 631: Bacula minbar-dir 15.0.2 (21Mar24): ... Non-fatal FD errors: 0 SD Errors: 0 FD termination status: OK SD termination status: OK Termination: Backup OK -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Phil S. <ph...@ca...> - 2024-05-10 18:33:27
|
On 5/9/24 21:07, Marcin Haba wrote: > Runscript { > Runs When = After > Runs On Client = no > Console = ".bvfs_update jobid = %i" > } Marcin, FYI I just tried this and it appears to have worked. MySQL 127.0.0.1> SELECT table_name, table_rows, data_length FROM information_schema.tables WHERE table_schema = 'bacula'; +----------------+------------+-------------+ | table_name | table_rows | data_length | +----------------+------------+-------------+ | BaseFiles | 0 | 16384 | ... | PathHierarchy | 11406 | 1589248 | | PathVisibility | 60408 | 6832128 | From a small incremental job that ran in a few seconds -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |
From: Martin S. <ma...@li...> - 2024-05-10 18:29:43
|
>>>>> On Fri, 10 May 2024 14:00:42 -0400, Phil Stracchino said: > > On 5/10/24 13:39, Martin Simmons wrote: > >> Column declared NOT NULL without DEFAULT (SQL99 violation) > > > > Where does SQL99 say that this is a violation for CREATE TABLE? You mentioned > > AUTO_INCREMENT previously, but that is MySQL-specific. > > > I can't point you at the specific line, but I do know that the reason > reason cited by Oracle for why MySQL 8 started flagging this as an error > is *because* it is a SQL99 violation. OK, the violation is probably when you come to use INSERT rather than the CREATE TABLE statement itself. __Martin |
From: Martin S. <ma...@li...> - 2024-05-10 18:22:20
|
>>>>> On Fri, 10 May 2024 14:04:50 -0400, Phil Stracchino said: > > On 5/10/24 14:01, Martin Simmons wrote: > > But I think you are mixing two different concepts: in CREATE TABLE, a DEFAULT > > just specifies what INSERT does if the column is omitted; NOT NULL is a > > constraint that applies all the time. It is quite reasonable to specify NOT > > NULL for a column that must always be given a value in an INSERT statement. > > > Oh yes, absolutely. That's its *purpose*. "Column is not allowed to be > NULL." Where thew problem can arise is if you don't supply a value for > a column that you've told the DB is not allowed to be NULL, but you > haven't given it an explicit DEFAULT to use if you don't supply a value, > and you're in STRICT mode so the DB won't fill in an *implicit* default. > > What happens then is that the DB throws an error on the query — which is > probably what you *want* to happen. You describe that as a problem, but would call it a solution -- it tells you that you have a bug in the program doing the INSERT. File.FileIndex is a good example of this in fact, because each row is supposed to have a unique value within each job, so any fixed default will be wrong. __Martin |
From: Phil S. <ph...@ca...> - 2024-05-10 18:04:58
|
On 5/10/24 14:01, Martin Simmons wrote: > But I think you are mixing two different concepts: in CREATE TABLE, a DEFAULT > just specifies what INSERT does if the column is omitted; NOT NULL is a > constraint that applies all the time. It is quite reasonable to specify NOT > NULL for a column that must always be given a value in an INSERT statement. Oh yes, absolutely. That's its *purpose*. "Column is not allowed to be NULL." Where thew problem can arise is if you don't supply a value for a column that you've told the DB is not allowed to be NULL, but you haven't given it an explicit DEFAULT to use if you don't supply a value, and you're in STRICT mode so the DB won't fill in an *implicit* default. What happens then is that the DB throws an error on the query — which is probably what you *want* to happen. -- Phil Stracchino Fenian House Publishing ph...@ca... ph...@co... Landline: +1.603.293.8485 Mobile: +1.603.998.6958 |