Re: [Bacula-users] Hash options ???

[Eric B. <er...@ba...>]

Hello John,

Some of the algorithm are not fully implemented, they are more like place
holders.

In our tests, the best one was XXHASH64, not sure that XXH3 was fully done
as it was less good as XXHASH64.

Maybe the Accurate fileset option is not ready for the check with XXHASH64
or XXH3, it should be available in the next major version.

Do not forget that using the option at this place will read all
files of your system at each incremental backup, modified files will be
read twice.

Best Regards,
Eric

On 4/18/25 22:48, John Lockard wrote:
>  From various places online I see that these are the options and signature
> comparison values for the various hash methods.
> 
> 1 : SHA1
> 2 : SHA256
> 3 : SHA512
> 5 : MD5
> 6 : XXHASH64
> 7 : XXH3_64
> 8 : XXH3_128
> 
> But, then diving into the code (15.0.2) I see that there are issues with
> SHA512 and that's disabled in the code, in some places and not in others.
> I also see that for XXH3_64 the code says it was never implemented, and
> again disabled in some sections of the code, but not others.
> 
> So, I'd like to use a Signature of: XXH3_128, that seems to pass the muster
> for the "Signature =" line in the Fileset.  However in the "Accurate ="
> line, the "8" is being rejected, as in "Accurate = pino8" and "Verify =
> pins8".
> 
> Am I reading the code wrong and "8" is not the value associated with
> XXH3_128?
> 
> Also, of note, that none of this is updated in the documentation.  There
> are mentions in the ChangeLog about various things being added, but in the
> appropriate section (FileSet Resource) they aren't there, or are completely
> wrong.
> 
> 
>     - XXHASH is mentioned as a Signature option in the docs, yet that
>     specific string can't be found as an option in the source.
>     - There is no mention in the docs of XXH3_64 or XXH3_128.  In the code
>     both are acceptable for the Signature, yet they fail validation.
> 
> Best I can see is that the options are not included in scan_include_options
> within src/dird/inc_conf.c.  At line 281:
>     /* Check if the options are correct */
>     switch(keyword) {
>     case INC_KW_VERIFY:
>        fs_options = "ipnugsamcd5123:V"; /* From dird/verify.c */
>        break;
>     case INC_KW_BASEJOB:
>     case INC_KW_ACCURATE:
>        fs_options = "oipnugsamMcdA5123:JC"; /* From filed/accurate.c
> accurate_check_file() */
>        break;
>     default:
>        break;
> 
> I suspect the first fs_options should be:
>          fs_options = "ipnugsamcd5123678:V"; /* From dird/verify.c */
> and, the second fs_options should be:
>          fs_options = "oipnugsamMcdA5123678:JC"; /* From filed/accurate.c
> accurate_check_file() */
> 
> 
> Though, in src/filed/accurate.c in the accurate_check_file function, at
> line no 640, in this chunk:
>     /* Go through opts once again, this time check only for checksum-related
> opts */
>     for (char *p=opts; *p; p++) {
>        switch (*p) {
>        case '5':                /* compare MD5 */
>        case '1':                /* compare SHA1 */
>        case '2':                /* compare SHA256 */
>        case '3':                /* compare SHA512 */
>        case '6':                /* compare XXHASH64 */
>        case '7':                /* compare XXH3_64 */
>        case '8':                /* compare XXH3_128 */
>           if (ff_pkt->type != FT_LNKSAVED &&
>                 (S_ISREG(ff_pkt->statp.st_mode) &&
>                  ff_pkt->flags & (FO_MD5|FO_SHA1|FO_SHA256|FO_SHA512))) {
>              checksum = true;
>           }
>           break;
>        case 'o':
>           only_changed  = true;
>           break;
>        default:
>           break;
>        }
> 
> 
> I'm thinking this line:
>                  ff_pkt->flags & (FO_MD5|FO_SHA1|FO_SHA256|FO_SHA512))) {
> Is probably supposed to read:
>                  ff_pkt->flags &
> (FO_MD5|FO_SHA1|FO_SHA256|FO_SHA512|O_XXHASH64|FO_XXH3_64|FO_XXH3_128)))
> {
> 
> 
> I'm pondering making these changes and seeing what happens.
> 
> -John
> 
> 
> 
> _______________________________________________
> Bacula-users mailing list
> Bac...@li...
> https://lists.sourceforge.net/lists/listinfo/bacula-users