Valknut / Bugs / #49 libdc0: Fail to pass test 1 of 10

Edward Sheldrake - 2010-09-29

I built an arm cross compiler, cross compiled dclib, but can't reproduce the problem running under qemu-arm. There are command line programs dclib-tth and dclib-tiger in the "extras" directory in svn, they also produced the expected output.

The TTH code comes from DC++, can you check that the arm build of linuxdcpp calculates correct TTH values?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Edward Sheldrake - 2010-09-29

labels: --> dclib
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zumbi - 2010-09-30

Hello,

I am not much familiar with this software, could you please let me know which files to use and how to perform the needed tests?

For now I have done
$ svn co https://wxdcgui.svn.sourceforge.net/svnroot/wxdcgui/trunk/dclib/
$ cd dclib/ && autoreconf --install
configure.in:1: warning: AC_INIT: not a literal: http://sourceforge.net/tracker/?group_id=181579&atid=897767
configure.in:1: warning: AC_INIT: not a literal: http://sourceforge.net/tracker/?group_id=181579&atid=897767
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `admin'.
libtoolize: copying file `admin/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
configure.in:1: warning: AC_INIT: not a literal: http://sourceforge.net/tracker/?group_id=181579&atid=897767
configure.in:1: warning: AC_INIT: not a literal: http://sourceforge.net/tracker/?group_id=181579&atid=897767
configure.in:1: warning: AC_INIT: not a literal: http://sourceforge.net/tracker/?group_id=181579&atid=897767
configure.in:1: warning: AC_INIT: not a literal: http://sourceforge.net/tracker/?group_id=181579&atid=897767
configure.in:11: installing `admin/config.guess'
configure.in:11: installing `admin/config.sub'
configure.in:13: installing `admin/install-sh'
configure.in:13: installing `admin/missing'
dclib/Makefile.am: installing `admin/depcomp'
Makefile.am: installing `./INSTALL'
$ ./configure && make && make check
[...]
PASS: threads-mutex
tth-test: incorrect tth calculated!
FAIL: tth-test
============================================================================
1 of 11 tests failed
Please report to http://sourceforge.net/tracker/?group_id=181579&atid=897767
============================================================================
make[2]: *** [check-TESTS] Error 1
make[2]: Leaving directory `/home/zumbi/dclib/tests'
make[1]: *** [check-am] Error 2
make[1]: Leaving directory `/home/zumbi/dclib/tests'
make: *** [check-recursive] Error 1

$ cd extras/
$ ./dclib-tiger
$ ./dclib-tth
$

Best regards

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Edward Sheldrake - 2010-10-01

dclib-tth and dclib-tiger take one filename argument, dclib-tth should produce the same output as tthsum:

$ ./dclib-tth dclib-tth.cpp
FSFQMTJPBIPS6D2C76YKV7E3RQGOQB5ENTCNIYY dclib-tth.cpp
$ tthsum dclib-tth.cpp
FSFQMTJPBIPS6D2C76YKV7E3RQGOQB5ENTCNIYY dclib-tth.cpp

dclib-tiger's output can be checked against http://www.cs.technion.ac.il/~biham/Reports/Tiger/test-vectors-nessie-format.dat

$ echo -n "abcdefghijklmnopqrstuvwxyz" > az.txt
$ ./dclib-tiger az.txt
az.txt: 1714A472EEE57D30040412BFCC55032A0B11602FF37BEEE9

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zumbi - 2010-10-01

Hello,

Thanks for the comments.

(sid)zumbi@abel:~/dclib/extras$ tthsum dclib-tth.cpp
GHCHQ2R6XC7YB4QE7LWYWBHEN76EYR4ZIDV2D7A dclib-tth.cpp
(sid)zumbi@abel:~/dclib/extras$ ./dclib-tth dclib-tth.cpp
GHCHQ2R6XC7YB4QE7LWYWBHEN76EYR4ZIDV2D7A dclib-tth.cpp

(sid)zumbi@abel:~/dclib/extras$ wget http://www.cs.technion.ac.il/~biham/Reports/Tiger/test-vectors-nessie-format.dat

(sid)zumbi@abel:~/dclib/extras$ echo -n "abcdefghijklmnopqrstuvwxyz" > az.txt
(sid)zumbi@abel:~/dclib/extras$ ./dclib-tiger az.txt
az.txt: 1714A472EEE57D30040412BFCC55032A0B11602FF37BEEE9

Any ideas?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Edward Sheldrake - 2010-10-01

That suggests tthsum is also broken on arm, for another check value I get:

LMHNA2VYO465P2RDOGTR2CL6XKHZNI2X4CCUY5Y az.txt

dclib-tiger working correctly suggests the bug must be in MerkleTree.h or HashValue.h, and not in TigerHash.cpp/.h.

Can you see if linuxdcpp is also affected, and if so, report it at https://launchpad.net/linuxdcpp/

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Edward Sheldrake - 2010-10-01

There is another possibility - that the base32 encoder is broken - so I have added another test in svn.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zumbi - 2010-10-04

Hi,

For az.txt I get:
(sid)zumbi@abel:~/dclib/extras$ tthsum az.txt
LMHNA2VYO465P2RDOGTR2CL6XKHZNI2X4CCUY5Y az.txt

Which checks are meaningful on the linuxdcpp package?

It builds succesfully,
See https://buildd.debian.org/fetch.cgi?pkg=linuxdcpp;ver=1.0.3-1;arch=armel;stamp=1235853338

But I am unsure it has a test suite

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zumbi - 2010-10-04

A rebuild with the new base32 check:

make[2]: Leaving directory `/home/zumbi/dclib/tests'
make check-TESTS
make[2]: Entering directory `/home/zumbi/dclib/tests'
PASS: base32
PASS: bz2-test
PASS: case-fold
PASS: ciconv-test
PASS: cstring-compare
PASS: cstring-find
PASS: cstring-number
PASS: cstring-self
PASS: cstring-upper-lower
PASS: priv-ips
high=10437930 low=-64937
count=10000000 expected=10000000
PASS: threads-mutex
tth-test: incorrect tth calculated!
FAIL: tth-test
============================================================================
1 of 12 tests failed
Please report to http://sourceforge.net/tracker/?group_id=181579&atid=897767
============================================================================
make[2]: *** [check-TESTS] Error 1
make[2]: Leaving directory `/home/zumbi/dclib/tests'
make[1]: *** [check-am] Error 2
make[1]: Leaving directory `/home/zumbi/dclib/tests'
make: *** [check-recursive] Error 1

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Edward Sheldrake - 2010-10-04

The only way to check linuxdcpp is to run it, add some files you know the TTH for to your share, then open your own filelist and compare the TTHs - here are some more:

$ md5sum dclib-0.3.23.tar.bz2 valknut-0.4.9.tar.bz2
6852e420e48d09b514fcb9f783611cdc dclib-0.3.23.tar.bz2
c25d68c447cb9deb4262befdde9fccea valknut-0.4.9.tar.bz2
$ tthsum dclib-0.3.23.tar.bz2 valknut-0.4.9.tar.bz2
BRNGRD2NBXBLCJHRE27KNGK4CDFTTB65MNZIXHI dclib-0.3.23.tar.bz2
LOA2JJCKKBA72WGRWU7GNJSAC4YXDMITWPLC52Y valknut-0.4.9.tar.bz2

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zumbi - 2010-10-05

Hello,

My results,

(sid)zumbi@abel:~$ md5sum dclib-0.3.23.tar.bz2 valknut-0.4.9.tar.bz2
6852e420e48d09b514fcb9f783611cdc dclib-0.3.23.tar.bz2
c25d68c447cb9deb4262befdde9fccea valknut-0.4.9.tar.bz2
(sid)zumbi@abel:~$ tthsum dclib-0.3.23.tar.bz2 valknut-0.4.9.tar.bz2
DWR6IOFUPA27YWXUK5XBN7IVNE5KGLYXAMFBHVY dclib-0.3.23.tar.bz2
B7LC35Y6ETJUWOVKO4EQVCP5NT2PP2UMN52EHVQ valknut-0.4.9.tar.bz2

Cheers

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Edward Sheldrake - 2010-10-10

Sorry, but I have just realised I missed something important in the initial bug report / about the tth-test program: the tth-test program had already calculated 2 correct TTH values before failing on a third. For the correct ones, the data was in memory, for the wrong one and with dclib-tth, the data is read from a file.

Although I still cannot reproduce the bug running with qemu-arm (qemu-arm doesn't show the problem with tthsum either), the dclib bug is probably in some file reading code.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Edward Sheldrake - 2010-10-10

try reading in 16KiB blocks not 1MiB

dclib-read-limit.patch

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Edward Sheldrake - 2010-10-10

Can you try the patch - if it solves the problem, dclib will need similar fixes in various other places.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zumbi - 2010-10-13

Hello,

Which patch should I test?

I test SVN HEAD (r3116) with ./configure && make && make check:
make[2]: Entering directory `/home/zumbi/dclib/tests'
PASS: base32
PASS: bz2-test
PASS: case-fold
PASS: ciconv-test
PASS: cstring-compare
PASS: cstring-find
PASS: cstring-number
PASS: cstring-self
PASS: cstring-upper-lower
PASS: priv-ips
high=10218480 low=-6707
count=10000000 expected=10000000
PASS: threads-mutex
tth-test: wrong tth EKRBBDO5OUOOZHGRYVPDMW2HHAN2GBUJNLLVQNA calculated for test file data
FAIL: tth-test
============================================================================
1 of 12 tests failed
Please report to http://sourceforge.net/tracker/?group_id=181579&atid=897767
============================================================================
make[2]: *** [check-TESTS] Error 1
make[2]: Leaving directory `/home/zumbi/dclib/tests'
make[1]: *** [check-am] Error 2
make[1]: Leaving directory `/home/zumbi/dclib/tests'
make: *** [check-recursive] Error 1

The bug can not be reproduced under ubuntu-arm either, that might explain why you can not see it under Qemu.
It might also be a bug on tthsum, which we have been discussing at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=517619

Best regards

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Edward Sheldrake - 2010-10-13

Please try r3118 - dclib may have 2 issues:
- generating the test data wrongly
- trying to read more than SSIZE_MAX of a file at once

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zumbi - 2010-10-14

make[2]: Entering directory `/home/zumbi/dclib-r3118/tests'
PASS: base32
PASS: bz2-test
PASS: case-fold
PASS: ciconv-test
PASS: cstring-compare
PASS: cstring-find
PASS: cstring-number
PASS: cstring-self
PASS: cstring-upper-lower
PASS: priv-ips
high=10438785 low=-62734
count=10000000 expected=10000000
PASS: threads-mutex
tth-test: wrong tth EKRBBDO5OUOOZHGRYVPDMW2HHAN2GBUJNLLVQNA calculated for test file data
FAIL: tth-test
============================================================================
1 of 12 tests failed
Please report to http://sourceforge.net/tracker/?group_id=181579&atid=897767
============================================================================
make[2]: *** [check-TESTS] Error 1

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

libdc0: Fail to pass test 1 of 10

Group

Searches

Help

#49 libdc0: Fail to pass test 1 of 10

Discussion