I've been seeing people posting PractRand test results that don't match mine these days, so I sat down and tried to figure out why:
http://www.pcg-random.org/posts/more-practrand-passes.html
Her tests show ChaCha at 3 rounds passing PractRand out to 32 TB. In my listed results, I show chacha(3 round) as failing at 32 MB, though chacha(4 round) as passing. Here, I think the discrepency is due to the ChaCha implementation she uses silently rounding odd numbers of rounds up to the next even number, so I think she actually tested 4 rounds twice instead of 3 rounds once and 4 rounds once. This is a common behavior in ChaCha implementations, since most do two rounds at a time for optimization reasons. edit: to be clear, the issue I'm talking about can be seen on line 162 of the ChaCha implementation she links to, where it loops on the number of rounds with "for (int i = 0; i < R; i += 2)".
https://www.johndcook.com/blog/2017/08/14/testing-rngs-with-practrand/comment-page-1/
One of the commenters there said he got failures from "Bob Jenkins’ small noncryptographic PRNG" at around 1 TB IIRC. In my experience jsf32 never fails PractRand. What gives? My best guess is that he tested a different small non-cryptographic PRNG by Robert Jenkins, who has a habit of not giving his PRNGs unique names.
https://github.com/lemire/testingRNG/blob/master/practrand/results/testmitchellmoore.log
This guy tested the Mitchell-Moore PRNG, and saw failures at the first report, which is his case occurred at 256 MB. Which sounds very reasonable, as in my testing it fails any test of 2 MB or more. But he saw failures only in the binary ranks test, not the BCFN test, whereas I see clear failures on BCFN consistently with that PRNG. Drilling down a bit, it seems he is using an algorithm I haven't seen before, and calling it the Michell-Moore algorithm. Which is vaguelly plausible... I think those two people published multiple PRNGs they never bothered to name that others have informally called Mitchell-Moore. Though the first several results that show up when I google for "Mitchell Moore random number generator" all match the algorithm I used, not the one he used. Also, he told PractRand that his was a 64 bit PRNG when it was actually a 32 bit PRNG, but that wouldn't make the BCFN test pass the PRNG.
Last edit: 2017-09-06
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've been seeing people posting PractRand test results that don't match mine these days, so I sat down and tried to figure out why:
http://www.pcg-random.org/posts/more-practrand-passes.html
Her tests show ChaCha at 3 rounds passing PractRand out to 32 TB. In my listed results, I show chacha(3 round) as failing at 32 MB, though chacha(4 round) as passing. Here, I think the discrepency is due to the ChaCha implementation she uses silently rounding odd numbers of rounds up to the next even number, so I think she actually tested 4 rounds twice instead of 3 rounds once and 4 rounds once. This is a common behavior in ChaCha implementations, since most do two rounds at a time for optimization reasons. edit: to be clear, the issue I'm talking about can be seen on line 162 of the ChaCha implementation she links to, where it loops on the number of rounds with "for (int i = 0; i < R; i += 2)".
https://www.johndcook.com/blog/2017/08/14/testing-rngs-with-practrand/comment-page-1/
One of the commenters there said he got failures from "Bob Jenkins’ small noncryptographic PRNG" at around 1 TB IIRC. In my experience jsf32 never fails PractRand. What gives? My best guess is that he tested a different small non-cryptographic PRNG by Robert Jenkins, who has a habit of not giving his PRNGs unique names.
https://github.com/lemire/testingRNG/blob/master/practrand/results/testmitchellmoore.log
This guy tested the Mitchell-Moore PRNG, and saw failures at the first report, which is his case occurred at 256 MB. Which sounds very reasonable, as in my testing it fails any test of 2 MB or more. But he saw failures only in the binary ranks test, not the BCFN test, whereas I see clear failures on BCFN consistently with that PRNG. Drilling down a bit, it seems he is using an algorithm I haven't seen before, and calling it the Michell-Moore algorithm. Which is vaguelly plausible... I think those two people published multiple PRNGs they never bothered to name that others have informally called Mitchell-Moore. Though the first several results that show up when I google for "Mitchell Moore random number generator" all match the algorithm I used, not the one he used. Also, he told PractRand that his was a 64 bit PRNG when it was actually a 32 bit PRNG, but that wouldn't make the BCFN test pass the PRNG.
Last edit: 2017-09-06