First of all let me presents my contratulations for the very nice and usefull project.
Here is my question: I am dealing with a legacy PRNG, the one described in the paper "George Marsaglia, Arif Zaman, Wai Wan Tsang - Toward a universal random number generator,
Statistics & Probability Letters, Volume 9, Issue 1, 1990, Pages 35-39, https://doi.org/10.1016/0167-7152(90)90092-L" , which generates uniform doubles in the range [0. , 1. ]. Is it possible to test it using practrand ? If yes, how ?
Thank you very much!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think it is possible to test it, but you have to perform a few passages to produce a stream of random bits (which is what PractRand wants) from the mantissa of a 32-bit float. Perhaps you can create a 96-bit buffer in which you can put 4 24-bit mantissas; then you extract from the buffer 3 32-bit unsigned integers (there may be better ways of doing this):
BTW, I found this implementation of the generator: http://paulbourke.net/miscellaneous/random/randomlib.c
In any case, even if it passes all tests (if!), I think such a generator would be of historical interest only. It has a very large 3264-bit state but a period of 'only' 2^144. Its speed should be checked but I don't expect it to be as fast as today's fast generators. In any case, please do try it and report your results here!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The version I am tested is part of a major, and quite legacy, simulation tool and was originaly coded in FORTRAN at CERNLIB. It produces doubles as output. The generator passes TestU01, but the amount of tested data there is relatively small, as you know.
Basically, for each generated double x I do :
uint64_ty;uint32_tz;memcpy(&y, &x, sizeof(double));z=(y>>20); //for the 32 higher random bits of the mantissa//orz=y; // for the 32 lower random bits of the mantissa
then I stream out to stdin and practrand/tools/run_test .
It is failing ugly. ...
I wonder if am not doing something wrong, given the generator passes testU01's bigcrush.
Last edit: A. Augusto Alves Junior 2020-10-27
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Let me guess: Pythia?
If it passes bigcrush it shouldn't, I don't think, fail PractRand very quickly. Could you perhaps post the code (in Fortran and/or C++) used for the generator?
The original generator by Marsaglia used, if I'm not mistaken, 32bit floating point numbers, which have a 24 bit mantissa. If your program generates doubles (64bit floats) then perhaps you're assuming its whole mantissa is random and not only 24 bits of it? If you can post the code (Fortran and/or C++) used for the generator.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
-s: sequence number: the generator is generating 3 sequences in chunks
of 1024 numbers.
-m: test mode. 0 [ naive conversion of double to uint64_t ] ; 1 [ test
the higher 32 bits of the 52bit mantissa ] and 2 [ test the lower 32
bits of the 52bit mantissa ]
Please, let me know if you can't manage to access and run it.
Look forward for listening your impressions.
A.A.
On 28/10/2020 01:06, Lorenzo wrote:
Let me guess: Pythia?
If it passes bigcrush it shouldn't, I don't think, fail PractRand very
quickly. Could you perhaps post the code (in Fortran and/or C++) used
for the generator?
The original generator by Marsaglia used, if I'm not mistaken, 32bit
floating point numbers, which have a 24 bit mantissa. If your program
generates doubles (64bit floats) then perhaps you're assuming its
whole mantissa is random and not only 24 bits of it? If you can post
the code (Fortran and/or C++) used for the generator.
-s: sequence number: the generator is generating 3 sequences in chunks of 1024 numbers.
-m: test mode. 0 [ naive conversion of double to uint64_t ] ; 1 [ test the higher 32 bits of the 52bit mantissa ] and 2 [ test the lower 32 bits of the 52bit mantissa ]
Please, let me know if you can't manage to access and run it.
Look forward for listening your impressions.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello, I had a look, but please note that I'm no expect in such things and I may be doing something trivially wrong.
First, I used prng_streamer with options -m from 0 to 2, and the otput stream I got is clearly non-random (it's sufficient to open, say, the first megabyte as a raw greyscale image to see lots of patterns in the data). Consequently, PractRand fails instantly for these streams. I haven't checked your source code, but it seems that you're extracting the random bits incorrectly.
I wrote a rough-and-ready Fortran program (quoted at the end of this message) to do some independent tests.
The program extracts one byte at a time from the double precision number returned by the routine with the RNG, with the first byte corresponding to the least-significant bits.
I output to disk and then pass the file to PractRand (output to screen for some reason was very slow on my system; for small tests this is okay). Results are:
byte 1 : fails at 1K
byte 2: fails at 16K
byte 3: fails at 2GB
byte 4: fails at 16GB
byte 5: fails at 32GB
byte 6: doesn't fail up to and including 64GB
So it would seem that the low bits are very non-random and then progressively increase in quality, but I'd bet that byte 6 will also fail soon, perhaps at 256GB.
There should be 4 more random bits in byte 7, in principle the best, but I wasn't succesful in extracting and testing them so far.
So, unless I'm doing something wrong, it seems that this generator kind-of sucks and that only the very highest bits have somewhat good randomness; this is perhaps not surprising for a generator from 1990. Also, in the paper the generator producesonly random 24bits, and I'm not sure the version you have got, returning 64bit floating point numbers, has extended and tested the algorithm for this case.
I'm also not 100% that it's correct to test the bits of the mantissa of a floating point number in this way.
By the way, I've open a thread on this topic on the newsgroup comp.lang.fortran (thread 'how to convert mantissa of double precision number to bit stream'); you should have a look there, many of the people there are seasoned programmers which are much more competent than I am in such things.
In any case my preliminary conclusion is that the generator is, by today's standard, poor.
Also, it's fairly slow, has a huge state of ~64000 bits (but a period of only 2^144 if I recall correctly) and a very complicated initialization procedure.
Thank you very much Lorenzo. Your results just confirmed my suspicions. Ï will update my testing wrapper to perform the byte extraction in the way you did, and document well the whole thing, to recomend the core developers the adoption of a more reliable PRNG.
Thank you very much.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've done some hacks to your c++ code. I think the approach is slightly different to Lorenzo's and possibly a more useful way to convert double to 32 bit unsigned int. Unfortunately it is untested as i don't yet have a fortran compiler. Notes follow in the next comment. Ignore if you're happy with what you've already got.
And here's the notes. I'll add that i suspect the prng is not all that good. Whether it's bad enough to invalidate the actual application is hard to know. Often even quite bad prngs can be useful, but not always. Hopefully in the rewrite of the application, there will be some thought to being able to replace the prng with minimal pain, so at least two different prngs can be tested to see how consistent results are. Best of luck.
Thank you very much G. Jones, I will look into it right now and update the testing code. I will post back here the results. About the impact of the PRNG quality in the results, it is hard to say for sure, agree. Also, it would be harder to find out than just update the PRNG for a reliable one. It is plenty of choice around, with very good performances. Let's see...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi Guys,
First of all let me presents my contratulations for the very nice and usefull project.
Here is my question: I am dealing with a legacy PRNG, the one described in the paper
"George Marsaglia, Arif Zaman, Wai Wan Tsang - Toward a universal random number generator,
Statistics & Probability Letters, Volume 9, Issue 1, 1990, Pages 35-39, https://doi.org/10.1016/0167-7152(90)90092-L" , which generates uniform doubles in the range [0. , 1. ]. Is it possible to test it using practrand ? If yes, how ?
Thank you very much!
I think it is possible to test it, but you have to perform a few passages to produce a stream of random bits (which is what PractRand wants) from the mantissa of a 32-bit float. Perhaps you can create a 96-bit buffer in which you can put 4 24-bit mantissas; then you extract from the buffer 3 32-bit unsigned integers (there may be better ways of doing this):
BTW, I found this implementation of the generator: http://paulbourke.net/miscellaneous/random/randomlib.c
In any case, even if it passes all tests (if!), I think such a generator would be of historical interest only. It has a very large 3264-bit state but a period of 'only' 2^144. Its speed should be checked but I don't expect it to be as fast as today's fast generators. In any case, please do try it and report your results here!
The version I am tested is part of a major, and quite legacy, simulation tool and was originaly coded in FORTRAN at CERNLIB. It produces doubles as output. The generator passes TestU01, but the amount of tested data there is relatively small, as you know.
Basically, for each generated double x I do :
then I stream out to stdin and practrand/tools/run_test .
It is failing ugly. ...
I wonder if am not doing something wrong, given the generator passes testU01's bigcrush.
Last edit: A. Augusto Alves Junior 2020-10-27
Let me guess: Pythia?
If it passes bigcrush it shouldn't, I don't think, fail PractRand very quickly. Could you perhaps post the code (in Fortran and/or C++) used for the generator?
The original generator by Marsaglia used, if I'm not mistaken, 32bit floating point numbers, which have a 24 bit mantissa. If your program generates doubles (64bit floats) then perhaps you're assuming its whole mantissa is random and not only 24 bits of it? If you can post the code (Fortran and/or C++) used for the generator.
Hi Lorenzo, thank you very much. Can you access the repository below?
https://gitlab.ikp.kit.edu/AAAlvesJr/C7PRNG
It is a cmake project. Building:
You can use prng_streamer to run PractRand.
Usage:
/prng_streamer -s{0,1,2} -m{0,1,2}| /opt/practrand/RNG_test stdin
Options:
-s: sequence number: the generator is generating 3 sequences in chunks
of 1024 numbers.
-m: test mode. 0 [ naive conversion of double to uint64_t ] ; 1 [ test
the higher 32 bits of the 52bit mantissa ] and 2 [ test the lower 32
bits of the 52bit mantissa ]
Please, let me know if you can't manage to access and run it.
Look forward for listening your impressions.
A.A.
On 28/10/2020 01:06, Lorenzo wrote:
Hi Lorenzo, thank you very much. Can you access the repository below?
https://gitlab.ikp.kit.edu/AAAlvesJr/C7PRNG
It is a cmake project. Building:
You can use prng_streamer to run PractRand.
Usage:
/prng_streamer -s{0,1,2} -m{0,1,2}| /opt/practrand/RNG_test stdin
Options:
-s: sequence number: the generator is generating 3 sequences in chunks of 1024 numbers.
-m: test mode. 0 [ naive conversion of double to uint64_t ] ; 1 [ test the higher 32 bits of the 52bit mantissa ] and 2 [ test the lower 32 bits of the 52bit mantissa ]
Please, let me know if you can't manage to access and run it.
Look forward for listening your impressions.
Thanks! I'll have a look at it during the weekend.
Hello, I had a look, but please note that I'm no expect in such things and I may be doing something trivially wrong.
First, I used prng_streamer with options -m from 0 to 2, and the otput stream I got is clearly non-random (it's sufficient to open, say, the first megabyte as a raw greyscale image to see lots of patterns in the data). Consequently, PractRand fails instantly for these streams. I haven't checked your source code, but it seems that you're extracting the random bits incorrectly.
I wrote a rough-and-ready Fortran program (quoted at the end of this message) to do some independent tests.
The program extracts one byte at a time from the double precision number returned by the routine with the RNG, with the first byte corresponding to the least-significant bits.
I output to disk and then pass the file to PractRand (output to screen for some reason was very slow on my system; for small tests this is okay). Results are:
byte 1 : fails at 1K
byte 2: fails at 16K
byte 3: fails at 2GB
byte 4: fails at 16GB
byte 5: fails at 32GB
byte 6: doesn't fail up to and including 64GB
So it would seem that the low bits are very non-random and then progressively increase in quality, but I'd bet that byte 6 will also fail soon, perhaps at 256GB.
There should be 4 more random bits in byte 7, in principle the best, but I wasn't succesful in extracting and testing them so far.
So, unless I'm doing something wrong, it seems that this generator kind-of sucks and that only the very highest bits have somewhat good randomness; this is perhaps not surprising for a generator from 1990. Also, in the paper the generator producesonly random 24bits, and I'm not sure the version you have got, returning 64bit floating point numbers, has extended and tested the algorithm for this case.
I'm also not 100% that it's correct to test the bits of the mantissa of a floating point number in this way.
By the way, I've open a thread on this topic on the newsgroup comp.lang.fortran (thread 'how to convert mantissa of double precision number to bit stream'); you should have a look there, many of the people there are seasoned programmers which are much more competent than I am in such things.
In any case my preliminary conclusion is that the generator is, by today's standard, poor.
Also, it's fairly slow, has a huge state of ~64000 bits (but a period of only 2^144 if I recall correctly) and a very complicated initialization procedure.
I hope this helps a bit.
Lorenzo
BTW, I confirm that byte6 does fail PractRand at 256GB:
Thank you very much Lorenzo. Your results just confirmed my suspicions. Ï will update my testing wrapper to perform the byte extraction in the way you did, and document well the whole thing, to recomend the core developers the adoption of a more reliable PRNG.
Thank you very much.
I've done some hacks to your c++ code. I think the approach is slightly different to Lorenzo's and possibly a more useful way to convert double to 32 bit unsigned int. Unfortunately it is untested as i don't yet have a fortran compiler. Notes follow in the next comment. Ignore if you're happy with what you've already got.
And here's the notes. I'll add that i suspect the prng is not all that good. Whether it's bad enough to invalidate the actual application is hard to know. Often even quite bad prngs can be useful, but not always. Hopefully in the rewrite of the application, there will be some thought to being able to replace the prng with minimal pain, so at least two different prngs can be tested to see how consistent results are. Best of luck.
Thank you very much G. Jones, I will look into it right now and update the testing code. I will post back here the results. About the impact of the PRNG quality in the results, it is hard to say for sure, agree. Also, it would be harder to find out than just update the PRNG for a reliable one. It is plenty of choice around, with very good performances. Let's see...