From: Simen Kvaal <simen.kvaal@gm...>  20080519 07:44:21

Hi, and thank you all for replies. Doug, your suggestion for improving the algorithm makes perfectly sense, thank you. I will test a modified implentation ASAP. As for the scaling statement, i.e., Doug Baskins wrote: > Second, I do not understand your statement: > >> However, it seems to scale not so well when the largest indicex > present grows. > If in your example that the number "20" were changed to 1234567890, > I think the performance would be about the same because Judy1 > does not store 0's (missing Indexes) in the array. You might be right; it may be other factors that cause this apparent slowdown. I will address this in a proper benchmark run. Doug Baskins wrote: > > I can answer more of your questions if you like. I am surprised that Judy1 > is used for this "small" of problem. I am use to thinking in 100's of > million > of Indexes. > Thank you. The reason why I try Judy1 is this: I work in computational quantum mechanics, in which a finite set of socalled slater determinants are constructed. These are identified with bit patterns  each bit set corresponds to the whereabouts of a quantum particle. Think of the particles "living" in the set of integers, and a bit set says that the particle at the momemt resides "there". Thus, a bit pattern specifies the state of a manybody quantum system. The number K of particles is fixes, while the number of possible bit positions N varies, and quickly approaches ~1000 for a realistic calculation. The number of bit patterns needed grows exponentially with K, and very fast and efficient way to store and process the bit patterns is needed. The "classical" approach is to store the patterns in real bit bit patterns in words. This is wasteful, especially when N is large. My tests show that Judy1 outperforms this naive approach. This answers, by the way, the comment by John Skaller, i.e, John Skaller wrote: >However given your small set sizes (5) and small maximum element value, >it's not clear how to improve performance algorithmically: it's likely small >machine/compiler dependent quirks will play a significant role in >performance. 1024 bits is 128 bytes, which is pretty close to a cache line >size, so your original bit array was probably quite fast. Yes, the original approach was *very* fast when N was in the range 64128. However, when adding more words to create a Huge Integer class, things got slow ... Of course, there might be other ways to implement a bitpattern. I have considered a simple packed storage, simply storing the indices, but I believe it will be slow to to the comparison thing with it. As for the hybrid storage scheme suggested by John: Maybe, it is not entirely clear to me how it could improve performance, and the size K is small, so it could introduce a lot of overhead. ... I will come back with more test results in a short while. Thank you all for your suggestions. regards, Simen Kvaal.   Simen Kvaal  Ph.D student in Physics/Applied Maths  Centre of Mathematics for Applications, University of Oslo web: http://folk.uio.no/simenkva/ office: +47 22857708 