## Re: [Inkscape-devel] Mask calculation error.

 Re: [Inkscape-devel] Mask calculation error. From: Jasper van de Gronde - 2010-11-29 14:10:23 ```On 2010-11-29 10:18, LucaDC wrote: > Krzysztof Kosiński wrote: >> >> In this case it doesn't matter because the input values are limited to >> 0...255. We can use any multiplier we want - as long as the coeffs sum >> up to the value we use as the divisor, it will work correctly. There >> are many other cases where it won't, precisely because of the reason >> you mentioned (e.g. in blending, or when alpha is used as a >> coefficient) and then we can't avoid the division. >> > Well, I was speaking about avoiding the final multiplication and the use of > floats: if you want to get 255 as final maximum starting from 256 (or 512, > or 2^n), And that's the point. We do NOT want to get 255 as maximum starting from 256, we want to get 255 as maximum from 255. We have three values (r,g,b) in the range [0,255] and want to weigh these according to certain coefficients (giving l, the luminance, in the range [0,255]), lets call these cr, cg and cb (and they are defined such that cr+cg+cb=1). So the ideal computation would be: l = cr*r+cg*g+cb*b Now, as we've noted we'd like to avoid floating point computations, as they're usually still a bit slower, and we're using integers, so why not keep them as integers. So instead of multiplying by cr,cg,cb we multiply by Bcr,Bcg,Bcb (which are approximately the old coefficients scaled by B, such that Bcr+Bcg+Bcb=B), and then divide by B again to get the unscaled l: l = (Bcr*r+Bcg*g+Bcb*b)/B To choose a proper B we need two things: it should be large enough to give a relatively precise answer and the division should ideally be easy. To make it large enough we just take some value that is at least as large as the one divided by the smallest coefficient, and we have plenty of spare room, so we can choose something that's comfortably large. To make the division as easy as possible we can choose a power of two so the division becomes a shift. This make sense now? ```