From: LucaDC <dicappello@am...>  20101129 15:10:03

Jasper van de Gronde wrote: > > And that's the point. We do NOT want to get 255 as maximum starting from > 256, we want to get 255 as maximum from 255. > ... > This make sense now? > Yes: I understand I didn't explain myself well enough. Sorry. I was trying to say the exact same think. I meant that if the final sum can be up to 2^n (not the initial values! I know that the range is 0255 i.e. 8 bits) you get into troubles when dividing by 2^(n8). Of course if the coefficients sum up to 2^n but the starting values are up to 255 (i.e. (2^8)1), the sum never reaches 2^n and the result of the final shift >>(n8) will never be more than 255. Of course this means that your maximum sum is not 2^n but 255*2^(n8). This is what I was referring as '1' while, if I correctly understood, you considered the sum of the coefficients and this can be done only because the starting variables all have the same maximum. Try doing the same reasoning for three different variables going from 0 respectively to 255, 127 and 63 to obtain a maximum final of 255. You'll see that you need a much different approach. Anyway, I'm glad that we agree on this simpler case and I hope this clarifies a bit more my thoughts. Regards Luca  View this message in context: http://old.nabble.com/Maskcalculationerror.tp30273691p30331302.html Sent from the Inkscape  Dev mailing list archive at Nabble.com. 