I updated the code in FPGA designed to facilitate automatic brightness and color balance adjustments. It also can help now to focus the camera (for now - just as indicator, as there is no electrical control for the lenses yet). Another application of this code - motion detection software.
As there is little resources left in the FPGA I decided not to implement additional programmed window where average brightness and colors are calculated, but provide this data individually for each MCU (16x16 pixels tile) when it is processed in the JPEG compressor.
The data is output in PIO mode through 2 ping-pong buffers in series. First one is 2*128*16bit - while one of the 128x16 pages is being filled, the other is available for the output. Output of that buffer is fed to the 2*2*32 bit one so a pair of 16-bit words is packed in one 32-bit one (LSW first). These 2 buffers were made for a PIO access to the frame buffer SDRAM (channel 3) so they are mutually exclusive at any given moment. To put channel 3 in this mode a control word with bit 12 set (0x1000) should be written to the register X313_WA_SDCH3_CTL0 (0x2c), to reset - write 0 to the same location.
When the DCC ("DC components") mode is initialized the buffer is reset and the data will start to arrive to the buffer at the start of new frame compressing (if it happened in the middle of the frame no data will arrive before the next frame). Data will arrive in groups of 4 16-bit words:
1 - unsigned average intensity, scaled so that 0x1000 corresponds to middle scale (pixel value of 128). Currently the very first word in a frame has MSB set (OR-ed with 0x8000)
2 - signed Cb color difference (+/- 0x400 full scale);
3 - signed Cr color difference (+/- 0x400 full scale);
4 - unsigned sum of absolute values of high frequency components (actually limited now to 13 bits). The high frequency components are defined on the array of 8x8 by having at least one on the coordinates higher than a specified 3 bit number (bits 28-30 of control register - see os/linux/arch/cris/drivers/x313.h). I.e. if the number is 0, then all components except (0,0) will be added together, if 1 - then all but (0,0), (0,1), (1,0) and (1,1), etc.
Specifying value 7 will completely bypass output of high frequency components - only 3 words per MCU will be sent out.
As the data will be sent to the buffer interrupt requests will be generated (bit 5 in the mask). Interrupt is generated when the half of the buffer is full (128 16 bit words) or the frame is over and the last 128 word page is filled partially). Interrupt is reset if the DCC mode is reset (writing 0 to X313_WA_SDCH3_CTL0) or all the data is read out (there is a way to skip data in groups of 8x32bit words).
When reading data from the output (2*8*32bits) buffer it is possible to check data availability (not needed when using interrupts and knowing the frame length) by polling bit X313_SR__PIORDY (7) of the status register (X313__RA__STATUS) - you may use macro X313_SR(PIORDY) to get the value. If the buffer is ready it is mapped to registers X313_WA_SD_PIOWIN to X313_WA_SD_PIOWIN+7 (0x30..0x37). After the data is read out (or just skipped) the buffer may be advanced to the next 8*32 page by writing any data to the FPGA register X313_WA_SDPIO_NEXT (0x2f), after reading 8 of such windows the first buffer (2*128*16) will switch to alternative page. Data transfer between buffers is faster than access from the CPU and the data is being sent to a shadowed 8x32 page wile the other is accessible to the CPU so normally you don't need to test X313_SR(PIORDY) after each write to X313_WA_SDPIO_NEXT, but you may want to do it after reading 8. Or if using interrupts you may just read all 64 of 32-bit words (or less at the end of a frame).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
To try this mode manually (while there is no software support yet) you may do the following:
1. Set the active window size so the whole data will fit in the buffer - it is up to 64 MCUs (16x16 pixels each) - i.e. 128x128 pixels or smaller;
2. Put SDRAM channel3 access into this mode (access to DC components):
fpcf -w 2c 1000
(to reset the mode enter "fpcf -w 2c 0")
3. Set cutoff frequency for the high frequency components in bits 28-30 of control register. You may do it with "hello -c <hex_bit_number> <command>" - see "hello -?", i.e. to write cutoff frequency of 4 (including) enter 3 commands to set these bits to "3":
hello -c 1c 2
hello -c 1d 2
hello -c 1e 1
4. Acquire the image
5. You may verify that the coefficients are acquired with
fpcf -r 10
and see that bit 7 is "1"
6. Now read data in groups of 8 advancing the output buffer pages. You may use a simple script like this:
And see that the last output (status register) has bit 7 set before advancing. It can go low only when crossing page boundaries - multiple of 8 pages (64 32-bit words). As the output data is 32-bit there will be 2 coefficients it each output word, they are packed lower word first.
Have fun!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Added more features for the automatic exposure/black level adjustment. For each MCU (16x16 tile) the numbers of saturated pixels (after gamma correction, before color conversion) having values of 0x00 and 0xff is counted (replacing 0x100 with 0xff to fit in 8 bits). These 2 bites are combined in a 16 bit word (number of 0x00 values - in LSB, number of 0xff-s - in MSB) and output before other 4 (3) words as described in the original posting. So now the corrected list will be as the following:
0 - Two bytes - LSB - number of the pixels with 0x00 value, MSB - number of pixels with 0xff value in the current tile. If all the 256 pixels are white (0xff) or black (0x00) the number will be reduced by 1 to fit in a byte;
1 - unsigned average intensity, scaled so that 0x1000 corresponds to middle scale (pixel value of 128). Currently the very first word in a frame has MSB set (OR-ed with 0x8000)
2 - signed Cb color difference (+/- 0x400 full scale);
3 - signed Cr color difference (+/- 0x400 full scale);
4 - unsigned sum of absolute values of high frequency components (actually limited now to 13 bits). The high frequency components are defined on the array of 8x8 by having at least one on the coordinates higher than a specified 3 bit number (bits 28-30 of control register - see os/linux/arch/cris/drivers/x313.h). I.e. if the number is 0, then all components except (0,0) will be added together, if 1 - then all but (0,0), (0,1), (1,0) and (1,1), etc.
Specifying value 7 will completely bypass output of high frequency components - only 3 words per MCU will be sent out.
Those additions increased the utilization of the FPGA to 99% - 3057 out of 3072 slices, 100% of the memory blocks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I updated the code in FPGA designed to facilitate automatic brightness and color balance adjustments. It also can help now to focus the camera (for now - just as indicator, as there is no electrical control for the lenses yet). Another application of this code - motion detection software.
As there is little resources left in the FPGA I decided not to implement additional programmed window where average brightness and colors are calculated, but provide this data individually for each MCU (16x16 pixels tile) when it is processed in the JPEG compressor.
The data is output in PIO mode through 2 ping-pong buffers in series. First one is 2*128*16bit - while one of the 128x16 pages is being filled, the other is available for the output. Output of that buffer is fed to the 2*2*32 bit one so a pair of 16-bit words is packed in one 32-bit one (LSW first). These 2 buffers were made for a PIO access to the frame buffer SDRAM (channel 3) so they are mutually exclusive at any given moment. To put channel 3 in this mode a control word with bit 12 set (0x1000) should be written to the register X313_WA_SDCH3_CTL0 (0x2c), to reset - write 0 to the same location.
When the DCC ("DC components") mode is initialized the buffer is reset and the data will start to arrive to the buffer at the start of new frame compressing (if it happened in the middle of the frame no data will arrive before the next frame). Data will arrive in groups of 4 16-bit words:
1 - unsigned average intensity, scaled so that 0x1000 corresponds to middle scale (pixel value of 128). Currently the very first word in a frame has MSB set (OR-ed with 0x8000)
2 - signed Cb color difference (+/- 0x400 full scale);
3 - signed Cr color difference (+/- 0x400 full scale);
4 - unsigned sum of absolute values of high frequency components (actually limited now to 13 bits). The high frequency components are defined on the array of 8x8 by having at least one on the coordinates higher than a specified 3 bit number (bits 28-30 of control register - see os/linux/arch/cris/drivers/x313.h). I.e. if the number is 0, then all components except (0,0) will be added together, if 1 - then all but (0,0), (0,1), (1,0) and (1,1), etc.
Specifying value 7 will completely bypass output of high frequency components - only 3 words per MCU will be sent out.
As the data will be sent to the buffer interrupt requests will be generated (bit 5 in the mask). Interrupt is generated when the half of the buffer is full (128 16 bit words) or the frame is over and the last 128 word page is filled partially). Interrupt is reset if the DCC mode is reset (writing 0 to X313_WA_SDCH3_CTL0) or all the data is read out (there is a way to skip data in groups of 8x32bit words).
When reading data from the output (2*8*32bits) buffer it is possible to check data availability (not needed when using interrupts and knowing the frame length) by polling bit X313_SR__PIORDY (7) of the status register (X313__RA__STATUS) - you may use macro X313_SR(PIORDY) to get the value. If the buffer is ready it is mapped to registers X313_WA_SD_PIOWIN to X313_WA_SD_PIOWIN+7 (0x30..0x37). After the data is read out (or just skipped) the buffer may be advanced to the next 8*32 page by writing any data to the FPGA register X313_WA_SDPIO_NEXT (0x2f), after reading 8 of such windows the first buffer (2*128*16) will switch to alternative page. Data transfer between buffers is faster than access from the CPU and the data is being sent to a shadowed 8x32 page wile the other is accessible to the CPU so normally you don't need to test X313_SR(PIORDY) after each write to X313_WA_SDPIO_NEXT, but you may want to do it after reading 8. Or if using interrupts you may just read all 64 of 32-bit words (or less at the end of a frame).
I noticed a bug in the code - it is fixed now.
To try this mode manually (while there is no software support yet) you may do the following:
1. Set the active window size so the whole data will fit in the buffer - it is up to 64 MCUs (16x16 pixels each) - i.e. 128x128 pixels or smaller;
2. Put SDRAM channel3 access into this mode (access to DC components):
fpcf -w 2c 1000
(to reset the mode enter "fpcf -w 2c 0")
3. Set cutoff frequency for the high frequency components in bits 28-30 of control register. You may do it with "hello -c <hex_bit_number> <command>" - see "hello -?", i.e. to write cutoff frequency of 4 (including) enter 3 commands to set these bits to "3":
hello -c 1c 2
hello -c 1d 2
hello -c 1e 1
4. Acquire the image
5. You may verify that the coefficients are acquired with
fpcf -r 10
and see that bit 7 is "1"
6. Now read data in groups of 8 advancing the output buffer pages. You may use a simple script like this:
#!/bin/sh
fpcf -r 30;fpcf -r 31;fpcf -r 32;fpcf -r 33;fpcf -r 34;fpcf -r 35;fpcf -r 36;fpcf -r 37;
fpcf -w 2f 0
fpcf -r 10
exit 0
And see that the last output (status register) has bit 7 set before advancing. It can go low only when crossing page boundaries - multiple of 8 pages (64 32-bit words). As the output data is 32-bit there will be 2 coefficients it each output word, they are packed lower word first.
Have fun!
Added more features for the automatic exposure/black level adjustment. For each MCU (16x16 tile) the numbers of saturated pixels (after gamma correction, before color conversion) having values of 0x00 and 0xff is counted (replacing 0x100 with 0xff to fit in 8 bits). These 2 bites are combined in a 16 bit word (number of 0x00 values - in LSB, number of 0xff-s - in MSB) and output before other 4 (3) words as described in the original posting. So now the corrected list will be as the following:
0 - Two bytes - LSB - number of the pixels with 0x00 value, MSB - number of pixels with 0xff value in the current tile. If all the 256 pixels are white (0xff) or black (0x00) the number will be reduced by 1 to fit in a byte;
1 - unsigned average intensity, scaled so that 0x1000 corresponds to middle scale (pixel value of 128). Currently the very first word in a frame has MSB set (OR-ed with 0x8000)
2 - signed Cb color difference (+/- 0x400 full scale);
3 - signed Cr color difference (+/- 0x400 full scale);
4 - unsigned sum of absolute values of high frequency components (actually limited now to 13 bits). The high frequency components are defined on the array of 8x8 by having at least one on the coordinates higher than a specified 3 bit number (bits 28-30 of control register - see os/linux/arch/cris/drivers/x313.h). I.e. if the number is 0, then all components except (0,0) will be added together, if 1 - then all but (0,0), (0,1), (1,0) and (1,1), etc.
Specifying value 7 will completely bypass output of high frequency components - only 3 words per MCU will be sent out.
Those additions increased the utilization of the FPGA to 99% - 3057 out of 3072 slices, 100% of the memory blocks.
To acquire small grayscale picture try the test programs from CVS:
prepchan3 - script with step 2 and 3
Then acquire the image
tumb > /etc/httpd/html/tumb.pnm
and load http://camera/tumb.pnm