Histogram - HOG/PHOG, How is the data stored in the Histogram array?

[Anonymous]

Hi all,

I am using the HOG class that extract the HOG features of an image and return an Histogram (Histogram class).
The histogram class is simple, it's a double[]. However, I do not find any vizualisations methods or class to permit me to see this output.

Here is what I use:

HOG hog = new HOG(bins, true , FImageGradients.Mode.Unsigned,new FixedHOGStrategy(cellSize, cellSize, FixedHOGStrategy.BlockNormalisation.L1sqrt));
hog.analyseImage(img);
Histogram hist = hog.getFeatureVector(new Rectangle(0, 0, 640, 480));

I tried to code my own method, but I can see that it displays the local Histogram at the wrong place as the histogram doesn't make sense with the image and some artifacts are clearly noticeable. I clearly don't print the right data in the right place. That's because I have no idea how the data is stored in the double[], in what order, what sequence?

If anyone knows how to it is stored, or know a way to display it, let me know.
Many thanks,
Yoann Varquet

[Anonymous]

After a long time on it, I worked it out. It was mostly because I didn't understand fully the use of:`

:::java
FixedHOGStrategy(cellSize, 2,FixedHOGStrategy.BlockNormalisation.L1sqrt));

Say you have an image 64x128
you bin it 8x8 (cell size)
that's 8x16 cells to form the image
it is also 7x15 blocks (8-1 and 16-1 as they overlap by 50%)
each block are 16x16pixels, and also 2x2cells;

Here is where my error laid:
the strategy I was using didn't have the right parameters:

public FixedHOGStrategy(int cellSize,
int cellsPerBlock,
FixedHOGStrategy.BlockNormalisation norm)

in this case, cellSize = 8 & cellsPerBlock = 2. (I understood cells per block as in 2x2=4, but in facts what it requires is the BlockSize)

anyway once I work that out, the function I used was pretty straight forward:

static FImage getHOGVisualisation(FImage color_origImg, Histogram hist, int cellSize, int gradientBinSize) {

        int DIMX = color_origImg.width;
        int DIMY = color_origImg.height;

        double max = hist.max();
        double[] descriptorValues = hist.values;

        float zoomFac = 2; //to see better
        ResizeProcessor resizer = new ResizeProcessor(zoomFac);
        resizer.processImage(color_origImg);
        FImageRenderer imgRenderer = new FImageRenderer(color_origImg);

        float radRangeForOneBin = (float) (Math.PI / (float) gradientBinSize); // dividing 180 into 9 bins, how large (in rad) is one bin?

        // prepare data structure: 9 orientation / gradient strengths for each cell
        int cells_in_x_dir = DIMX / cellSize;
        int cells_in_y_dir = DIMY / cellSize;
        float[][][] gradientStrengths = new float[cells_in_y_dir][cells_in_x_dir][gradientBinSize];
        int[][] cellUpdateCounter = new int[cells_in_y_dir][cells_in_x_dir];

        // nr of blocks = nr of cells - 1
        // since there is a new block on each cell (overlapping blocks!) but the last one
        int blocks_in_x_dir = cells_in_x_dir - 1;
        int blocks_in_y_dir = cells_in_y_dir - 1;

        // compute gradient strengths per cell
        int descriptorDataIdx = 0;
        int cellx = 0;
        int celly = 0;

        for (int blocky = 0; blocky < blocks_in_y_dir; blocky++) {
            for (int blockx = 0; blockx < blocks_in_x_dir; blockx++) {
                // 4 cells per block ...
                for (int cellNr = 0; cellNr < 4; cellNr++) {
                    // compute corresponding cell nr
                    cellx = blockx;
                    celly = blocky;
                    if (cellNr == 1) {
                        cellx++;
                    }
                    if (cellNr == 2) {
                        celly++;
                    }
                    if (cellNr == 3) {
                        cellx++;
                        celly++;
                    }
                    for (int bin = 0; bin < gradientBinSize; bin++) {

                        //double gradientStrength = descriptorValues[descriptorDataIdx]; (normal values)
                        double gradientStrength = map(descriptorValues[descriptorDataIdx], 0, max, 0, 1);
                        descriptorDataIdx++; //values maped between 0 and 1

                        gradientStrengths[celly][cellx][bin] += gradientStrength;

                    } // for (all bins)

                    // note: overlapping blocks lead to multiple updates of this sum!
                    // we therefore keep track how often a cell was updated,
                    // to compute average gradient strengths
                    cellUpdateCounter[celly][cellx]++;

                } // for (all cells)

            } // for (all block x pos)
        } // for (all block y pos)

        // compute average gradient strengths
        for (cellx = 0; cellx < cells_in_x_dir; cellx++) {
            for (celly = 0; celly < cells_in_y_dir; celly++) {

                float NrUpdatesForThisCell = (float) cellUpdateCounter[celly][cellx];

                // compute average gradient strenghts for each gradient bin direction
                for (int bin = 0; bin < gradientBinSize; bin++) {
                    gradientStrengths[celly][cellx][bin] /= NrUpdatesForThisCell;
                }
            }
        }

        // draw cells
        for (celly = 0; celly < cells_in_y_dir; celly++) {
            for (cellx = 0; cellx < cells_in_x_dir; cellx++) {
                int drawX = cellx * cellSize;
                int drawY = celly * cellSize;

                int mx = drawX + cellSize / 2;
                int my = drawY + cellSize / 2;

                //rectangle(visu, Point((int) (drawX), (int) (drawY)), Point((int) ((drawX + cellSize)), (int) ((drawY + cellSize))), Scalar(100, 100, 100), 1);
                imgRenderer.drawLine(drawX * zoomFac, drawY * zoomFac, drawX * zoomFac, (drawY + cellSize) * zoomFac, 1, 1.0f);
                imgRenderer.drawLine(drawX * zoomFac, drawY * zoomFac, (drawX + cellSize) * zoomFac, drawY * zoomFac, 1, 1.0f);
                imgRenderer.drawLine(drawX * zoomFac, (drawY + cellSize) * zoomFac, (drawX + cellSize) * zoomFac, (drawY + cellSize) * zoomFac, 1, 1.0f);
                imgRenderer.drawLine((drawX + cellSize) * zoomFac, drawY * zoomFac, (drawX + cellSize) * zoomFac, (drawY + cellSize) * zoomFac, 1, 1.0f);

                // draw in each cell all 9 gradient strengths
                for (int bin = 0; bin < gradientBinSize; bin++) {
                    float currentGradStrength = gradientStrengths[celly][cellx][bin];

                    // no line to draw?
                    if (currentGradStrength == 0) {
                        continue;
                    }

                    double currRad = bin * radRangeForOneBin + radRangeForOneBin / 2;

                    float dirVecX = (float) Math.cos(currRad);
                    float dirVecY = (float) Math.sin(currRad);
                    float maxVecLen = (float) (cellSize / 2.f);
                    float scale = 1.5f; // just a visualization scale, to see the lines better

                    // compute line coordinates
                    float x1 = mx - dirVecX * currentGradStrength * maxVecLen * scale;
                    float y1 = my - dirVecY * currentGradStrength * maxVecLen * scale;
                    float x2 = mx + dirVecX * currentGradStrength * maxVecLen * scale;
                    float y2 = my + dirVecY * currentGradStrength * maxVecLen * scale;

                    // draw gradient visualization
                    //line(visu, Point((int) (x1 * zoomFac), (int) (y1 * zoomFac)), Point((int) (x2 * zoomFac), (int) (y2 * zoomFac)), Scalar(0, 255, 0), 1);
                    imgRenderer.drawLine(x1 * zoomFac, y1 * zoomFac, x2 * zoomFac, y2 * zoomFac, 1, 0f);

                } // for (all bins)

            } // for (cellx)
        } // for (celly)

        return color_origImg;

    } // get_hogdescriptor_visu



    //mapping function (from arduino)
    private static double map(double x, double in_min, double in_max, double out_min, double out_max) {
        return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
    }

I hope that helps.
Good luck to all.

Yoann