Face recognition exception

[Anonymous]

Hi!

I'm currently working on a university project using OpenIMAJ. However, I have problems with correctly using the face recognition API. Face detection always seems to work correctly, but recognition causes an exception at the second call to faceEngine.recogniseBest(). I already tried to normalise the images and scale them to a common resolution but with no success. Even calls with the exact same image twice fail.

Inspired by a Stackoverflow post I put together a small demo to illustrate the problem:

package test;

import org.openimaj.feature.DoubleFVComparison;
import org.openimaj.image.FImage;
import org.openimaj.image.ImageUtilities;
import org.openimaj.image.processing.face.alignment.RotateScaleAligner;
import org.openimaj.image.processing.face.detection.HaarCascadeDetector;
import org.openimaj.image.processing.face.detection.keypoints.FKEFaceDetector;
import org.openimaj.image.processing.face.detection.keypoints.KEDetectedFace;
import org.openimaj.image.processing.face.recognition.EigenFaceRecogniser;
import org.openimaj.image.processing.face.recognition.FaceRecognitionEngine;
import org.openimaj.ml.annotation.ScoredAnnotation;
import org.openimaj.util.pair.IndependentPair;

import java.io.File;
import java.io.IOException;
import java.util.List;

public class FaceRecognitionTest {
    private static FKEFaceDetector faceDetector = new FKEFaceDetector(new HaarCascadeDetector());
    private static EigenFaceRecogniser<KEDetectedFace, String> faceRecognizer = EigenFaceRecogniser.create(20, new RotateScaleAligner(), 1, DoubleFVComparison.CORRELATION, 0.9f);
    private static FaceRecognitionEngine<KEDetectedFace, String> faceEngine = FaceRecognitionEngine.create(faceDetector, faceRecognizer);
    private static int id = 0;

    public static void main(String[] args) throws IOException {
        recognizeFace("lenna.png");
        recognizeFace("lenna.png");
    }

    public static void recognizeFace(String path) throws IOException {
        FImage fimg = ImageUtilities.readF(new File(path));
        List<KEDetectedFace> faces = faceEngine.getDetector().detectFaces(fimg);

        for (KEDetectedFace face : faces) {
            String person = null;
            try {
                List<IndependentPair<KEDetectedFace, ScoredAnnotation<String>>> rfaces = faceEngine.recogniseBest(face.getFacePatch());   // crashes here
                ScoredAnnotation<String> score = rfaces.get(0).getSecondObject();
                if (score != null)
                    person = score.annotation;
            } catch (Exception e) {
            }

            if (person == null) {
                person = new String(String.valueOf(id++));
                System.out.println("Identified new person: " + person);
                faceEngine.train(person, face.getFacePatch());                    // necessary?
            } else {
                System.out.println("Identified existing person: " + person);
            }
        }
    }
}

I used the following image as input:
http://tutorial.simplecv.org/en/latest/_images/lenna.png

Output:

Identified new person: 0
Exception in thread "main" java.lang.Error: dsaupd ERRNO = -2, see http://www.caam.rice.edu/software/ARPACK/UG/node136.html
    at ch.akuhn.matrix.eigenvalues.FewEigenvalues.run(FewEigenvalues.java:212)
    at ch.akuhn.matrix.eigenvalues.SingularValues.decompose(SingularValues.java:51)
    at org.openimaj.math.matrix.ThinSingularValueDecomposition.<init>(ThinSingularValueDecomposition.java:107)
    at org.openimaj.math.matrix.ThinSingularValueDecomposition.<init>(ThinSingularValueDecomposition.java:68)
    at org.openimaj.math.matrix.algorithm.pca.ThinSvdPrincipalComponentAnalysis.learnBasisNorm(ThinSvdPrincipalComponentAnalysis.java:56)
    at org.openimaj.math.matrix.algorithm.pca.PrincipalComponentAnalysis.learnBasis(PrincipalComponentAnalysis.java:183)
    at org.openimaj.math.matrix.algorithm.pca.PrincipalComponentAnalysis.learnBasis(PrincipalComponentAnalysis.java:170)
    at org.openimaj.ml.pca.FeatureVectorPCA.learnBasis(FeatureVectorPCA.java:113)
    at org.openimaj.image.model.EigenImages.train(EigenImages.java:125)
    at org.openimaj.image.processing.face.feature.EigenFaceFeature$Extractor.train(EigenFaceFeature.java:167)
    at org.openimaj.image.processing.face.recognition.EigenFaceRecogniser.beforeBatchTrain(EigenFaceRecogniser.java:159)
    at org.openimaj.image.processing.face.recognition.LazyFaceRecogniser.retrain(LazyFaceRecogniser.java:139)
    at org.openimaj.image.processing.face.recognition.LazyFaceRecogniser.annotate(LazyFaceRecogniser.java:153)
    at org.openimaj.image.processing.face.recognition.EigenFaceRecogniser.annotate(EigenFaceRecogniser.java:55)
    at org.openimaj.image.processing.face.recognition.FaceRecogniser.annotateBest(FaceRecogniser.java:115)
    at org.openimaj.image.processing.face.recognition.FaceRecognitionEngine.recogniseBest(FaceRecognitionEngine.java:260)
    at test.FaceRecognitionTest.recognizeFace(FaceRecognitionTest.java:40)
    at test.FaceRecognitionTest.main(FaceRecognitionTest.java:30)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

Process finished with exit code 1

I am using a gradle build with the following dependency:
compile('org.openimaj:faces:1.3.1')

Of course I am aware that a single image as training input is insufficient for a good recognition rate, but in the example I use the exact same image twice. Anyway, even if the person is not recognized correctly, I would expect the call to not crash but just yield an empty result.

By the way, is the explicit call to faceEngine.train() necessary? It seems faceEngine.recogniseBest() is training automatically (as seen in the stack trace). If I remove that call the program finishes successfully, but the face is not recognized to be the same (also not if I lower the threshold).

Am I missing something? Any idea how to fix this?

Daniel

[Jonathon Hare]

The problem is that you can't use an EigenFaceRecogniser with a single training image - it needs a suitably large number of samples in order to learn the PCA basis before recognition can proceed (they can't all be the same image either as the input matrix would have rank-1).

As for calling faceEngine.train(), yes that's necessary to tell the faceRecogniser that you want it to learn a new example. The reason that you see a call to train (a different train method btw) in the stacktrace is that the EigenFaceRecogniser is a LazyRecogniser and doesn't actually do the training internally until you call one of the recognise methods for the first time (it will re-learn as necessary, so if you call train with new faces, it will internally redo the training on the next call to recognise).

[Anonymous]

Hi Jonathon!

Thanks a lot for your fast and informative reply.
As far as I can see there is no way to check whether the PCA basis has already been learned. I replaced "Exception" by "Throwable" to pass the error.

Daniel

[Anonymous]

After ignoring all previous assert-failures and out-of-bounds exception for 5-7 different images I get another exception:

java.lang.IllegalArgumentException: Vectors have differing lengths
    at org.openimaj.feature.DoubleFVComparison$3.compare(DoubleFVComparison.java:98)
    at org.openimaj.feature.DoubleFVComparison.compare(DoubleFVComparison.java:355)
    at org.openimaj.feature.DoubleFVComparison$3.compare(DoubleFVComparison.java:94)
    at org.openimaj.knn.ObjectNearestNeighbours.distanceFunc(ObjectNearestNeighbours.java:76)
    at org.openimaj.knn.ObjectNearestNeighboursExact.search(ObjectNearestNeighboursExact.java:230)
    at org.openimaj.knn.ObjectNearestNeighboursExact.searchKNN(ObjectNearestNeighboursExact.java:175)
    at org.openimaj.knn.ObjectNearestNeighboursExact.searchKNN(ObjectNearestNeighboursExact.java:49)
    at org.openimaj.ml.annotation.basic.KNNAnnotator.annotate(KNNAnnotator.java:320)
    at org.openimaj.image.processing.face.recognition.AnnotatorFaceRecogniser.annotate(AnnotatorFaceRecogniser.java:139)
    at org.openimaj.image.processing.face.recognition.LazyFaceRecogniser.annotate(LazyFaceRecogniser.java:154)
    at org.openimaj.image.processing.face.recognition.EigenFaceRecogniser.annotate(EigenFaceRecogniser.java:55)
    at org.openimaj.image.processing.face.recognition.FaceRecogniser.annotateBest(FaceRecogniser.java:115)
    at org.openimaj.image.processing.face.recognition.FaceRecognitionEngine.recogniseBest(FaceRecognitionEngine.java:260)
...

Where h1.length is 2 and increases with each trained image and h2.length keeps 1. I tried to resample all images to an equal size but with the same result. Can you help me out?

[Jonathon Hare]

It looks like the faces you're training/testing with have different sizes. It's important for EigenFaces that the face images are exactly the same height and width. In your code above, the face patch returned by face.getFacePatch() would need to be resampled to a standardised size.

[Anonymous]

Got it working, thanks a lot for your support :)

[Anonymous]

Would you please share your working code.