Face detection: can I also get info on the orientation of the detected face?

[spamrefuse]

Hello,

I have been checking out OpenCV and Deepgaze software packages, which can detect faces and tell in what direction faces in a video are looking.

Recently I discovered OpenIMAJ. Very interesting, but can it also do that?

Thank you.
Rob Lahaye.

[Jonathon Hare]

Yes (within reason) if you use the CLMFaceDetector - the returned CLMDetectedFace instances have methods to get the roll, pitch and yaw of the faces.

[spamrefuse]

I want to apply your suggestion in the Tutorial Chapter 8.
Therefore I first decided to solve the exercise 8.1.1.
I suspect the exercise 8.1.1 and the roll/pitch/yaw detection might be related, is it not?

However, I get already stuck with the exercise 8.1.1.
I tried to modify the "public void beforeUpdate( MBFImage frame )" method using the extractFacialKeypoints method, as follows:

public void beforeUpdate( MBFImage frame ) {
  FaceDetector<DetectedFace,FImage> fd = new HaarCascadeDetector(40);
  List<DetectedFace> faces = fd.detectFaces(   Transforms.calculateIntensity(frame));

  for( DetectedFace face : faces ) {
   frame.drawShape(face.getBounds(), RGBColour.RED);

   FacialKeypointExtractor fke;
   FacialKeypoint fk[] = fke.extractFacialKeypoints(face.getFacePatch());
  }
}

The error here is that "fke" is not initialized.....but what should I initialize it with?

I tried to find code examples that use the extractFacialKeypoints(), but Google found none; neither do the apidocs on the OpenImaj website provide example code.....so I'm lost what to do.

Can you please give me a useful hint and explanation?

Thank you.
R.

[Jonathon Hare]

I'm not sure why you're trying to use a FacialKeypointExtractor directly? As the tutorial says, all you need to do is replace the HaarCascadeDetector with an FKEFaceDetector which will return you KEDetectedFace objects instead of plain DetectedFace objects:

public void beforeUpdate( MBFImage frame ) {
  FKEFaceDetector fd = new FKEFaceDetector();
  List<KEDetectedFace> faces = fd.detectFaces( Transforms.calculateIntensity( frame ) );

  for( KEDetectedFace face : faces ) {
    frame.drawShape(face.getBounds(), RGBColour.RED);

     //you can now do things with the keypoints
     FacialKeypoint[] keypoints = face.getKeypoints();
     // ...
  }
}

Theoretically you could estimate roll, pitch and yaw from the keypoints, although you'd have to implement something yourself to do that. However, as mentioned in the earlier post, you can just use the CLMFaceDetector instead:

public void beforeUpdate( MBFImage frame ) {
  CLMFaceDetector fd = new CLMFaceDetector();
  List<CLMDetectedFace> faces = fd.detectFaces( Transforms.calculateIntensity( frame ) );

  for( CLMDetectedFace face : faces ) {
    frame.drawShape(face.getBounds(), RGBColour.RED);

     System.out.println(face.getRoll());
  }
}

[spamrefuse]

Thanks!
It takes me a bit longer to grasp the philosophy of OpenImaj. I did not realize the solutions were so straightforward. Very interesting indeed.

By the way: is there a new version of openimaj-quickstart-archetype (version 1.3.6)?
If I select this latest one and import to Eclipse, the editor cannot generate the appropriate import suggestions. Version 1.3.5 seems to be OK.

I used the latest version for the Tutorial 9 (Audio) and could not import the relevant Audio packages.
Is this a problem at my end, or a bug in version 1.3.6?

Regards,
R.

[Jonathon Hare]

I'm about half-way through deploying 1.3.6, so not all the jars have been uploaded yet (although it doesn't require user-interaction it's a very slow process that takes a few hours to deploy all the jars, sort out the git release, update the new site, etc). Once this has completed 1.3.6 should work just fine.

[spamrefuse]

I suppose 1.3.6 should be up and working by now.

When I use it, Eclipse complains about errors after I imported the standard "Hello World" quick start.
I followed the updated instructions from the Tutorial.

In Eclipse, the project listing in the left panel has that small error marker attached to the pom.xml file. When I run the "Hello World" App.java code, the error message is:
"Error: Could not find or load main class XYZ.App"
where XYZ is the GroupID I typed, when creating maven.

1.3.5 does not have this issue; so for now I keep going with 1.3.5....

Or do I have to tell maven to "reset" or "reinitialize" itself when switching from 1.3.5 to 1.3.6?

Thanks!
R.