[Scaffoldhunter-devel] GSOC project: Interactive Image Query Formation

[Girish M. <gir...@gm...>]

Hi,

I found the description of this
project<http://scaffoldhunter.sourceforge.net/wiki/doku.php?id=project_ideas#image_visual_feature_selection>pretty
interesting. During my summer internship last year, I implemented a
supervised binary hashing based approach for a database retrieval system
for pose estimation using simple nearest neighbors. This project involved
implementing state-of-the-art ideas from 3 IEEE CVPR 2012 (Computer Vision
and Pattern Recognition) papers which ranged from the state of the art
methods for locality sensitive hashing to fast nearest neighbor search by
non-linear embedding and fast search in hamming space with multi index
hashing. A short presentation about my implementation can be seen
here<https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxnaXJpc2htYWxrYXJuZW5rYXJ8Z3g6N2I4YjZkYTQ1YjA0NTFkMQ>
.

I was wondering if a similar approach could be used here where the problem
is formulated as a image retrieval task rather than a classification task.
Since we want a real time querying experience, a binary hashing method
(such as Spherical Hashing <http://sglab.kaist.ac.kr/Spherical_Hashing/>)
might be very useful. Assuming the task to be finding the most similar
image from a database of N images, we can train the binary hashing
functions on the features (we can use either
GIST<http://people.csail.mit.edu/torralba/code/spatialenvelope/>,
SIFT <http://www.cs.ubc.ca/~lowe/keypoints/>,
SURF<http://www.vision.ee.ethz.ch/~surf/>or a high dimensional
combination of these using a bag of visual words
approach). The binary hashing training algorithm will take care of figuring
out which are the important dimensions/features from this combination of
features. Given the high dimensional representation of each image, apart
from using binary hashing which is a form of approximate k-Nearest
Neighbors, we can also use an exact fast k-NN method such as
this<http://research.yoonho.info/fnnne/>,
which would still be much faster than a normal k-NN approach). The
approximate k-NN & the exact k-NN can be used in a filter & refine method
where we first obtain the X most similar image to the query image from the
N images in the database using the (super fast) binary hashing approach and
then re-rank these using the exact k-NN algorithm.

I would appreciate any comments/suggestions on this proposal.

Thanks,
Girish
http://www.girishmalkarnenkar.com/