Re: [Scaffoldhunter-devel] GSOC project: Interactive Image Query Formation

[Girish M. <gir...@gm...>]

Hi Jinman,

Thanks for the feedback! :)

Regarding retrieval v/s classification, I meant we could perform retrieval
the same way as done in nearest neighbor (either exact or approximate)
based classification where we first find the most similar image. We can
skip the classification step (where we would label the query image with the
class of the most similar image) as we are interested only in finding the
most similar image(s). I missed your point, about it being impossible to
translate classification to retrieval. Can you explain a bit more?

Regarding the real time feature selection, I was thinking we could perhaps
train different classifiers for the various features and at run time,
depending on the user's input, we run only the necessary classifiers. Since
each of the classifiers would be based on a super fast binary hashing
(hamming distance) based calculation, we can guarantee real time
performance for any of the classifiers or a combination of them (if the
user selects similarity based on 2 features say). What do you feel?

Thanks,
Girish





On Thu, Apr 25, 2013 at 11:15 AM, Jinman Kim <jin...@sy...>wrote:

>  Hi Girish,
>
>  Thanks for your interest and sharing your funny slides - gold membership
> is some achievement! Its good to see you applying state-of-the-art
> algorithms. I think you can build up to a strong proposal. Your proposal
> is interesting - it is impossible to translate classification to retrieval
> - may I suggest you explore these two points?
>
>  * In this project, we want to in real-time change the feature to use in
> the retrieval e.g. a dynamic feature selection / wieght
> * Your idea of coarse-to-fine filtering could work - one could use
> classifier to narrow the large database and then use more conventional
> image-to-image matching algorithms....
>
>  best
>
>  Jinman
>  ------------------------------
> *From:* Girish Malkarnenkar [gir...@gm...]
> *Sent:* Wednesday, 24 April 2013 2:06 AM
> *To:* sca...@li...
> *Subject:* [Scaffoldhunter-devel] GSOC project: Interactive Image Query
> Formation
>
>   Hi,
>
>  I found the description of this project<http://scaffoldhunter.sourceforge.net/wiki/doku.php?id=project_ideas#image_visual_feature_selection>pretty interesting. During my summer internship last year, I implemented a
> supervised binary hashing based approach for a database retrieval system
> for pose estimation using simple nearest neighbors. This project involved
> implementing state-of-the-art ideas from 3 IEEE CVPR 2012 (Computer
> Vision and Pattern Recognition) papers which ranged from the state of the
> art methods for locality sensitive hashing to fast nearest neighbor search
> by non-linear embedding and fast search in hamming space with multi index
> hashing. A short presentation about my implementation can be seen here<https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxnaXJpc2htYWxrYXJuZW5rYXJ8Z3g6N2I4YjZkYTQ1YjA0NTFkMQ>
> .
>
>  I was wondering if a similar approach could be used here where the
> problem is formulated as a image retrieval task rather than a
> classification task. Since we want a real time querying experience, a
> binary hashing method (such as Spherical Hashing<http://sglab.kaist.ac.kr/Spherical_Hashing/>)
> might be very useful. Assuming the task to be finding the most similar
> image from a database of N images, we can train the binary hashing
> functions on the features (we can use either GIST<http://people.csail.mit.edu/torralba/code/spatialenvelope/>,
> SIFT <http://www.cs.ubc.ca/~lowe/keypoints/>, SURF<http://www.vision.ee.ethz.ch/~surf/>or a high dimensional combination of these using a bag of visual words
> approach). The binary hashing training algorithm will take care of figuring
> out which are the important dimensions/features from this combination of
> features. Given the high dimensional representation of each image, apart
> from using binary hashing which is a form of approximate k-Nearest
> Neighbors, we can also use an exact fast k-NN method such as this<http://research.yoonho.info/fnnne/>,
> which would still be much faster than a normal k-NN approach). The
> approximate k-NN & the exact k-NN can be used in a filter & refine method
> where we first obtain the X most similar image to the query image from the
> N images in the database using the (super fast) binary hashing approach and
> then re-rank these using the exact k-NN algorithm.
>
>  I would appreciate any comments/suggestions on this proposal.
>
>  Thanks,
> Girish
> http://www.girishmalkarnenkar.com/
>
>