Re: [Scaffoldhunter-devel] GSOC project: Interactive Image Query Formation

[Jinman K. <jin...@sy...>]

Hi Girish,

I am starting to better understand your idea - I think I misunderstood you in my initial response.... In any case, my point about the difference between retrieval/classification is that if we want to find the most relevant image from the database to the query, we cant assume that there is a class (or a training sample) that the query image will relate to, at least not optimally. Exact or Approximate matching, as you describe it, however would work

As for your feature selection, your approach to defining classifiers is interesting and would provide real-time calculations. How flexible it is and how to ensure proper classes are defined needs to be thought about. In this project, we will use medical images and hence you should look into how features are used in e.g., CT data, see for example L. Dettori and L. Semler, 'A comparison of wavelet, ridgelet, and curvelet-based texture classification algorithms in computed tomogarphy' Computers in Biology and Medicine, 2007

Could you, in your proposal, elaborate on how your approach would fit into the Scaffold Hunter framework?

Best wishes.
Jinman

From: Girish Malkarnenkar [mailto:gir...@gm...]
Sent: Friday, 26 April 2013 3:52 AM
To: Jinman Kim
Cc: sca...@li...
Subject: Re: [Scaffoldhunter-devel] GSOC project: Interactive Image Query Formation

Hi Jinman,
Thanks for the feedback! :)

Regarding retrieval v/s classification, I meant we could perform retrieval the same way as done in nearest neighbor (either exact or approximate) based classification where we first find the most similar image. We can skip the classification step (where we would label the query image with the class of the most similar image) as we are interested only in finding the most similar image(s). I missed your point, about it being impossible to translate classification to retrieval. Can you explain a bit more?
Regarding the real time feature selection, I was thinking we could perhaps train different classifiers for the various features and at run time, depending on the user's input, we run only the necessary classifiers. Since each of the classifiers would be based on a super fast binary hashing (hamming distance) based calculation, we can guarantee real time performance for any of the classifiers or a combination of them (if the user selects similarity based on 2 features say). What do you feel?

Thanks,
Girish


On Thu, Apr 25, 2013 at 11:15 AM, Jinman Kim <jin...@sy...<mailto:jin...@sy...>> wrote:
Hi Girish,

Thanks for your interest and sharing your funny slides - gold membership is some achievement! Its good to see you applying state-of-the-art algorithms. I think you can build up to a strong proposal. Your proposal is interesting - it is impossible to translate classification to retrieval - may I suggest you explore these two points?

* In this project, we want to in real-time change the feature to use in the retrieval e.g. a dynamic feature selection / wieght
* Your idea of coarse-to-fine filtering could work - one could use classifier to narrow the large database and then use more conventional image-to-image matching algorithms....

best

Jinman
________________________________
From: Girish Malkarnenkar [gir...@gm...<mailto:gir...@gm...>]
Sent: Wednesday, 24 April 2013 2:06 AM
To: sca...@li...<mailto:sca...@li...>
Subject: [Scaffoldhunter-devel] GSOC project: Interactive Image Query Formation
Hi,

I found the description of this project<http://scaffoldhunter.sourceforge.net/wiki/doku.php?id=project_ideas#image_visual_feature_selection> pretty interesting. During my summer internship last year, I implemented a supervised binary hashing based approach for a database retrieval system for pose estimation using simple nearest neighbors. This project involved implementing state-of-the-art ideas from 3 IEEE CVPR 2012 (Computer Vision and Pattern Recognition) papers which ranged from the state of the art methods for locality sensitive hashing to fast nearest neighbor search by non-linear embedding and fast search in hamming space with multi index hashing. A short presentation about my implementation can be seen here<https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxnaXJpc2htYWxrYXJuZW5rYXJ8Z3g6N2I4YjZkYTQ1YjA0NTFkMQ>.
I was wondering if a similar approach could be used here where the problem is formulated as a image retrieval task rather than a classification task. Since we want a real time querying experience, a binary hashing method (such as Spherical Hashing<http://sglab.kaist.ac.kr/Spherical_Hashing/>) might be very useful. Assuming the task to be finding the most similar image from a database of N images, we can train the binary hashing functions on the features (we can use either GIST<http://people.csail.mit.edu/torralba/code/spatialenvelope/>, SIFT<http://www.cs.ubc.ca/~lowe/keypoints/>, SURF<http://www.vision.ee.ethz.ch/~surf/> or a high dimensional combination of these using a bag of visual words approach). The binary hashing training algorithm will take care of figuring out which are the important dimensions/features from this combination of features. Given the high dimensional representation of each image, apart from using binary hashing which is a form of approximate k-Nearest Neighbors, we can also use an exact fast k-NN method such as this<http://research.yoonho.info/fnnne/>, which would still be much faster than a normal k-NN approach). The approximate k-NN & the exact k-NN can be used in a filter & refine method where we first obtain the X most similar image to the query image from the N images in the database using the (super fast) binary hashing approach and then re-rank these using the exact k-NN algorithm.
I would appreciate any comments/suggestions on this proposal.
Thanks,
Girish
http://www.girishmalkarnenkar.com/