From: Jimmy D. <cri...@gm...> - 2006-08-02 06:30:07
|
On 8/1/06, Maciej Sakrejda <m.s...@gm...> wrote: > > 2. SearchTree needs add/remove/change events, but the re-design > > doesn't emit any. I'll try to add these events back in somewhere. > > Ah. Well, are we still doing FS monitoring on watched folders at all, > then? Yup, we're still using FS monitoring, except the code for handling the add/deleted/changed notifications has been unified with the code for scanning. > This seems like a feature we want to keep. I don't know how much > this would play into the redesign, but we had talked about abstracting > away raw FS events into actually relevant SnapFSEvents or something > like that--to, among other things, resolve the exiv2 file update > (copy/delete/rename) nonsense. If this is something that you think > would dovetail nicely with your current redesign, I can jump on it now > (I'm still feeling massively exhausted after each day of work, but > I'll get used to it soon). I don't remember the details of SnapFSEvents, so I can't really say how easy it would be to implement with this new design. I did, however, just add PicAdded, PicRemoved, and PicChanged events (but no code actually uses those events right now) to PictureCatalog, so PictureCatalog might be a good place to look for implementing SnapFSEvents. In that same class, I've also added a private instance of SearchTree so that we can update the search tree as new files are scanned, added, removed, or updated. To take advantage of this SearchTree instance, PictureCatalog now has a Search() method for performing searches. Great job with the SearchTree, Maciek. I just gave it a try and it appears to be blazingly fast. I've gotten search to work now, but you might want to take a look at everything to make sure I didn't do anything crazy during my merge. Jimmy Otheriwse, if that part can be just as > easily done independently, I'll get on it in a week or two. > > -Maciek > > > > > Hmm...actually, I think I'm going to commit with the non-functional > > search, and then try to fix it within the next couple days. My hard > > drive seems to be having some issues, so I'd feel safer knowing that > > all changes are in SVN. > > > > Jimmy > > > > > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > > Wow. No DB at all... Well, are you letting the IWhateverFilter > > > > > elements do all the legwork then? I'm reluctant to make such big > > > > > changes without a solid reconsideration of the whole design... I'm > > > > > working on the Trie tests now, and should be able to commit my stuff > > > > > later today. There's not *that* much changes to the stuff you changed, > > > > > but it still might be tricky... > > > > > > > > > > I guess my main concern about dumping the DB is what avenues of > > > > > expansion it closes off to us--right now it seems it'd be pretty > > > > > trivial to expand the DB stuff to do very complex queries ((this label > > > > > and this label) or (this label and not that label) or earlier than > > > > > this date) if we figure out how to expose this to the GUI. I guess the > > > > > real questions are > > > > > > > > > > 1. How useful would this really be in the first place? > > > > > 2. How difficult would this be to replicate without the DB, if we *do* > > > > > want to do it? > > > > > > > > > > A four-fold speed-up is nothing to sneeze at, but I'd hate to think it > > > > > comes at the cost of totally crippled functionality... > > > > > > > > > > -Maciek > > > > > > > > > > On 7/30/06, Jimmy Do <cri...@gm...> wrote: > > > > > > Nope, not using the DB for queries. We weren't using very complex DB > > > > > > queries anyway because we just kept asking the DB for all pictures. > > > > > > Consequently, I don't think we really need QueryLanguage. But then > > > > > > again, maybe you guys want to keep queries and QueryLanguage, so > > > > > > perhaps I shouldn't merge my changes. But if you want the changes, I > > > > > > guess I should merge into main first and then you merge your changes, > > > > > > just to make sure I don't overwrite your changes. > > > > > > > > > > > > Jimmy > > > > > > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > > > > But we're still using the DB for queries, right? Just not for the > > > > > > > cache? I guess I should take a look at your changes. I'll try to do > > > > > > > that today, maybe. Also, should I commit my changes, and have you > > > > > > > merge the ScanFaster branch with that, or do you want to merge with > > > > > > > the current trunk and then I can merge my changes with the new trunk? > > > > > > > > > > > > > > > > > > > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > > > > > > > Hey guys, > > > > > > > > > > > > > > > > So I think I've got the ScanFaster branch working pretty well. Please > > > > > > > > check it out and see if you can break it. I'll merge it into the main > > > > > > > > trunk if you guys approve. > > > > > > > > > > > > > > > > Major Changes: > > > > > > > > * Faster scanning by storing all IPictureInfos directly into a > > > > > > > > List<>. Previously, the database inserts during scans were slowing > > > > > > > > things down a lot (about 4 times slower with database inserts). We now > > > > > > > > have a separate thread that periodically syncs the current list of > > > > > > > > IPictureInfos into a file by serializing it. SQLite is no longer used > > > > > > > > for storing the cache because syncing the pictures to an SQLite DB > > > > > > > > took about 20 *seconds* for only 60 pictures. With serialization, it > > > > > > > > took less than 1 second to sync almost 1000 pictures. > > > > > > > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > > > > > > > to 3 minutes. > > > > > > > > > > > > > > > > * New design also results in quicker discovery of missing files upon > > > > > > > > startup. For example, close the app, remove all pictures, and start > > > > > > > > the app again. Before, if you had a lot of pictures, they would slowly > > > > > > > > disappear from the UI. Now, all removed pictures will disappear right > > > > > > > > away. > > > > > > > > > > > > > > > > * Preliminary UI for showing scan progress. > > > > > > > > > > > > > > > > > > > > > > > > Jimmy > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > > _______________________________________________ > > > > > > > > Snap-photo-devel mailing list > > > > > > > > Sna...@li... > > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > _______________________________________________ > > > > > > > Snap-photo-devel mailing list > > > > > > > Sna...@li... > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > _______________________________________________ > > > > > > Snap-photo-devel mailing list > > > > > > Sna...@li... > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > _______________________________________________ > > > > > Snap-photo-devel mailing list > > > > > Sna...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of IT > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > opinions on IT & business topics through brief surveys -- and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Snap-photo-devel mailing list > > > Sna...@li... > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Snap-photo-devel mailing list > > Sna...@li... > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |