Menu
▾
▴
snap-photo-devel — A medium-traffic list for interested developers.
You can subscribe to this list here.
| 2006 |
Jan
|
Feb
|
Mar
(1) |
Apr
(7) |
May
|
Jun
|
Jul
(17) |
Aug
(10) |
Sep
|
Oct
|
Nov
|
Dec
|
|---|
|
From: Maciek S. <m.s...@gm...> - 2006-08-29 02:20:20
|
http://slashdot.org/article.pl?sid=06/08/29/0046237 |
|
From: Jimmy D. <cri...@gm...> - 2006-08-13 18:17:17
|
Separation of logic and presentation would be nice. That was actually what I was aiming for in the initial design, but it didn't turn out quite that way. At least initially, refactoring the existing design might be helpful because it'll keep you aware of the current needs of the interface. Jimmy On 8/12/06, Maciek Sakrejda <m.s...@gm...> wrote: > How difficult would it be to try to rework Snap to separate its logic > and presentation, a la MVC (well, M(VC))? I've been thinking about > this, and it seems that a separation like this would make reworking > individual pieces, like the PictureCatalog or what have you, a lot > easier. I want to go back in and try to redo asynchronous, > transaction-based, batched database access instead of the > synchronization. I guess this would not be impacted that much, but if > we want to redo other components, it might make sense... Since I'm not > that familiar with the GUI code, I wanted to ask you guys (well, I > guess mostly Jimmy) what this would entail... > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Maciek S. <m.s...@gm...> - 2006-08-12 17:24:53
|
How difficult would it be to try to rework Snap to separate its logic and presentation, a la MVC (well, M(VC))? I've been thinking about this, and it seems that a separation like this would make reworking individual pieces, like the PictureCatalog or what have you, a lot easier. I want to go back in and try to redo asynchronous, transaction-based, batched database access instead of the synchronization. I guess this would not be impacted that much, but if we want to redo other components, it might make sense... Since I'm not that familiar with the GUI code, I wanted to ask you guys (well, I guess mostly Jimmy) what this would entail... |
|
From: Maciej S. <m.s...@gm...> - 2006-08-04 02:33:51
|
> Great job with the SearchTree, Maciek. I just gave it a try and it > appears to be blazingly fast. I've gotten search to work now, but you > might want to take a look at everything to make sure I didn't do > anything crazy during my merge. Awesome. I'll check out the merge soon. Also, have you taken a look at the memory usage? I've built SearchTree to be a little more memory-hungry to benefit performance and I just want to make sure we're making the right trade-off. The basic behavior can be tweaked by passing the PrefixTrie a different caching depth parameter in the constructor. The higher the caching depth, the less memory the PrefixTrie will use, at the expense of having to do more work on each query. I'm tempted to write an independent program to just look at how this trade-off behaves, actually... |
|
From: Jimmy D. <cri...@gm...> - 2006-08-04 00:55:55
|
On 8/2/06, Jimmy Do <cri...@gm...> wrote: > Quick note: I just discovered that a recent fix I committed for > correctly sorting IPictureInfos in MainWindow leads to a slow down in > scanning (although it's still faster than the old implementation). I'm > working on an alternative implementation that will not slow things > down as much. Ok, I the slow down has been fixed and I also fixed an exception that would sometimes occur when you typed in a search query. Jimmy > Jimmy > > On 8/1/06, Jimmy Do <cri...@gm...> wrote: > > On 8/1/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > 2. SearchTree needs add/remove/change events, but the re-design > > > > doesn't emit any. I'll try to add these events back in somewhere. > > > > > > Ah. Well, are we still doing FS monitoring on watched folders at all, > > > then? > > > > Yup, we're still using FS monitoring, except the code for handling the > > add/deleted/changed notifications has been unified with the code for > > scanning. > > > > > > > This seems like a feature we want to keep. I don't know how much > > > this would play into the redesign, but we had talked about abstracting > > > away raw FS events into actually relevant SnapFSEvents or something > > > like that--to, among other things, resolve the exiv2 file update > > > (copy/delete/rename) nonsense. If this is something that you think > > > would dovetail nicely with your current redesign, I can jump on it now > > > (I'm still feeling massively exhausted after each day of work, but > > > I'll get used to it soon). > > > > I don't remember the details of SnapFSEvents, so I can't really say > > how easy it would be to implement with this new design. I did, > > however, just add PicAdded, PicRemoved, and PicChanged events (but no > > code actually uses those events right now) to PictureCatalog, so > > PictureCatalog might be a good place to look for implementing > > SnapFSEvents. > > > > In that same class, I've also added a private instance of SearchTree > > so that we can update the search tree as new files are scanned, added, > > removed, or updated. To take advantage of this SearchTree instance, > > PictureCatalog now has a Search() method for performing searches. > > Great job with the SearchTree, Maciek. I just gave it a try and it > > appears to be blazingly fast. I've gotten search to work now, but you > > might want to take a look at everything to make sure I didn't do > > anything crazy during my merge. > > > > Jimmy > > > > > > Otheriwse, if that part can be just as > > > easily done independently, I'll get on it in a week or two. > > > > > > -Maciek > > > > > > > > > > > Hmm...actually, I think I'm going to commit with the non-functional > > > > search, and then try to fix it within the next couple days. My hard > > > > drive seems to be having some issues, so I'd feel safer knowing that > > > > all changes are in SVN. > > > > > > > > Jimmy > > > > > > > > > > > > > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > > > > Wow. No DB at all... Well, are you letting the IWhateverFilter > > > > > > > elements do all the legwork then? I'm reluctant to make such big > > > > > > > changes without a solid reconsideration of the whole design... I'm > > > > > > > working on the Trie tests now, and should be able to commit my stuff > > > > > > > later today. There's not *that* much changes to the stuff you changed, > > > > > > > but it still might be tricky... > > > > > > > > > > > > > > I guess my main concern about dumping the DB is what avenues of > > > > > > > expansion it closes off to us--right now it seems it'd be pretty > > > > > > > trivial to expand the DB stuff to do very complex queries ((this label > > > > > > > and this label) or (this label and not that label) or earlier than > > > > > > > this date) if we figure out how to expose this to the GUI. I guess the > > > > > > > real questions are > > > > > > > > > > > > > > 1. How useful would this really be in the first place? > > > > > > > 2. How difficult would this be to replicate without the DB, if we *do* > > > > > > > want to do it? > > > > > > > > > > > > > > A four-fold speed-up is nothing to sneeze at, but I'd hate to think it > > > > > > > comes at the cost of totally crippled functionality... > > > > > > > > > > > > > > -Maciek > > > > > > > > > > > > > > On 7/30/06, Jimmy Do <cri...@gm...> wrote: > > > > > > > > Nope, not using the DB for queries. We weren't using very complex DB > > > > > > > > queries anyway because we just kept asking the DB for all pictures. > > > > > > > > Consequently, I don't think we really need QueryLanguage. But then > > > > > > > > again, maybe you guys want to keep queries and QueryLanguage, so > > > > > > > > perhaps I shouldn't merge my changes. But if you want the changes, I > > > > > > > > guess I should merge into main first and then you merge your changes, > > > > > > > > just to make sure I don't overwrite your changes. > > > > > > > > > > > > > > > > Jimmy > > > > > > > > > > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > > > > > > But we're still using the DB for queries, right? Just not for the > > > > > > > > > cache? I guess I should take a look at your changes. I'll try to do > > > > > > > > > that today, maybe. Also, should I commit my changes, and have you > > > > > > > > > merge the ScanFaster branch with that, or do you want to merge with > > > > > > > > > the current trunk and then I can merge my changes with the new trunk? > > > > > > > > > > > > > > > > > > > > > > > > > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > > > > > > > > > Hey guys, > > > > > > > > > > > > > > > > > > > > So I think I've got the ScanFaster branch working pretty well. Please > > > > > > > > > > check it out and see if you can break it. I'll merge it into the main > > > > > > > > > > trunk if you guys approve. > > > > > > > > > > > > > > > > > > > > Major Changes: > > > > > > > > > > * Faster scanning by storing all IPictureInfos directly into a > > > > > > > > > > List<>. Previously, the database inserts during scans were slowing > > > > > > > > > > things down a lot (about 4 times slower with database inserts). We now > > > > > > > > > > have a separate thread that periodically syncs the current list of > > > > > > > > > > IPictureInfos into a file by serializing it. SQLite is no longer used > > > > > > > > > > for storing the cache because syncing the pictures to an SQLite DB > > > > > > > > > > took about 20 *seconds* for only 60 pictures. With serialization, it > > > > > > > > > > took less than 1 second to sync almost 1000 pictures. > > > > > > > > > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > > > > > > > > > to 3 minutes. > > > > > > > > > > > > > > > > > > > > * New design also results in quicker discovery of missing files upon > > > > > > > > > > startup. For example, close the app, remove all pictures, and start > > > > > > > > > > the app again. Before, if you had a lot of pictures, they would slowly > > > > > > > > > > disappear from the UI. Now, all removed pictures will disappear right > > > > > > > > > > away. > > > > > > > > > > > > > > > > > > > > * Preliminary UI for showing scan progress. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jimmy > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > > > > _______________________________________________ > > > > > > > > > > Snap-photo-devel mailing list > > > > > > > > > > Sna...@li... > > > > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > > > _______________________________________________ > > > > > > > > > Snap-photo-devel mailing list > > > > > > > > > Sna...@li... > > > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > > _______________________________________________ > > > > > > > > Snap-photo-devel mailing list > > > > > > > > Sna...@li... > > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > _______________________________________________ > > > > > > > Snap-photo-devel mailing list > > > > > > > Sna...@li... > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > _______________________________________________ > > > > > Snap-photo-devel mailing list > > > > > Sna...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > _______________________________________________ > > > > Snap-photo-devel mailing list > > > > Sna...@li... > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of IT > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > opinions on IT & business topics through brief surveys -- and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Snap-photo-devel mailing list > > > Sna...@li... > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > |
|
From: Jimmy D. <cri...@gm...> - 2006-08-02 08:33:21
|
Quick note: I just discovered that a recent fix I committed for correctly sorting IPictureInfos in MainWindow leads to a slow down in scanning (although it's still faster than the old implementation). I'm working on an alternative implementation that will not slow things down as much. Jimmy On 8/1/06, Jimmy Do <cri...@gm...> wrote: > On 8/1/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > 2. SearchTree needs add/remove/change events, but the re-design > > > doesn't emit any. I'll try to add these events back in somewhere. > > > > Ah. Well, are we still doing FS monitoring on watched folders at all, > > then? > > Yup, we're still using FS monitoring, except the code for handling the > add/deleted/changed notifications has been unified with the code for > scanning. > > > > This seems like a feature we want to keep. I don't know how much > > this would play into the redesign, but we had talked about abstracting > > away raw FS events into actually relevant SnapFSEvents or something > > like that--to, among other things, resolve the exiv2 file update > > (copy/delete/rename) nonsense. If this is something that you think > > would dovetail nicely with your current redesign, I can jump on it now > > (I'm still feeling massively exhausted after each day of work, but > > I'll get used to it soon). > > I don't remember the details of SnapFSEvents, so I can't really say > how easy it would be to implement with this new design. I did, > however, just add PicAdded, PicRemoved, and PicChanged events (but no > code actually uses those events right now) to PictureCatalog, so > PictureCatalog might be a good place to look for implementing > SnapFSEvents. > > In that same class, I've also added a private instance of SearchTree > so that we can update the search tree as new files are scanned, added, > removed, or updated. To take advantage of this SearchTree instance, > PictureCatalog now has a Search() method for performing searches. > Great job with the SearchTree, Maciek. I just gave it a try and it > appears to be blazingly fast. I've gotten search to work now, but you > might want to take a look at everything to make sure I didn't do > anything crazy during my merge. > > Jimmy > > > Otheriwse, if that part can be just as > > easily done independently, I'll get on it in a week or two. > > > > -Maciek > > > > > > > > Hmm...actually, I think I'm going to commit with the non-functional > > > search, and then try to fix it within the next couple days. My hard > > > drive seems to be having some issues, so I'd feel safer knowing that > > > all changes are in SVN. > > > > > > Jimmy > > > > > > > > > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > > > Wow. No DB at all... Well, are you letting the IWhateverFilter > > > > > > elements do all the legwork then? I'm reluctant to make such big > > > > > > changes without a solid reconsideration of the whole design... I'm > > > > > > working on the Trie tests now, and should be able to commit my stuff > > > > > > later today. There's not *that* much changes to the stuff you changed, > > > > > > but it still might be tricky... > > > > > > > > > > > > I guess my main concern about dumping the DB is what avenues of > > > > > > expansion it closes off to us--right now it seems it'd be pretty > > > > > > trivial to expand the DB stuff to do very complex queries ((this label > > > > > > and this label) or (this label and not that label) or earlier than > > > > > > this date) if we figure out how to expose this to the GUI. I guess the > > > > > > real questions are > > > > > > > > > > > > 1. How useful would this really be in the first place? > > > > > > 2. How difficult would this be to replicate without the DB, if we *do* > > > > > > want to do it? > > > > > > > > > > > > A four-fold speed-up is nothing to sneeze at, but I'd hate to think it > > > > > > comes at the cost of totally crippled functionality... > > > > > > > > > > > > -Maciek > > > > > > > > > > > > On 7/30/06, Jimmy Do <cri...@gm...> wrote: > > > > > > > Nope, not using the DB for queries. We weren't using very complex DB > > > > > > > queries anyway because we just kept asking the DB for all pictures. > > > > > > > Consequently, I don't think we really need QueryLanguage. But then > > > > > > > again, maybe you guys want to keep queries and QueryLanguage, so > > > > > > > perhaps I shouldn't merge my changes. But if you want the changes, I > > > > > > > guess I should merge into main first and then you merge your changes, > > > > > > > just to make sure I don't overwrite your changes. > > > > > > > > > > > > > > Jimmy > > > > > > > > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > > > > > But we're still using the DB for queries, right? Just not for the > > > > > > > > cache? I guess I should take a look at your changes. I'll try to do > > > > > > > > that today, maybe. Also, should I commit my changes, and have you > > > > > > > > merge the ScanFaster branch with that, or do you want to merge with > > > > > > > > the current trunk and then I can merge my changes with the new trunk? > > > > > > > > > > > > > > > > > > > > > > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > > > > > > > > Hey guys, > > > > > > > > > > > > > > > > > > So I think I've got the ScanFaster branch working pretty well. Please > > > > > > > > > check it out and see if you can break it. I'll merge it into the main > > > > > > > > > trunk if you guys approve. > > > > > > > > > > > > > > > > > > Major Changes: > > > > > > > > > * Faster scanning by storing all IPictureInfos directly into a > > > > > > > > > List<>. Previously, the database inserts during scans were slowing > > > > > > > > > things down a lot (about 4 times slower with database inserts). We now > > > > > > > > > have a separate thread that periodically syncs the current list of > > > > > > > > > IPictureInfos into a file by serializing it. SQLite is no longer used > > > > > > > > > for storing the cache because syncing the pictures to an SQLite DB > > > > > > > > > took about 20 *seconds* for only 60 pictures. With serialization, it > > > > > > > > > took less than 1 second to sync almost 1000 pictures. > > > > > > > > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > > > > > > > > to 3 minutes. > > > > > > > > > > > > > > > > > > * New design also results in quicker discovery of missing files upon > > > > > > > > > startup. For example, close the app, remove all pictures, and start > > > > > > > > > the app again. Before, if you had a lot of pictures, they would slowly > > > > > > > > > disappear from the UI. Now, all removed pictures will disappear right > > > > > > > > > away. > > > > > > > > > > > > > > > > > > * Preliminary UI for showing scan progress. > > > > > > > > > > > > > > > > > > > > > > > > > > > Jimmy > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > > > _______________________________________________ > > > > > > > > > Snap-photo-devel mailing list > > > > > > > > > Sna...@li... > > > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > > _______________________________________________ > > > > > > > > Snap-photo-devel mailing list > > > > > > > > Sna...@li... > > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > _______________________________________________ > > > > > > > Snap-photo-devel mailing list > > > > > > > Sna...@li... > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > _______________________________________________ > > > > > > Snap-photo-devel mailing list > > > > > > Sna...@li... > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > _______________________________________________ > > > > Snap-photo-devel mailing list > > > > Sna...@li... > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of IT > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > opinions on IT & business topics through brief surveys -- and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Snap-photo-devel mailing list > > > Sna...@li... > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Snap-photo-devel mailing list > > Sna...@li... > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > |
|
From: Jimmy D. <cri...@gm...> - 2006-08-02 06:30:07
|
On 8/1/06, Maciej Sakrejda <m.s...@gm...> wrote: > > 2. SearchTree needs add/remove/change events, but the re-design > > doesn't emit any. I'll try to add these events back in somewhere. > > Ah. Well, are we still doing FS monitoring on watched folders at all, > then? Yup, we're still using FS monitoring, except the code for handling the add/deleted/changed notifications has been unified with the code for scanning. > This seems like a feature we want to keep. I don't know how much > this would play into the redesign, but we had talked about abstracting > away raw FS events into actually relevant SnapFSEvents or something > like that--to, among other things, resolve the exiv2 file update > (copy/delete/rename) nonsense. If this is something that you think > would dovetail nicely with your current redesign, I can jump on it now > (I'm still feeling massively exhausted after each day of work, but > I'll get used to it soon). I don't remember the details of SnapFSEvents, so I can't really say how easy it would be to implement with this new design. I did, however, just add PicAdded, PicRemoved, and PicChanged events (but no code actually uses those events right now) to PictureCatalog, so PictureCatalog might be a good place to look for implementing SnapFSEvents. In that same class, I've also added a private instance of SearchTree so that we can update the search tree as new files are scanned, added, removed, or updated. To take advantage of this SearchTree instance, PictureCatalog now has a Search() method for performing searches. Great job with the SearchTree, Maciek. I just gave it a try and it appears to be blazingly fast. I've gotten search to work now, but you might want to take a look at everything to make sure I didn't do anything crazy during my merge. Jimmy Otheriwse, if that part can be just as > easily done independently, I'll get on it in a week or two. > > -Maciek > > > > > Hmm...actually, I think I'm going to commit with the non-functional > > search, and then try to fix it within the next couple days. My hard > > drive seems to be having some issues, so I'd feel safer knowing that > > all changes are in SVN. > > > > Jimmy > > > > > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > > Wow. No DB at all... Well, are you letting the IWhateverFilter > > > > > elements do all the legwork then? I'm reluctant to make such big > > > > > changes without a solid reconsideration of the whole design... I'm > > > > > working on the Trie tests now, and should be able to commit my stuff > > > > > later today. There's not *that* much changes to the stuff you changed, > > > > > but it still might be tricky... > > > > > > > > > > I guess my main concern about dumping the DB is what avenues of > > > > > expansion it closes off to us--right now it seems it'd be pretty > > > > > trivial to expand the DB stuff to do very complex queries ((this label > > > > > and this label) or (this label and not that label) or earlier than > > > > > this date) if we figure out how to expose this to the GUI. I guess the > > > > > real questions are > > > > > > > > > > 1. How useful would this really be in the first place? > > > > > 2. How difficult would this be to replicate without the DB, if we *do* > > > > > want to do it? > > > > > > > > > > A four-fold speed-up is nothing to sneeze at, but I'd hate to think it > > > > > comes at the cost of totally crippled functionality... > > > > > > > > > > -Maciek > > > > > > > > > > On 7/30/06, Jimmy Do <cri...@gm...> wrote: > > > > > > Nope, not using the DB for queries. We weren't using very complex DB > > > > > > queries anyway because we just kept asking the DB for all pictures. > > > > > > Consequently, I don't think we really need QueryLanguage. But then > > > > > > again, maybe you guys want to keep queries and QueryLanguage, so > > > > > > perhaps I shouldn't merge my changes. But if you want the changes, I > > > > > > guess I should merge into main first and then you merge your changes, > > > > > > just to make sure I don't overwrite your changes. > > > > > > > > > > > > Jimmy > > > > > > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > > > > But we're still using the DB for queries, right? Just not for the > > > > > > > cache? I guess I should take a look at your changes. I'll try to do > > > > > > > that today, maybe. Also, should I commit my changes, and have you > > > > > > > merge the ScanFaster branch with that, or do you want to merge with > > > > > > > the current trunk and then I can merge my changes with the new trunk? > > > > > > > > > > > > > > > > > > > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > > > > > > > Hey guys, > > > > > > > > > > > > > > > > So I think I've got the ScanFaster branch working pretty well. Please > > > > > > > > check it out and see if you can break it. I'll merge it into the main > > > > > > > > trunk if you guys approve. > > > > > > > > > > > > > > > > Major Changes: > > > > > > > > * Faster scanning by storing all IPictureInfos directly into a > > > > > > > > List<>. Previously, the database inserts during scans were slowing > > > > > > > > things down a lot (about 4 times slower with database inserts). We now > > > > > > > > have a separate thread that periodically syncs the current list of > > > > > > > > IPictureInfos into a file by serializing it. SQLite is no longer used > > > > > > > > for storing the cache because syncing the pictures to an SQLite DB > > > > > > > > took about 20 *seconds* for only 60 pictures. With serialization, it > > > > > > > > took less than 1 second to sync almost 1000 pictures. > > > > > > > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > > > > > > > to 3 minutes. > > > > > > > > > > > > > > > > * New design also results in quicker discovery of missing files upon > > > > > > > > startup. For example, close the app, remove all pictures, and start > > > > > > > > the app again. Before, if you had a lot of pictures, they would slowly > > > > > > > > disappear from the UI. Now, all removed pictures will disappear right > > > > > > > > away. > > > > > > > > > > > > > > > > * Preliminary UI for showing scan progress. > > > > > > > > > > > > > > > > > > > > > > > > Jimmy > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > > _______________________________________________ > > > > > > > > Snap-photo-devel mailing list > > > > > > > > Sna...@li... > > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > _______________________________________________ > > > > > > > Snap-photo-devel mailing list > > > > > > > Sna...@li... > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > _______________________________________________ > > > > > > Snap-photo-devel mailing list > > > > > > Sna...@li... > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > _______________________________________________ > > > > > Snap-photo-devel mailing list > > > > > Sna...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of IT > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > opinions on IT & business topics through brief surveys -- and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Snap-photo-devel mailing list > > > Sna...@li... > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Snap-photo-devel mailing list > > Sna...@li... > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Maciej S. <m.s...@gm...> - 2006-08-02 02:18:09
|
> 2. SearchTree needs add/remove/change events, but the re-design > doesn't emit any. I'll try to add these events back in somewhere. Ah. Well, are we still doing FS monitoring on watched folders at all, then? This seems like a feature we want to keep. I don't know how much this would play into the redesign, but we had talked about abstracting away raw FS events into actually relevant SnapFSEvents or something like that--to, among other things, resolve the exiv2 file update (copy/delete/rename) nonsense. If this is something that you think would dovetail nicely with your current redesign, I can jump on it now (I'm still feeling massively exhausted after each day of work, but I'll get used to it soon). Otheriwse, if that part can be just as easily done independently, I'll get on it in a week or two. -Maciek > > Hmm...actually, I think I'm going to commit with the non-functional > search, and then try to fix it within the next couple days. My hard > drive seems to be having some issues, so I'd feel safer knowing that > all changes are in SVN. > > Jimmy > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > Wow. No DB at all... Well, are you letting the IWhateverFilter > > > > elements do all the legwork then? I'm reluctant to make such big > > > > changes without a solid reconsideration of the whole design... I'm > > > > working on the Trie tests now, and should be able to commit my stuff > > > > later today. There's not *that* much changes to the stuff you changed, > > > > but it still might be tricky... > > > > > > > > I guess my main concern about dumping the DB is what avenues of > > > > expansion it closes off to us--right now it seems it'd be pretty > > > > trivial to expand the DB stuff to do very complex queries ((this label > > > > and this label) or (this label and not that label) or earlier than > > > > this date) if we figure out how to expose this to the GUI. I guess the > > > > real questions are > > > > > > > > 1. How useful would this really be in the first place? > > > > 2. How difficult would this be to replicate without the DB, if we *do* > > > > want to do it? > > > > > > > > A four-fold speed-up is nothing to sneeze at, but I'd hate to think it > > > > comes at the cost of totally crippled functionality... > > > > > > > > -Maciek > > > > > > > > On 7/30/06, Jimmy Do <cri...@gm...> wrote: > > > > > Nope, not using the DB for queries. We weren't using very complex DB > > > > > queries anyway because we just kept asking the DB for all pictures. > > > > > Consequently, I don't think we really need QueryLanguage. But then > > > > > again, maybe you guys want to keep queries and QueryLanguage, so > > > > > perhaps I shouldn't merge my changes. But if you want the changes, I > > > > > guess I should merge into main first and then you merge your changes, > > > > > just to make sure I don't overwrite your changes. > > > > > > > > > > Jimmy > > > > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > > > But we're still using the DB for queries, right? Just not for the > > > > > > cache? I guess I should take a look at your changes. I'll try to do > > > > > > that today, maybe. Also, should I commit my changes, and have you > > > > > > merge the ScanFaster branch with that, or do you want to merge with > > > > > > the current trunk and then I can merge my changes with the new trunk? > > > > > > > > > > > > > > > > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > > > > > > Hey guys, > > > > > > > > > > > > > > So I think I've got the ScanFaster branch working pretty well. Please > > > > > > > check it out and see if you can break it. I'll merge it into the main > > > > > > > trunk if you guys approve. > > > > > > > > > > > > > > Major Changes: > > > > > > > * Faster scanning by storing all IPictureInfos directly into a > > > > > > > List<>. Previously, the database inserts during scans were slowing > > > > > > > things down a lot (about 4 times slower with database inserts). We now > > > > > > > have a separate thread that periodically syncs the current list of > > > > > > > IPictureInfos into a file by serializing it. SQLite is no longer used > > > > > > > for storing the cache because syncing the pictures to an SQLite DB > > > > > > > took about 20 *seconds* for only 60 pictures. With serialization, it > > > > > > > took less than 1 second to sync almost 1000 pictures. > > > > > > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > > > > > > to 3 minutes. > > > > > > > > > > > > > > * New design also results in quicker discovery of missing files upon > > > > > > > startup. For example, close the app, remove all pictures, and start > > > > > > > the app again. Before, if you had a lot of pictures, they would slowly > > > > > > > disappear from the UI. Now, all removed pictures will disappear right > > > > > > > away. > > > > > > > > > > > > > > * Preliminary UI for showing scan progress. > > > > > > > > > > > > > > > > > > > > > Jimmy > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > _______________________________________________ > > > > > > > Snap-photo-devel mailing list > > > > > > > Sna...@li... > > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > _______________________________________________ > > > > > > Snap-photo-devel mailing list > > > > > > Sna...@li... > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > _______________________________________________ > > > > > Snap-photo-devel mailing list > > > > > Sna...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > _______________________________________________ > > > > Snap-photo-devel mailing list > > > > Sna...@li... > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Snap-photo-devel mailing list > > Sna...@li... > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Jimmy D. <cri...@gm...> - 2006-08-01 20:15:42
|
On 7/31/06, Maciej Sakrejda <m.s...@gm...> wrote: > FYI, now that I'm employed (started today), my output may drop for a > couple of weeks. It should pick back up once I acquire a laptop so I > can work on Snap (among other things) on BART. No problem. I've already merged everything, but I haven't committed yet because I have to get search to work. Right now, there are two problems: 1. SearchTree is in Mdbc, but in my re-design I've completely removed the need for Mdbc. Can probably be fixed by just moving SearchTree elsewhere. 2. SearchTree needs add/remove/change events, but the re-design doesn't emit any. I'll try to add these events back in somewhere. Hmm...actually, I think I'm going to commit with the non-functional search, and then try to fix it within the next couple days. My hard drive seems to be having some issues, so I'd feel safer knowing that all changes are in SVN. Jimmy > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > Wow. No DB at all... Well, are you letting the IWhateverFilter > > > elements do all the legwork then? I'm reluctant to make such big > > > changes without a solid reconsideration of the whole design... I'm > > > working on the Trie tests now, and should be able to commit my stuff > > > later today. There's not *that* much changes to the stuff you changed, > > > but it still might be tricky... > > > > > > I guess my main concern about dumping the DB is what avenues of > > > expansion it closes off to us--right now it seems it'd be pretty > > > trivial to expand the DB stuff to do very complex queries ((this label > > > and this label) or (this label and not that label) or earlier than > > > this date) if we figure out how to expose this to the GUI. I guess the > > > real questions are > > > > > > 1. How useful would this really be in the first place? > > > 2. How difficult would this be to replicate without the DB, if we *do* > > > want to do it? > > > > > > A four-fold speed-up is nothing to sneeze at, but I'd hate to think it > > > comes at the cost of totally crippled functionality... > > > > > > -Maciek > > > > > > On 7/30/06, Jimmy Do <cri...@gm...> wrote: > > > > Nope, not using the DB for queries. We weren't using very complex DB > > > > queries anyway because we just kept asking the DB for all pictures. > > > > Consequently, I don't think we really need QueryLanguage. But then > > > > again, maybe you guys want to keep queries and QueryLanguage, so > > > > perhaps I shouldn't merge my changes. But if you want the changes, I > > > > guess I should merge into main first and then you merge your changes, > > > > just to make sure I don't overwrite your changes. > > > > > > > > Jimmy > > > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > > But we're still using the DB for queries, right? Just not for the > > > > > cache? I guess I should take a look at your changes. I'll try to do > > > > > that today, maybe. Also, should I commit my changes, and have you > > > > > merge the ScanFaster branch with that, or do you want to merge with > > > > > the current trunk and then I can merge my changes with the new trunk? > > > > > > > > > > > > > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > > > > > Hey guys, > > > > > > > > > > > > So I think I've got the ScanFaster branch working pretty well. Please > > > > > > check it out and see if you can break it. I'll merge it into the main > > > > > > trunk if you guys approve. > > > > > > > > > > > > Major Changes: > > > > > > * Faster scanning by storing all IPictureInfos directly into a > > > > > > List<>. Previously, the database inserts during scans were slowing > > > > > > things down a lot (about 4 times slower with database inserts). We now > > > > > > have a separate thread that periodically syncs the current list of > > > > > > IPictureInfos into a file by serializing it. SQLite is no longer used > > > > > > for storing the cache because syncing the pictures to an SQLite DB > > > > > > took about 20 *seconds* for only 60 pictures. With serialization, it > > > > > > took less than 1 second to sync almost 1000 pictures. > > > > > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > > > > > to 3 minutes. > > > > > > > > > > > > * New design also results in quicker discovery of missing files upon > > > > > > startup. For example, close the app, remove all pictures, and start > > > > > > the app again. Before, if you had a lot of pictures, they would slowly > > > > > > disappear from the UI. Now, all removed pictures will disappear right > > > > > > away. > > > > > > > > > > > > * Preliminary UI for showing scan progress. > > > > > > > > > > > > > > > > > > Jimmy > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > _______________________________________________ > > > > > > Snap-photo-devel mailing list > > > > > > Sna...@li... > > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > _______________________________________________ > > > > > Snap-photo-devel mailing list > > > > > Sna...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > _______________________________________________ > > > > Snap-photo-devel mailing list > > > > Sna...@li... > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of IT > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > opinions on IT & business topics through brief surveys -- and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Snap-photo-devel mailing list > > > Sna...@li... > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Maciej S. <m.s...@gm...> - 2006-08-01 04:18:03
|
Ok, if you can merge it, go ahead and commit. If you have problems, you can either e-mail me, or just overwrite my changes and I can sort things out. FYI, now that I'm employed (started today), my output may drop for a couple of weeks. It should pick back up once I acquire a laptop so I can work on Snap (among other things) on BART. >If necessary, we can rewrite > PictureCatalog/LivePictureCatalog to use some kind of DB rather than a > List<>. Okay. I guess, ideally, we'd have an "IPictureCatalog" interface where we could swap out those components without much fuss, but let's see how this thing works all pulled together as is... *crosses fingers* -Maciek > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > Wow. No DB at all... Well, are you letting the IWhateverFilter > > elements do all the legwork then? I'm reluctant to make such big > > changes without a solid reconsideration of the whole design... I'm > > working on the Trie tests now, and should be able to commit my stuff > > later today. There's not *that* much changes to the stuff you changed, > > but it still might be tricky... > > > > I guess my main concern about dumping the DB is what avenues of > > expansion it closes off to us--right now it seems it'd be pretty > > trivial to expand the DB stuff to do very complex queries ((this label > > and this label) or (this label and not that label) or earlier than > > this date) if we figure out how to expose this to the GUI. I guess the > > real questions are > > > > 1. How useful would this really be in the first place? > > 2. How difficult would this be to replicate without the DB, if we *do* > > want to do it? > > > > A four-fold speed-up is nothing to sneeze at, but I'd hate to think it > > comes at the cost of totally crippled functionality... > > > > -Maciek > > > > On 7/30/06, Jimmy Do <cri...@gm...> wrote: > > > Nope, not using the DB for queries. We weren't using very complex DB > > > queries anyway because we just kept asking the DB for all pictures. > > > Consequently, I don't think we really need QueryLanguage. But then > > > again, maybe you guys want to keep queries and QueryLanguage, so > > > perhaps I shouldn't merge my changes. But if you want the changes, I > > > guess I should merge into main first and then you merge your changes, > > > just to make sure I don't overwrite your changes. > > > > > > Jimmy > > > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > > But we're still using the DB for queries, right? Just not for the > > > > cache? I guess I should take a look at your changes. I'll try to do > > > > that today, maybe. Also, should I commit my changes, and have you > > > > merge the ScanFaster branch with that, or do you want to merge with > > > > the current trunk and then I can merge my changes with the new trunk? > > > > > > > > > > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > > > > Hey guys, > > > > > > > > > > So I think I've got the ScanFaster branch working pretty well. Please > > > > > check it out and see if you can break it. I'll merge it into the main > > > > > trunk if you guys approve. > > > > > > > > > > Major Changes: > > > > > * Faster scanning by storing all IPictureInfos directly into a > > > > > List<>. Previously, the database inserts during scans were slowing > > > > > things down a lot (about 4 times slower with database inserts). We now > > > > > have a separate thread that periodically syncs the current list of > > > > > IPictureInfos into a file by serializing it. SQLite is no longer used > > > > > for storing the cache because syncing the pictures to an SQLite DB > > > > > took about 20 *seconds* for only 60 pictures. With serialization, it > > > > > took less than 1 second to sync almost 1000 pictures. > > > > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > > > > to 3 minutes. > > > > > > > > > > * New design also results in quicker discovery of missing files upon > > > > > startup. For example, close the app, remove all pictures, and start > > > > > the app again. Before, if you had a lot of pictures, they would slowly > > > > > disappear from the UI. Now, all removed pictures will disappear right > > > > > away. > > > > > > > > > > * Preliminary UI for showing scan progress. > > > > > > > > > > > > > > > Jimmy > > > > > > > > > > ------------------------------------------------------------------------- > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > _______________________________________________ > > > > > Snap-photo-devel mailing list > > > > > Sna...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > _______________________________________________ > > > > Snap-photo-devel mailing list > > > > Sna...@li... > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of IT > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > opinions on IT & business topics through brief surveys -- and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Snap-photo-devel mailing list > > > Sna...@li... > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Snap-photo-devel mailing list > > Sna...@li... > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > |
|
From: Jimmy D. <cri...@gm...> - 2006-07-31 06:51:11
|
Yeah, the IWhateverFilters have always done all the work of properly sorting and grouping the pictures. I think the fanciest SQL query we made was to grab pictures in chronological order. I agree that we should carefully consider getting rid of the DB, but it doesn't seem too hard to eventually swap in a DB in the future. Right now, most of the system just fetches a list of IPictureInfos from LivePictureCatalog. If necessary, we can rewrite PictureCatalog/LivePictureCatalog to use some kind of DB rather than a List<>. I'll check out your changes and see if I can merge my ScanFaster branch. Let me know if you want me to commit. Jimmy On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > Wow. No DB at all... Well, are you letting the IWhateverFilter > elements do all the legwork then? I'm reluctant to make such big > changes without a solid reconsideration of the whole design... I'm > working on the Trie tests now, and should be able to commit my stuff > later today. There's not *that* much changes to the stuff you changed, > but it still might be tricky... > > I guess my main concern about dumping the DB is what avenues of > expansion it closes off to us--right now it seems it'd be pretty > trivial to expand the DB stuff to do very complex queries ((this label > and this label) or (this label and not that label) or earlier than > this date) if we figure out how to expose this to the GUI. I guess the > real questions are > > 1. How useful would this really be in the first place? > 2. How difficult would this be to replicate without the DB, if we *do* > want to do it? > > A four-fold speed-up is nothing to sneeze at, but I'd hate to think it > comes at the cost of totally crippled functionality... > > -Maciek > > On 7/30/06, Jimmy Do <cri...@gm...> wrote: > > Nope, not using the DB for queries. We weren't using very complex DB > > queries anyway because we just kept asking the DB for all pictures. > > Consequently, I don't think we really need QueryLanguage. But then > > again, maybe you guys want to keep queries and QueryLanguage, so > > perhaps I shouldn't merge my changes. But if you want the changes, I > > guess I should merge into main first and then you merge your changes, > > just to make sure I don't overwrite your changes. > > > > Jimmy > > > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > > But we're still using the DB for queries, right? Just not for the > > > cache? I guess I should take a look at your changes. I'll try to do > > > that today, maybe. Also, should I commit my changes, and have you > > > merge the ScanFaster branch with that, or do you want to merge with > > > the current trunk and then I can merge my changes with the new trunk? > > > > > > > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > > > Hey guys, > > > > > > > > So I think I've got the ScanFaster branch working pretty well. Please > > > > check it out and see if you can break it. I'll merge it into the main > > > > trunk if you guys approve. > > > > > > > > Major Changes: > > > > * Faster scanning by storing all IPictureInfos directly into a > > > > List<>. Previously, the database inserts during scans were slowing > > > > things down a lot (about 4 times slower with database inserts). We now > > > > have a separate thread that periodically syncs the current list of > > > > IPictureInfos into a file by serializing it. SQLite is no longer used > > > > for storing the cache because syncing the pictures to an SQLite DB > > > > took about 20 *seconds* for only 60 pictures. With serialization, it > > > > took less than 1 second to sync almost 1000 pictures. > > > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > > > to 3 minutes. > > > > > > > > * New design also results in quicker discovery of missing files upon > > > > startup. For example, close the app, remove all pictures, and start > > > > the app again. Before, if you had a lot of pictures, they would slowly > > > > disappear from the UI. Now, all removed pictures will disappear right > > > > away. > > > > > > > > * Preliminary UI for showing scan progress. > > > > > > > > > > > > Jimmy > > > > > > > > ------------------------------------------------------------------------- > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > > opinions on IT & business topics through brief surveys -- and earn cash > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > _______________________________________________ > > > > Snap-photo-devel mailing list > > > > Sna...@li... > > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of IT > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > opinions on IT & business topics through brief surveys -- and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Snap-photo-devel mailing list > > > Sna...@li... > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Snap-photo-devel mailing list > > Sna...@li... > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Maciej S. <m.s...@gm...> - 2006-07-30 23:09:09
|
Well, so I committed my changes to the main branch, in case you want to check it out. The preferences class is really ugly right now (it totally breaks the gui if you resize it, for example), but it works. The main catch is that you need to restart the app once you update the watched folders for it to take effect. There was also no good place to put in a preferences button so I slammed one into the middle of MainWindow with the understanding that we would soon figure out a proper place for it. Maybe we could merge the About box into Preferences and have the About button be the Preferences button instead? SearchTree should actually merge rather transparently, since I conformed to the interface. The only problem *should* be Preferences and its interaction with FSNotify/ScanNotify--instead of hardcoding "My Pictures", you should ask the Preferences singleton for the list of watched folders. Loading/Saving should be more robust, but we can take care of that later... Let me know if there's any problems merging ScanFaster, and I'll see what I can see... -Maciek On 7/30/06, Jimmy Do <cri...@gm...> wrote: > Nope, not using the DB for queries. We weren't using very complex DB > queries anyway because we just kept asking the DB for all pictures. > Consequently, I don't think we really need QueryLanguage. But then > again, maybe you guys want to keep queries and QueryLanguage, so > perhaps I shouldn't merge my changes. But if you want the changes, I > guess I should merge into main first and then you merge your changes, > just to make sure I don't overwrite your changes. > > Jimmy > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > But we're still using the DB for queries, right? Just not for the > > cache? I guess I should take a look at your changes. I'll try to do > > that today, maybe. Also, should I commit my changes, and have you > > merge the ScanFaster branch with that, or do you want to merge with > > the current trunk and then I can merge my changes with the new trunk? > > > > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > > Hey guys, > > > > > > So I think I've got the ScanFaster branch working pretty well. Please > > > check it out and see if you can break it. I'll merge it into the main > > > trunk if you guys approve. > > > > > > Major Changes: > > > * Faster scanning by storing all IPictureInfos directly into a > > > List<>. Previously, the database inserts during scans were slowing > > > things down a lot (about 4 times slower with database inserts). We now > > > have a separate thread that periodically syncs the current list of > > > IPictureInfos into a file by serializing it. SQLite is no longer used > > > for storing the cache because syncing the pictures to an SQLite DB > > > took about 20 *seconds* for only 60 pictures. With serialization, it > > > took less than 1 second to sync almost 1000 pictures. > > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > > to 3 minutes. > > > > > > * New design also results in quicker discovery of missing files upon > > > startup. For example, close the app, remove all pictures, and start > > > the app again. Before, if you had a lot of pictures, they would slowly > > > disappear from the UI. Now, all removed pictures will disappear right > > > away. > > > > > > * Preliminary UI for showing scan progress. > > > > > > > > > Jimmy > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of IT > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > opinions on IT & business topics through brief surveys -- and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Snap-photo-devel mailing list > > > Sna...@li... > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Snap-photo-devel mailing list > > Sna...@li... > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Maciej S. <m.s...@gm...> - 2006-07-30 21:54:05
|
Wow. No DB at all... Well, are you letting the IWhateverFilter elements do all the legwork then? I'm reluctant to make such big changes without a solid reconsideration of the whole design... I'm working on the Trie tests now, and should be able to commit my stuff later today. There's not *that* much changes to the stuff you changed, but it still might be tricky... I guess my main concern about dumping the DB is what avenues of expansion it closes off to us--right now it seems it'd be pretty trivial to expand the DB stuff to do very complex queries ((this label and this label) or (this label and not that label) or earlier than this date) if we figure out how to expose this to the GUI. I guess the real questions are 1. How useful would this really be in the first place? 2. How difficult would this be to replicate without the DB, if we *do* want to do it? A four-fold speed-up is nothing to sneeze at, but I'd hate to think it comes at the cost of totally crippled functionality... -Maciek On 7/30/06, Jimmy Do <cri...@gm...> wrote: > Nope, not using the DB for queries. We weren't using very complex DB > queries anyway because we just kept asking the DB for all pictures. > Consequently, I don't think we really need QueryLanguage. But then > again, maybe you guys want to keep queries and QueryLanguage, so > perhaps I shouldn't merge my changes. But if you want the changes, I > guess I should merge into main first and then you merge your changes, > just to make sure I don't overwrite your changes. > > Jimmy > > On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > > But we're still using the DB for queries, right? Just not for the > > cache? I guess I should take a look at your changes. I'll try to do > > that today, maybe. Also, should I commit my changes, and have you > > merge the ScanFaster branch with that, or do you want to merge with > > the current trunk and then I can merge my changes with the new trunk? > > > > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > > Hey guys, > > > > > > So I think I've got the ScanFaster branch working pretty well. Please > > > check it out and see if you can break it. I'll merge it into the main > > > trunk if you guys approve. > > > > > > Major Changes: > > > * Faster scanning by storing all IPictureInfos directly into a > > > List<>. Previously, the database inserts during scans were slowing > > > things down a lot (about 4 times slower with database inserts). We now > > > have a separate thread that periodically syncs the current list of > > > IPictureInfos into a file by serializing it. SQLite is no longer used > > > for storing the cache because syncing the pictures to an SQLite DB > > > took about 20 *seconds* for only 60 pictures. With serialization, it > > > took less than 1 second to sync almost 1000 pictures. > > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > > to 3 minutes. > > > > > > * New design also results in quicker discovery of missing files upon > > > startup. For example, close the app, remove all pictures, and start > > > the app again. Before, if you had a lot of pictures, they would slowly > > > disappear from the UI. Now, all removed pictures will disappear right > > > away. > > > > > > * Preliminary UI for showing scan progress. > > > > > > > > > Jimmy > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of IT > > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > > opinions on IT & business topics through brief surveys -- and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Snap-photo-devel mailing list > > > Sna...@li... > > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Snap-photo-devel mailing list > > Sna...@li... > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Jimmy D. <cri...@gm...> - 2006-07-30 21:38:05
|
Nope, not using the DB for queries. We weren't using very complex DB queries anyway because we just kept asking the DB for all pictures. Consequently, I don't think we really need QueryLanguage. But then again, maybe you guys want to keep queries and QueryLanguage, so perhaps I shouldn't merge my changes. But if you want the changes, I guess I should merge into main first and then you merge your changes, just to make sure I don't overwrite your changes. Jimmy On 7/30/06, Maciej Sakrejda <m.s...@gm...> wrote: > But we're still using the DB for queries, right? Just not for the > cache? I guess I should take a look at your changes. I'll try to do > that today, maybe. Also, should I commit my changes, and have you > merge the ScanFaster branch with that, or do you want to merge with > the current trunk and then I can merge my changes with the new trunk? > > > On 7/29/06, Jimmy Do <cri...@gm...> wrote: > > Hey guys, > > > > So I think I've got the ScanFaster branch working pretty well. Please > > check it out and see if you can break it. I'll merge it into the main > > trunk if you guys approve. > > > > Major Changes: > > * Faster scanning by storing all IPictureInfos directly into a > > List<>. Previously, the database inserts during scans were slowing > > things down a lot (about 4 times slower with database inserts). We now > > have a separate thread that periodically syncs the current list of > > IPictureInfos into a file by serializing it. SQLite is no longer used > > for storing the cache because syncing the pictures to an SQLite DB > > took about 20 *seconds* for only 60 pictures. With serialization, it > > took less than 1 second to sync almost 1000 pictures. > > Overall, for around 1900 pictures, scanning went from about 8 minutes > > to 3 minutes. > > > > * New design also results in quicker discovery of missing files upon > > startup. For example, close the app, remove all pictures, and start > > the app again. Before, if you had a lot of pictures, they would slowly > > disappear from the UI. Now, all removed pictures will disappear right > > away. > > > > * Preliminary UI for showing scan progress. > > > > > > Jimmy > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Snap-photo-devel mailing list > > Sna...@li... > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Maciej S. <m.s...@gm...> - 2006-07-30 20:54:32
|
But we're still using the DB for queries, right? Just not for the cache? I guess I should take a look at your changes. I'll try to do that today, maybe. Also, should I commit my changes, and have you merge the ScanFaster branch with that, or do you want to merge with the current trunk and then I can merge my changes with the new trunk? On 7/29/06, Jimmy Do <cri...@gm...> wrote: > Hey guys, > > So I think I've got the ScanFaster branch working pretty well. Please > check it out and see if you can break it. I'll merge it into the main > trunk if you guys approve. > > Major Changes: > * Faster scanning by storing all IPictureInfos directly into a > List<>. Previously, the database inserts during scans were slowing > things down a lot (about 4 times slower with database inserts). We now > have a separate thread that periodically syncs the current list of > IPictureInfos into a file by serializing it. SQLite is no longer used > for storing the cache because syncing the pictures to an SQLite DB > took about 20 *seconds* for only 60 pictures. With serialization, it > took less than 1 second to sync almost 1000 pictures. > Overall, for around 1900 pictures, scanning went from about 8 minutes > to 3 minutes. > > * New design also results in quicker discovery of missing files upon > startup. For example, close the app, remove all pictures, and start > the app again. Before, if you had a lot of pictures, they would slowly > disappear from the UI. Now, all removed pictures will disappear right > away. > > * Preliminary UI for showing scan progress. > > > Jimmy > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Jimmy D. <cri...@gm...> - 2006-07-30 05:39:19
|
Hey guys, So I think I've got the ScanFaster branch working pretty well. Please check it out and see if you can break it. I'll merge it into the main trunk if you guys approve. Major Changes: * Faster scanning by storing all IPictureInfos directly into a List<>. Previously, the database inserts during scans were slowing things down a lot (about 4 times slower with database inserts). We now have a separate thread that periodically syncs the current list of IPictureInfos into a file by serializing it. SQLite is no longer used for storing the cache because syncing the pictures to an SQLite DB took about 20 *seconds* for only 60 pictures. With serialization, it took less than 1 second to sync almost 1000 pictures. Overall, for around 1900 pictures, scanning went from about 8 minutes to 3 minutes. * New design also results in quicker discovery of missing files upon startup. For example, close the app, remove all pictures, and start the app again. Before, if you had a lot of pictures, they would slowly disappear from the UI. Now, all removed pictures will disappear right away. * Preliminary UI for showing scan progress. Jimmy |
|
From: Maciej S. <m.s...@gm...> - 2006-07-29 21:28:02
|
Well, I guess we have two options: 1. Rewrite any code that depends on ordering not to depend on ordering. 2. Rewrite Set<T> to implement IList<T>. It sounds kind of crazy, but if Set<T> uses two data structures, a List<T> to hold the actual T items, and a Dictionary<T,int> to hold a mapping from the T items to positions in the list, we should be able to get it to work without too big a performance hit. Of course, this raises memory requirements for the Set<T> by at least 50%. Although if we're really concerned about memory usage, we should not use a Dictionary<T,bool> internally but rewrite Set<T> to do its own hashing... I'll give Option 1 a shot first, once I commit the Trie/SearchTree business. If that doesn't pan out, we can consider Option 2. -Maciek On 7/29/06, Jimmy Do <cri...@gm...> wrote: > I like the idea, but I'm hesitant because some of the filtering or > grouping code does rely on ordering so that the pictures show up in > the right order in the thumbnail view. But it would be nice to use > Set<T>, so maybe you can give it a try and see if it can be done? > > Jimmy > > On 7/28/06, Maciej Sakrejda <m.s...@gm...> wrote: > > So we use List<T>s (or at least IList<T>s) all over the place, where > > we are implicitly ignoring the occurence of duplicates. Should we > > maybe move this all to Set<T>s? I've added a lot of stuff to the > > Set<T> class, and have tested it thoroughly. Maybe we should move to > > ISet<T>s instead, in case we ever feel like implementing a Set<T>-like > > class in another way. Either way, though, it seems like it's a bad > > idea to use IList<T>s where we really mean "IList<T>, but with no > > duplicates". I guess the major catch is that since Set<T> is > > implemented on top of a dictionary, and not a list, we lose the > > ordered property. I don't know if any code actually depended on this. > > Set<T> does implement ICollection (among other things), so it should > > be pretty easy to work with, but I thought I'd check with you guys in > > case there is a good reason to use IList<T> in some place (mostly I'm > > talking about passing around IList<T>s of IPictureInfos for filtering > > purposes)... > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Snap-photo-devel mailing list > > Sna...@li... > > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Jimmy D. <cri...@gm...> - 2006-07-29 19:09:47
|
I like the idea, but I'm hesitant because some of the filtering or grouping code does rely on ordering so that the pictures show up in the right order in the thumbnail view. But it would be nice to use Set<T>, so maybe you can give it a try and see if it can be done? Jimmy On 7/28/06, Maciej Sakrejda <m.s...@gm...> wrote: > So we use List<T>s (or at least IList<T>s) all over the place, where > we are implicitly ignoring the occurence of duplicates. Should we > maybe move this all to Set<T>s? I've added a lot of stuff to the > Set<T> class, and have tested it thoroughly. Maybe we should move to > ISet<T>s instead, in case we ever feel like implementing a Set<T>-like > class in another way. Either way, though, it seems like it's a bad > idea to use IList<T>s where we really mean "IList<T>, but with no > duplicates". I guess the major catch is that since Set<T> is > implemented on top of a dictionary, and not a list, we lose the > ordered property. I don't know if any code actually depended on this. > Set<T> does implement ICollection (among other things), so it should > be pretty easy to work with, but I thought I'd check with you guys in > case there is a good reason to use IList<T> in some place (mostly I'm > talking about passing around IList<T>s of IPictureInfos for filtering > purposes)... > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Maciej S. <m.s...@gm...> - 2006-07-28 23:47:48
|
So we use List<T>s (or at least IList<T>s) all over the place, where we are implicitly ignoring the occurence of duplicates. Should we maybe move this all to Set<T>s? I've added a lot of stuff to the Set<T> class, and have tested it thoroughly. Maybe we should move to ISet<T>s instead, in case we ever feel like implementing a Set<T>-like class in another way. Either way, though, it seems like it's a bad idea to use IList<T>s where we really mean "IList<T>, but with no duplicates". I guess the major catch is that since Set<T> is implemented on top of a dictionary, and not a list, we lose the ordered property. I don't know if any code actually depended on this. Set<T> does implement ICollection (among other things), so it should be pretty easy to work with, but I thought I'd check with you guys in case there is a good reason to use IList<T> in some place (mostly I'm talking about passing around IList<T>s of IPictureInfos for filtering purposes)... |
|
From: Jimmy D. <cri...@gm...> - 2006-07-27 14:20:11
|
On 7/25/06, Maciej Sakrejda <m.s...@gm...> wrote: > Anyway, this seems to be working okay, except that I get an Exception > in DateTakenSelector.cs on line 198 after removing a watched folder > (right now, the app has to be restarted to get this to take effect, > and it happens as the app is removing the images from the folders that > have been removed). Hmm...I'll take a look at that. > I take it I should commit this as a branch? I'll read up on branching > (I've never really worked with branches in source control) and do > that. I guess a branch is a good idea. But if the app is fairly stable, then I don't see much harm with committing into the main trunk. If anything, committing into the main trunk will make it easier for me to check out your changes. :) I'm on dialup and checking out an entirely new branch could take a while. > Once this thing is resolved on at least a basic level, I'll move to > moving FS events to use the blocking queue. You might want to hold off on this for a little bit. I've been experimenting with picture scanning and I think I've come up with a faster (and so far, cleaner) way of scanning for pictures. The changes involve ripping out most of Mdbc and the file-change-related parts of StorageHandler and replacing them with 2 new classes (one of which is only a small subclass of the other). It's still in the experimental phase, so I'm not sure if it's anything I'd want to commit. Jimmy > > -Maciek > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Maciej S. <m.s...@gm...> - 2006-07-26 02:12:13
|
> Hmm....funny thing...I just tried it and it *seems* like the top-left > visible thumbnail already loads first. Now I'm confused. Maybe we > wanted to fix something else? I guess we'll figure it out eventually. Well... Right you are. I dunno. I guess it's something we addressed already. We can come back to it if need be... Anyway, I've got a basic Preferences GUI right now. Right now it does nothing but let you select watched folders, and it does not do it very well. I need to learn more about Forms. I have two chapters in my C# book to get through, and then optionally another 1k page book from A Press. Maybe after doing some of that I'll have a better idea of what's going on--right now my GUI programming is a framework of guesses and kludges. Anyway, this seems to be working okay, except that I get an Exception in DateTakenSelector.cs on line 198 after removing a watched folder (right now, the app has to be restarted to get this to take effect, and it happens as the app is removing the images from the folders that have been removed). I take it I should commit this as a branch? I'll read up on branching (I've never really worked with branches in source control) and do that. Once this thing is resolved on at least a basic level, I'll move to moving FS events to use the blocking queue. -Maciek |
|
From: Jimmy D. <cri...@gm...> - 2006-07-25 21:25:58
|
On 7/25/06, Maciej Sakrejda <m.s...@gm...> wrote: > Distance from upper left corner of the upper-left-most visible thumbnail > + ((y-coordinate < upper left corner of the upper-left-most visible > thumbnail) ? (height of viewing area) : 0) > > This should load all the thumbnails on the screen first, top to > bottom, and then everything else. Well, I suppose we should really > weigh the y coordinate more heavily, since we want things to load top > to bottom, left to right, and not in some sort of radial spray from > the upper left corner. Hmm....funny thing...I just tried it and it *seems* like the top-left visible thumbnail already loads first. Now I'm confused. Maybe we wanted to fix something else? I guess we'll figure it out eventually. :) Jimmy > -Maciek > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Snap-photo-devel mailing list > Sna...@li... > https://lists.sourceforge.net/lists/listinfo/snap-photo-devel > |
|
From: Maciej S. <m.s...@gm...> - 2006-07-25 15:30:40
|
Sorry about the delayed reply--this got sent to the Berkeley address for some reason. I changed it in my SF settings, and one message to snap-photo-devel went through okay, but for some reason, this one didn't... Anyway... >Using the center might not be a good idea because I think it could be >disorienting to see thumbnails in the center load first. In the early >prototype, thumbnails loaded from the bottom up (because I didn't >think direction mattered), but I found that it did seem a little >weird. I think most people expect thumbnails to load from top to >bottom and left to right. That's a good point, although we may want to optimize so that we're not loading off-screen thumbnails until all on-screen thumbnails are loaded. Maybe the priority (assuming lowest is best) should be something like the following Distance from upper left corner of the upper-left-most visible thumbnail + ((y-coordinate < upper left corner of the upper-left-most visible thumbnail) ? (height of viewing area) : 0) This should load all the thumbnails on the screen first, top to bottom, and then everything else. Well, I suppose we should really weigh the y coordinate more heavily, since we want things to load top to bottom, left to right, and not in some sort of radial spray from the upper left corner. >> 1. Directories to scan >We should definitely have this one. Okay, I've created a Preferences class, and a PreferencesGui class. The idea is that all the preference info will sit in the Singleton Preferences class, which can then be serialized to save the preferences. We can even include a static "default" Preferences instance member that we can Clone() if the user wants to revert to defaults. If any part of the program wants to interact with anything that might depend on preferences, it just asks the Preferences singleton. I've already done this for directories to scan (although I have not added any GUI--or really, any means--to change this from within the app yet) and it seems to be working. >> 2. Cache size? > >By cache size, do you mean the number of thumbnails we'll keep cached >(implying that we'll evict old thumbnails from memory)? If so, I don't >think we need caching exactly. We should just keep all thumbnails in >memory (never evict anything) and let virtual memory do its thing. If >someone has too many pictures and things slow down, then they can add >more RAM. The virtual memory system will probably be better at >predicting what thumbnails to keep in memory and what to page out. I'm >against the idea of explicitly setting cache sizes because a scheme >that doesn't automatically scale with more RAM seems kinda hacky. Just >an idea. Let me know if it sounds insane. Well, I was talking about the disk cache. Come to think of it, I have no idea what we're doing there, really. I'll take a look and comment some more once I know what I'm talking about. > 3. Language (if we get around to internationalization/localization) >Sounds good. But I was under the impression that the app would >automatically use the language of the host operating system? I guess >it would be good if we can also have a way to manually change the >language, but this should be low priority if the language can be >automatically adjusted. Yes on all counts. I was just throwing ideas out there. I guess ultimately, if we take it this far, we may want to have preferences for which type of thumbnail view widget the user wants, which type of main view widget, etc. Or something... >> By the way, do you read any .NET developer blogs? I've stumbled upon >> some googling things, and it looks like there might be some out there >> worth keeping up with... > >I've read some, but not on a regular basis. Any recommendations? Not really, no. I haven't found anything that's *really* caught my eye. If anything, Miguel de Icaza's Mono blog is an interesting read, although I still have barely looked at Mono. I noticed that if we're serious about cross-platform usage we should at some point abstract away the ShellLib stuff, but I don't think we have to worry about that quite yet. -Maciek |
|
From: Jimmy D. <cri...@gm...> - 2006-07-18 05:10:56
|
On 7/17/06, Maciej Sakrejda <m.s...@gm...> wrote: > > Good idea. Priority queue with the weight being the "distance" of a > > picture from the top-left-most visible thumbnail? > > Yeah, I think that makes sense. Heck, we could even do distance from > center of the viewpane of the thumbnail widget, since the user could > as likely scroll down as up. Using the center might not be a good idea because I think it could be disorienting to see thumbnails in the center load first. In the early prototype, thumbnails loaded from the bottom up (because I didn't think direction mattered), but I found that it did seem a little weird. I think most people expect thumbnails to load from top to bottom and left to right. > Oh, and any suggestions for what should go in the preferences? I > should get on this, 'cause there's not enough space on my main drive > for all our photos, so I either have to hardcode a different > directory, or be able to select what directory (or multiple > directories) I want from within the app. So... > > 1. Directories to scan We should definitely have this one. > 2. Cache size? By cache size, do you mean the number of thumbnails we'll keep cached (implying that we'll evict old thumbnails from memory)? If so, I don't think we need caching exactly. We should just keep all thumbnails in memory (never evict anything) and let virtual memory do its thing. If someone has too many pictures and things slow down, then they can add more RAM. The virtual memory system will probably be better at predicting what thumbnails to keep in memory and what to page out. I'm against the idea of explicitly setting cache sizes because a scheme that doesn't automatically scale with more RAM seems kinda hacky. Just an idea. Let me know if it sounds insane. > 3. Language (if we get around to internationalization/localization) Sounds good. But I was under the impression that the app would automatically use the language of the host operating system? I guess it would be good if we can also have a way to manually change the language, but this should be low priority if the language can be automatically adjusted. A way to change the scan folder seems to be the most crucial feature. Other than that, I can't think of anything else that we'd need. > By the way, do you read any .NET developer blogs? I've stumbled upon > some googling things, and it looks like there might be some out there > worth keeping up with... I've read some, but not on a regular basis. Any recommendations? Jimmy |
|
From: Maciej S. <m.s...@gm...> - 2006-07-17 18:09:03
|
> > 1. image cache not shared everywhere > > 1. We should probably develop the image cache into its own component, > > no? I guess in PictureStorage. How many distinct sizes of each image > > should we cache? I suppose that should be something that we can easily > > tweak... > Yeah, its own component in PictureStorage sounds good. For sizes, > maybe the standard power-of-two icon sizes would work: 64x64, 128x128, > 256x256, etc? That sounds reasonable, although for the most part, we won't be dealing with squares. I suppose we should do images that "fit into" those squares (i.e., longest side is at most 64, at most 128, etc.). I'm kind of surprised that there isn't some thumnail caching code out there, but then again, I haven't seen any apps that let you dynamically adjust thumnail size. > > 2. need smart thumbnail fetching from currently viewed location > > 2. Err, what was this one about again? > This one means that when the user scrolls the GUI, the picture > thumbnails of the *currently* viewable pictures should be fetched with > higher priority than the thumbnails that are not visible. Currently, I > think the GUI just fetches thumbnails in order from top to bottom, > regardless of where the user is currently looking. Hmm... a priority queue? Or simply two queues, one more important than the other... Is the drawing of off-screen thumbnails accomplished in an OnPaint method, or in some other way? ... Man, the last few weeks of freedom are slipping away fast. -Maciek |