Menu
▾
▴
fofredux-devel
|
From: Kevin <ke...@dr...> - 2005-12-12 18:21:33
|
The issue on where to stick the feed image url got me thinking... Q: Why do we need to store the image URL in the database at all? More generally, why do we need to store any of the extra RSS feed info in the DB? I'm not saying eliminate the feed table, just trim it down to what is absolutely necessary to maintain the relationships between the entities i= n the DB. For the following columns, what I am suggesting is use the magpie rss cache as the only place where the data is stored for the application.=20 Magpie does a great job with caching the parsed RSS/Atom data structure.=20 So, I don't think we will see much of a performance hit from doing it thi= s way. I've only listed the non-critical attributes for a feed. The interface will not brake horribly if these are not available. (ex: cache file missing and feed is down) * description * latitude * longitude * type * image Comments? --=20 Kevin |
|
From: Katie B. <ka...@ho...> - 2005-12-13 02:15:17
|
On Mon, Dec 12, 2005 at 11:21:29AM -0700, Kevin wrote: > I'm not saying eliminate the feed table, just trim it down to what is > absolutely necessary to maintain the relationships between the entities in > the DB. I say go for it. -- Katie Bechtold http://hoteldetective.org/ "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759 |
|
From: Kevin <ke...@dr...> - 2005-12-13 16:17:44
|
Katie Bechtold wrote: >On Mon, Dec 12, 2005 at 11:21:29AM -0700, Kevin wrote: > > >>I'm not saying eliminate the feed table, just trim it down to what is >>absolutely necessary to maintain the relationships between the entities in >>the DB. >> >> > >I say go for it. > > > It was easier then I thought. I had some time this morning to do it. Please review these changes: cvs diff -kk -r PRE_FEEDREFACTOR -r PRE_CR_FEEDREFACTOR -Kevin |
|
From: Andrew T. <ajt...@hi...> - 2005-12-14 13:53:21
|
Wow, that's a pretty big change. You don't forsee any other repurcussions of not storing this metadata in the database? I do see the benefits, in terms of not having to do updates of the feed, etc. Let the feed & cache be your storage mechanism. I did a quick test on my current DB and everything appeared to work well. I have yet to test an "upgrade" of a FoF v0.19 database. Also - what is the status of the RSS image Katie? Is this work complete, in progress for v0.2, or pushed back to v0.3? I wasn't sure if it changed. Thanks for the work. Andy On 12/13/05, Kevin <ke...@dr...> wrote: > Katie Bechtold wrote: > > >On Mon, Dec 12, 2005 at 11:21:29AM -0700, Kevin wrote: > > > > > >>I'm not saying eliminate the feed table, just trim it down to what is > >>absolutely necessary to maintain the relationships between the entities= in > >>the DB. > >> > >> > > > >I say go for it. > > > > > > > It was easier then I thought. I had some time this morning to do it. > > Please review these changes: > > cvs diff -kk -r PRE_FEEDREFACTOR -r PRE_CR_FEEDREFACTOR > > -Kevin > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log fi= les > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=3D7637&alloc_id=3D16865&op=3Dclick > _______________________________________________ > Fofredux-devel mailing list > Fof...@li... > https://lists.sourceforge.net/lists/listinfo/fofredux-devel > -- Andrew Turner ajt...@hi... 42.4266N x 83.4931W http://highearthorbit.com Northville, Michigan, USA |
|
From: Katie B. <ka...@ho...> - 2005-12-14 15:25:18
|
On Wed, Dec 14, 2005 at 08:53:16AM -0500, Andrew Turner wrote: > Also - what is the status of the RSS image Katie? Is this work > complete, in progress for v0.2, or pushed back to v0.3? I wasn't sure > if it changed. The RSS image stuff is complete within the store-everything-in-the-database paradigm that we're now questioning. So, if there will be a release where we're still storing everything in the database, I'd say include the RSS image code; otherwise, we should be getting the image URLs from the feed cache, as Kevin has suggested. -- Katie Bechtold http://hoteldetective.org/ |
|
From: Kevin <ke...@dr...> - 2005-12-14 15:23:43
|
Andrew Turner wrote: >Wow, that's a pretty big change. You don't forsee any other >repurcussions of not storing this metadata in the database? I do see >the benefits, in terms of not having to do updates of the feed, etc. >Let the feed & cache be your storage mechanism. > >I did a quick test on my current DB and everything appeared to work >well. I have yet to test an "upgrade" of a FoF v0.19 database. > >Also - what is the status of the RSS image Katie? Is this work >complete, in progress for v0.2, or pushed back to v0.3? I wasn't sure >if it changed. >Thanks for the work. >Andy > > > I haven't seen any problems with it either. Part of my testing was with an empty cache directory. The feed list simply shows "version unavailable" and "description unavailable" for feeds with no cache file. Same as it shows "never" in the last update column. For enriching items for view.php, I cache the rss info in memory for each feed, so it's not having to read in the cache file for each item.(although the performance wasn't that bad without the in-memory cache) As far as an upgrade from FoF, I don't see how it could be a problem. FoF has a description column on the feed table, it simply won't be used in FoFr and won't be populated for feeds added after the upgrade. The column doesn't have a "not null" restriction, so that shouldn't be an issue. I see one case where this could be a problem. On the feed edit page, the lat and long are editable fields. Under my new design, these attributes are read only and not able to be modified by the user. The problem with feed image is localized to one particular feed where they publish an image URL but do not allow foreign referrals. We could either: 1) do nothing. 2) cache the feed image locally. #2 can be a deferred to a future release, if we want to do it at all. --Kevin |
|
From: Andrew T. <ajt...@hi...> - 2005-12-14 16:18:54
|
I made the lat/lon fields editable since many feeds don't include their location, but it is sometimes discernable. For example, my weatherunderground.com feed does not have a location information, but I *know* where it's located. Perhaps just adding a test that if there is no lat/lon in the feed, check the db for a 'stored' location. I'm already adding functionality in a local version for displaying all this location stuff on a map. Also, Kevin, what did you do with the RSS image stored in the DB when you refactored how feeds are stored? Andy On 12/14/05, Kevin <ke...@dr...> wrote: > Andrew Turner wrote: > > >Wow, that's a pretty big change. You don't forsee any other > >repurcussions of not storing this metadata in the database? I do see > >the benefits, in terms of not having to do updates of the feed, etc. > >Let the feed & cache be your storage mechanism. > > > >I did a quick test on my current DB and everything appeared to work > >well. I have yet to test an "upgrade" of a FoF v0.19 database. > > > >Also - what is the status of the RSS image Katie? Is this work > >complete, in progress for v0.2, or pushed back to v0.3? I wasn't sure > >if it changed. > >Thanks for the work. > >Andy > > > > > > > I haven't seen any problems with it either. Part of my testing was with > an empty cache directory. The feed list simply shows "version > unavailable" and "description unavailable" for feeds with no cache > file. Same as it shows "never" in the last update column. For > enriching items for view.php, I cache the rss info in memory for each > feed, so it's not having to read in the cache file for each > item.(although the performance wasn't that bad without the in-memory cach= e) > > As far as an upgrade from FoF, I don't see how it could be a problem. > FoF has a description column on the feed table, it simply won't be used > in FoFr and won't be populated for feeds added after the upgrade. The > column doesn't have a "not null" restriction, so that shouldn't be an iss= ue. > > I see one case where this could be a problem. On the feed edit page, > the lat and long are editable fields. Under my new design, these > attributes are read only and not able to be modified by the user. > > > The problem with feed image is localized to one particular feed where > they publish an image URL but do not allow foreign referrals. We could > either: 1) do nothing. 2) cache the feed image locally. #2 can be a > deferred to a future release, if we want to do it at all. > > --Kevin > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log fi= les > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=3D7637&alloc_id=3D16865&op=3Dclick > _______________________________________________ > Fofredux-devel mailing list > Fof...@li... > https://lists.sourceforge.net/lists/listinfo/fofredux-devel > -- Andrew Turner ajt...@hi... 42.4266N x 83.4931W http://highearthorbit.com Northville, Michigan, USA |
|
From: Kevin <ke...@dr...> - 2005-12-14 19:06:24
|
> I made the lat/lon fields editable since many feeds don't include > their location, but it is sometimes discernable. For example, my > weatherunderground.com feed does not have a location information, but > I *know* where it's located. > > Perhaps just adding a test that if there is no lat/lon in the feed, > check the db for a 'stored' location. I'm already adding functionality > in a local version for displaying all this location stuff on a map. > > Also, Kevin, what did you do with the RSS image stored in the DB when > you refactored how feeds are stored? > > Andy So, lat and lon need to at least have fields in the db for users to enter/override the location info for a feed. Are these fields populated always or only when the user enters them? Feed image URL is pulled from the rss cache same as lat,lon, and description. I removed the column from the feeds table. --=20 Kevin |
|
From: Kevin <ke...@dr...> - 2005-12-14 23:03:47
|
I've backed out these changes until lat/lon handling can be sorted out. > >> I made the lat/lon fields editable since many feeds don't include >> their location, but it is sometimes discernable. For example, my >> weatherunderground.com feed does not have a location information, but >> I *know* where it's located. >> >> Perhaps just adding a test that if there is no lat/lon in the feed, >> check the db for a 'stored' location. I'm already adding functionality >> in a local version for displaying all this location stuff on a map. >> >> Also, Kevin, what did you do with the RSS image stored in the DB when >> you refactored how feeds are stored? >> >> Andy > > So, lat and lon need to at least have fields in the db for users to > enter/override the location info for a feed. Are these fields populate= d > always or only when the user enters them? > > Feed image URL is pulled from the rss cache same as lat,lon, and > description. I removed the column from the feeds table. --=20 Kevin |
|
From: Andrew T. <ajt...@hi...> - 2005-12-14 23:09:34
|
I think it's ok to leave the lat/lon handling readable from the rss, if it's available, but also allow the user to add a lat/lon to the db specifically if it exists. Make sense? Andy On 12/14/05, Kevin <ke...@dr...> wrote: > > I've backed out these changes until lat/lon handling can be sorted out. > > > > >> I made the lat/lon fields editable since many feeds don't include > >> their location, but it is sometimes discernable. For example, my > >> weatherunderground.com feed does not have a location information, but > >> I *know* where it's located. > >> > >> Perhaps just adding a test that if there is no lat/lon in the feed, > >> check the db for a 'stored' location. I'm already adding functionality > >> in a local version for displaying all this location stuff on a map. > >> > >> Also, Kevin, what did you do with the RSS image stored in the DB when > >> you refactored how feeds are stored? > >> > >> Andy > > > > So, lat and lon need to at least have fields in the db for users to > > enter/override the location info for a feed. Are these fields populate= d > > always or only when the user enters them? > > > > Feed image URL is pulled from the rss cache same as lat,lon, and > > description. I removed the column from the feeds table. > > -- > Kevin > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log fi= les > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_idv37&alloc_id=16865&opclick > _______________________________________________ > Fofredux-devel mailing list > Fof...@li... > https://lists.sourceforge.net/lists/listinfo/fofredux-devel > -- Andrew Turner ajt...@hi... 42.4266N x 83.4931W http://highearthorbit.com Northville, Michigan, USA |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X