Thread: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the d
Brought to you by:
teeks99
Menu
▾
▴
podserver-devel
|
From: Foster, R. - P. <RF...@qu...> - 2006-01-13 14:29:50
|
Hi Tom, Ah! I take it from that you still subscribe to the "design it completely first" school of development. I'm more in line with Martin Fowler's views (see http://martinfowler.com/articles/newMethodology.html for more details if you're unfamiliar with them). I think we do have at least the first part of what is needed. A mechanism to generate the RSS feed, including a behind-the-scenes database from which the information is pulled. Once we have that, we can add the second part from your roadmap - the "upload" screens. In my experience most people find it *extremely* difficult to visualize what they are agreeing to unless they see a demonstration. I'm thinking that if we get a proof-of-concept script ready people will find it easier to see what we've missed! Regards, Richard -----Original Message----- From: Tom Kent [mailto:li...@te...]=20 Sent: Friday, January 13, 2006 09:20 To: Foster, Richard - PAL Subject: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? Nope, I was hoping to get some more consensus before we started. I'd really like to flush out all the ideas and make sure the design is up to snuff before we start writing code. It only gets harder to go back and change stuff once you start with code. Tom |
|
From: Foster, R. - P. <RF...@qu...> - 2006-01-13 15:31:42
|
Tom, [Side issue... You might want to change the settings on podserver-devel so replies go to the list instead of the original poster.]=20 I was thinking that the first script would just generate the RSS feed. I would not bother importing the binary file into the database at this time, just have the database table include a reference to the file. This would allow the script to replace Fr Roderick's existing manual mechanism without any other modifications to his server. For that reason, I think the first script should be the one that generates the RSS feed from the database entries. The second script I think we should work on would be the one to populate the database when an episode is uploaded. Before deciding if the binary data should be in the database or as a standard URL reference I would like to do a bit more research. I would anticipate that serving the RSS feed will typically take a lot less in the way of resources than serving the episode files. When a podcast gets popular enough it may be desirable to have a single "feed" server and multiple "file" servers. The reason I think that configuration makes sense is that a database server cluster (again in my limited experience) will need more care and maintenance than a web server cluster. Obviously, if a feed gets *really* popular then a database server cluster could eventually become necessary too. Regards, Richard -----Original Message----- From: Tom Kent [mailto:li...@te...]=20 Sent: Friday, January 13, 2006 09:54 To: Foster, Richard - PAL Subject: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? I wouldn't say completely (as to me being a "design it completely" kind of guy). I know that no matter how much stuff we think of, more will come. However, if I start writing without a great idea of what I want to do, I always end up making shortcuts that are a pain to deal with later...and it frustrates me to no end ;-) Anyway, I'd like to get more user input before we have a "first release", but if you want to start working on some demo scripts that'd be cool. What scripts will we need first? My ideas: - Something to generate rss So call rss.sqpn.com/feed.php?Feedname.xml - Something to get files rss.sqpn.com/file.php?FileName.mp3 or rss.sqpn.com/file.php?FileIDNumber - Something to upload show and information production.sqpn.com/upload.php What are your feelings towards having binary data in a database vs. filesystem? I'd still really like to get it in the DB, but I understand the problem of file corruption. I'm thinking we should go the DB route, and add a MD5SUM to the media table. Then we could run periodic checks of file integrity. Would that be satisfactory? What do you think? Tom |
|
From: Tom K. <li...@te...> - 2006-01-13 16:48:17
|
Hmmm. How about if we add a field to the GenericMedia table for a URL. Then we could populate that OR the binary? (Plus add a MD5SUM field if that's the way we want to go). I was hoping to get away from the manual upload either way. No one should have to FTP into the server (except us to update these scripts). So this brings up an interesting point. If we have files on a server out there, does that pretty much get rid of my script that will use php to get you back a file when you give it an name or ID? I guess that's OK, I just thought it would be nice to have a central way for people to reference files on the server. As far as when feeds get popular, that's what the idea of the "Remote PodServer" was all about. It would be a mini copy of the database that could be put on any computer, and we'd do simple file transfers of the MP3s (from DB to DB) instead of a complete DB mirror, then the feed could alternately point to a different server based on their availability. I've thought about this a bunch and I really like it...but I don't want to worry about that stuff too soon. Tom -----Original Message----- From: Foster, Richard - PAL [mailto:RF...@qu...] Sent: Friday, January 13, 2006 9:31 AM To: Tom Kent Cc: pod...@li... Subject: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? Tom, [Side issue... You might want to change the settings on podserver-devel so replies go to the list instead of the original poster.] I was thinking that the first script would just generate the RSS feed. I would not bother importing the binary file into the database at this time, just have the database table include a reference to the file. This would allow the script to replace Fr Roderick's existing manual mechanism without any other modifications to his server. For that reason, I think the first script should be the one that generates the RSS feed from the database entries. The second script I think we should work on would be the one to populate the database when an episode is uploaded. Before deciding if the binary data should be in the database or as a standard URL reference I would like to do a bit more research. I would anticipate that serving the RSS feed will typically take a lot less in the way of resources than serving the episode files. When a podcast gets popular enough it may be desirable to have a single "feed" server and multiple "file" servers. The reason I think that configuration makes sense is that a database server cluster (again in my limited experience) will need more care and maintenance than a web server cluster. Obviously, if a feed gets *really* popular then a database server cluster could eventually become necessary too. Regards, Richard -----Original Message----- From: Tom Kent [mailto:li...@te...] Sent: Friday, January 13, 2006 09:54 To: Foster, Richard - PAL Subject: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? I wouldn't say completely (as to me being a "design it completely" kind of guy). I know that no matter how much stuff we think of, more will come. However, if I start writing without a great idea of what I want to do, I always end up making shortcuts that are a pain to deal with later...and it frustrates me to no end ;-) Anyway, I'd like to get more user input before we have a "first release", but if you want to start working on some demo scripts that'd be cool. What scripts will we need first? My ideas: - Something to generate rss So call rss.sqpn.com/feed.php?Feedname.xml - Something to get files rss.sqpn.com/file.php?FileName.mp3 or rss.sqpn.com/file.php?FileIDNumber - Something to upload show and information production.sqpn.com/upload.php What are your feelings towards having binary data in a database vs. filesystem? I'd still really like to get it in the DB, but I understand the problem of file corruption. I'm thinking we should go the DB route, and add a MD5SUM to the media table. Then we could run periodic checks of file integrity. Would that be satisfactory? What do you think? Tom |
|
From: Foster, R. - P. <RF...@qu...> - 2006-01-13 17:16:18
|
Tom, I agree that there should be no need for a manual FTP operation. It was certainly never my expectation that there should be. Uploading the file could (relatively) easily be accomplished from the same web page as the entry of the feed-related information. The original file name, and the name used after uploading may not (and probably wouldn't) be the same. Yes, if the files are available as "plain mp3" files out on a server (or server cluster), there is no requirement for your second script. (That's another reason for considering it, since it would mean less "code" to maintain). It would not break the "central reference" concept since information about the location of all the files would still need to be in the database so the RSS feeds could be generated. It does have the downside that if the files are moved manually (as opposed to through any script we provide) then the references would end up broken. Why reinvent the wheel with the "Remote Podserver"? MySQL (at least) can handle having copies of the database in multiple locations natively through database replication. (Note: I have not investigated if partial replication is possible, or if it is only possible to generate a full replica. I would be surprised if partial replication is not possible). Similarly there are already well designed and widely available clustering solutions for web servers. I have not investigated them, but I believe it should even be possible for listeners to "donate" server space effectively acting as a full or partial mirror of the primary site. Fr Roderick's server would effectively act as a manager, redirecting listeners to the appropriate mirror locations. Regards, Richard |
|
From: Tom K. <li...@te...> - 2006-01-13 17:28:14
|
As far as the Remote PodServer idea.... The hope was that it would be simpler to just update remote files than = to do database replication or clustering. Heck, if we wanted it could be a = simple application that just parsed the RSS feed (this would have to happen = before general release to the public) and grab the files it needs from there. = If we can keep all the junk (feeds, users, keywords, show notes, etc) off = the remote systems and just have them deal with the files, I think we'll = save ourselves some trouble. I've never done database replication or clustering, but I've heard that = its very difficult to setup and maintain. Does MySQL replication require = either server to go offline? Tom -----Original Message----- From: pod...@li... [mailto:pod...@li...] On Behalf Of = Foster, Richard - PAL Sent: Friday, January 13, 2006 11:15 AM To: Tom Kent; pod...@li... Subject: RE: [Podserver-devel] Does anyone have a database with the = schema (and or any PHP code to pull the details from it) yet? Tom, I agree that there should be no need for a manual FTP operation. It was certainly never my expectation that there should be. Uploading the file could (relatively) easily be accomplished from the same web page as the entry of the feed-related information. The original file name, and the name used after uploading may not (and probably wouldn't) be the same. Yes, if the files are available as "plain mp3" files out on a server (or server cluster), there is no requirement for your second script. (That's another reason for considering it, since it would mean less "code" to maintain). It would not break the "central reference" concept since information about the location of all the files would still need to be in the database so the RSS feeds could be generated. It does have the downside that if the files are moved manually (as opposed to through any script we provide) then the references would end up broken. Why reinvent the wheel with the "Remote Podserver"? MySQL (at least) can handle having copies of the database in multiple locations natively through database replication. (Note: I have not investigated if partial replication is possible, or if it is only possible to generate a full replica. I would be surprised if partial replication is not possible). Similarly there are already well designed and widely available clustering solutions for web servers. I have not investigated them, but I believe it should even be possible for listeners to "donate" server space effectively acting as a full or partial mirror of the primary site. Fr Roderick's server would effectively act as a manager, redirecting listeners to the appropriate mirror locations. Regards, Richard ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log = files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_idv37&alloc_id=16865&op=3Dick _______________________________________________ Podserver-devel mailing list Pod...@li... https://lists.sourceforge.net/lists/listinfo/podserver-devel |
|
From: Foster, R. - P. <RF...@qu...> - 2006-01-13 17:56:17
|
Tom, If you want the "remote podserver" to parse the RSS feed, then it would almost certainly be better to have things as file based. I could be misunderstanding you completely, but it almost sounds as if the "remote podserver" would effectively become a mirror of items necessary to support the active episodes in a particular feed. That would then lead me to wonder about the propagation mechanism. How would a feed update on a remote podserver be trigged? From a master site? Should a direct request to a remote site trigger a request back to the master site to determine if it is up to date? On every occasion, or only periodically? If periodically, would it only happen when something is requested from the remote site, or would there need to be something like a cron job (which may not be available through many providers). What happens if the remote site doesn't see a request for several days? If triggered from the master site, what happens if the remote site is unavailable? What happens if the master site triggers an update, but then suffers from a system outage? Like you, I have not yet needed to deal with them, and understand that replication and clustering are complicated... but you should be aware that what you are proposing is effectively the same thing. Replication in particular is complicated because it provide solutions to questions like those above. Regards, Richard P.S. No, to the best of my knowledge database replication (when correctly configured) does not require either server to go offline. Database backups, on the other hand, often do (or at least require the database to be placed in a read-only state) to preserve database integrity. |
Thanks for helping keep SourceForge clean.
X