podserver-devel Mailing List for PodServer
Brought to you by:
teeks99
You can subscribe to this list here.
2006 |
Jan
(10) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
---|
From: Foster, R. - P. <RF...@qu...> - 2006-01-13 17:56:17
|
Tom, If you want the "remote podserver" to parse the RSS feed, then it would almost certainly be better to have things as file based. I could be misunderstanding you completely, but it almost sounds as if the "remote podserver" would effectively become a mirror of items necessary to support the active episodes in a particular feed. That would then lead me to wonder about the propagation mechanism. How would a feed update on a remote podserver be trigged? From a master site? Should a direct request to a remote site trigger a request back to the master site to determine if it is up to date? On every occasion, or only periodically? If periodically, would it only happen when something is requested from the remote site, or would there need to be something like a cron job (which may not be available through many providers). What happens if the remote site doesn't see a request for several days? If triggered from the master site, what happens if the remote site is unavailable? What happens if the master site triggers an update, but then suffers from a system outage? Like you, I have not yet needed to deal with them, and understand that replication and clustering are complicated... but you should be aware that what you are proposing is effectively the same thing. Replication in particular is complicated because it provide solutions to questions like those above. Regards, Richard P.S. No, to the best of my knowledge database replication (when correctly configured) does not require either server to go offline. Database backups, on the other hand, often do (or at least require the database to be placed in a read-only state) to preserve database integrity. |
From: Tom K. <li...@te...> - 2006-01-13 17:28:14
|
As far as the Remote PodServer idea.... The hope was that it would be simpler to just update remote files than = to do database replication or clustering. Heck, if we wanted it could be a = simple application that just parsed the RSS feed (this would have to happen = before general release to the public) and grab the files it needs from there. = If we can keep all the junk (feeds, users, keywords, show notes, etc) off = the remote systems and just have them deal with the files, I think we'll = save ourselves some trouble. I've never done database replication or clustering, but I've heard that = its very difficult to setup and maintain. Does MySQL replication require = either server to go offline? Tom -----Original Message----- From: pod...@li... [mailto:pod...@li...] On Behalf Of = Foster, Richard - PAL Sent: Friday, January 13, 2006 11:15 AM To: Tom Kent; pod...@li... Subject: RE: [Podserver-devel] Does anyone have a database with the = schema (and or any PHP code to pull the details from it) yet? Tom, I agree that there should be no need for a manual FTP operation. It was certainly never my expectation that there should be. Uploading the file could (relatively) easily be accomplished from the same web page as the entry of the feed-related information. The original file name, and the name used after uploading may not (and probably wouldn't) be the same. Yes, if the files are available as "plain mp3" files out on a server (or server cluster), there is no requirement for your second script. (That's another reason for considering it, since it would mean less "code" to maintain). It would not break the "central reference" concept since information about the location of all the files would still need to be in the database so the RSS feeds could be generated. It does have the downside that if the files are moved manually (as opposed to through any script we provide) then the references would end up broken. Why reinvent the wheel with the "Remote Podserver"? MySQL (at least) can handle having copies of the database in multiple locations natively through database replication. (Note: I have not investigated if partial replication is possible, or if it is only possible to generate a full replica. I would be surprised if partial replication is not possible). Similarly there are already well designed and widely available clustering solutions for web servers. I have not investigated them, but I believe it should even be possible for listeners to "donate" server space effectively acting as a full or partial mirror of the primary site. Fr Roderick's server would effectively act as a manager, redirecting listeners to the appropriate mirror locations. Regards, Richard ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log = files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_idv37&alloc_id=16865&op=3Dick _______________________________________________ Podserver-devel mailing list Pod...@li... https://lists.sourceforge.net/lists/listinfo/podserver-devel |
From: Foster, R. - P. <RF...@qu...> - 2006-01-13 17:16:18
|
Tom, I agree that there should be no need for a manual FTP operation. It was certainly never my expectation that there should be. Uploading the file could (relatively) easily be accomplished from the same web page as the entry of the feed-related information. The original file name, and the name used after uploading may not (and probably wouldn't) be the same. Yes, if the files are available as "plain mp3" files out on a server (or server cluster), there is no requirement for your second script. (That's another reason for considering it, since it would mean less "code" to maintain). It would not break the "central reference" concept since information about the location of all the files would still need to be in the database so the RSS feeds could be generated. It does have the downside that if the files are moved manually (as opposed to through any script we provide) then the references would end up broken. Why reinvent the wheel with the "Remote Podserver"? MySQL (at least) can handle having copies of the database in multiple locations natively through database replication. (Note: I have not investigated if partial replication is possible, or if it is only possible to generate a full replica. I would be surprised if partial replication is not possible). Similarly there are already well designed and widely available clustering solutions for web servers. I have not investigated them, but I believe it should even be possible for listeners to "donate" server space effectively acting as a full or partial mirror of the primary site. Fr Roderick's server would effectively act as a manager, redirecting listeners to the appropriate mirror locations. Regards, Richard |
From: Tom K. <li...@te...> - 2006-01-13 16:48:17
|
Hmmm. How about if we add a field to the GenericMedia table for a URL. Then we could populate that OR the binary? (Plus add a MD5SUM field if that's the way we want to go). I was hoping to get away from the manual upload either way. No one should have to FTP into the server (except us to update these scripts). So this brings up an interesting point. If we have files on a server out there, does that pretty much get rid of my script that will use php to get you back a file when you give it an name or ID? I guess that's OK, I just thought it would be nice to have a central way for people to reference files on the server. As far as when feeds get popular, that's what the idea of the "Remote PodServer" was all about. It would be a mini copy of the database that could be put on any computer, and we'd do simple file transfers of the MP3s (from DB to DB) instead of a complete DB mirror, then the feed could alternately point to a different server based on their availability. I've thought about this a bunch and I really like it...but I don't want to worry about that stuff too soon. Tom -----Original Message----- From: Foster, Richard - PAL [mailto:RF...@qu...] Sent: Friday, January 13, 2006 9:31 AM To: Tom Kent Cc: pod...@li... Subject: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? Tom, [Side issue... You might want to change the settings on podserver-devel so replies go to the list instead of the original poster.] I was thinking that the first script would just generate the RSS feed. I would not bother importing the binary file into the database at this time, just have the database table include a reference to the file. This would allow the script to replace Fr Roderick's existing manual mechanism without any other modifications to his server. For that reason, I think the first script should be the one that generates the RSS feed from the database entries. The second script I think we should work on would be the one to populate the database when an episode is uploaded. Before deciding if the binary data should be in the database or as a standard URL reference I would like to do a bit more research. I would anticipate that serving the RSS feed will typically take a lot less in the way of resources than serving the episode files. When a podcast gets popular enough it may be desirable to have a single "feed" server and multiple "file" servers. The reason I think that configuration makes sense is that a database server cluster (again in my limited experience) will need more care and maintenance than a web server cluster. Obviously, if a feed gets *really* popular then a database server cluster could eventually become necessary too. Regards, Richard -----Original Message----- From: Tom Kent [mailto:li...@te...] Sent: Friday, January 13, 2006 09:54 To: Foster, Richard - PAL Subject: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? I wouldn't say completely (as to me being a "design it completely" kind of guy). I know that no matter how much stuff we think of, more will come. However, if I start writing without a great idea of what I want to do, I always end up making shortcuts that are a pain to deal with later...and it frustrates me to no end ;-) Anyway, I'd like to get more user input before we have a "first release", but if you want to start working on some demo scripts that'd be cool. What scripts will we need first? My ideas: - Something to generate rss So call rss.sqpn.com/feed.php?Feedname.xml - Something to get files rss.sqpn.com/file.php?FileName.mp3 or rss.sqpn.com/file.php?FileIDNumber - Something to upload show and information production.sqpn.com/upload.php What are your feelings towards having binary data in a database vs. filesystem? I'd still really like to get it in the DB, but I understand the problem of file corruption. I'm thinking we should go the DB route, and add a MD5SUM to the media table. Then we could run periodic checks of file integrity. Would that be satisfactory? What do you think? Tom |
From: Tom K. <li...@te...> - 2006-01-13 15:54:18
|
Here's the reply I didn't send correctly. -----Original Message----- From: Tom Kent [mailto:li...@te...]=20 Sent: Friday, January 13, 2006 8:54 AM To: 'Foster, Richard - PAL' Subject: RE: [Podserver-devel] Does anyone have a database with the = schema (and or any PHP code to pull the details from it) yet? I wouldn't say completely (as to me being a "design it completely" kind = of guy). I know that no matter how much stuff we think of, more will come. However, if I start writing without a great idea of what I want to do, I always end up making shortcuts that are a pain to deal with later...and = it frustrates me to no end ;-) Anyway, I'd like to get more user input before we have a "first = release", but if you want to start working on some demo scripts that'd be cool. What scripts will we need first? My ideas: - Something to generate rss So call rss.sqpn.com/feed.php?Feedname.xml - Something to get files=20 rss.sqpn.com/file.php?FileName.mp3 or rss.sqpn.com/file.php?FileIDNumber - Something to upload show and information production.sqpn.com/upload.php What are your feelings towards having binary data in a database vs. filesystem? I'd still really like to get it in the DB, but I understand = the problem of file corruption. I'm thinking we should go the DB route, and = add a MD5SUM to the media table. Then we could run periodic checks of file integrity. Would that be satisfactory? What do you think? Tom -----Original Message----- From: pod...@li... [mailto:pod...@li...] On Behalf Of = Foster, Richard - PAL Sent: Friday, January 13, 2006 8:29 AM To: Tom Kent Cc: pod...@li... Subject: RE: [Podserver-devel] Does anyone have a database with the = schema (and or any PHP code to pull the details from it) yet? Hi Tom, Ah! I take it from that you still subscribe to the "design it completely first" school of development. I'm more in line with Martin Fowler's views (see http://martinfowler.com/articles/newMethodology.html for more details if you're unfamiliar with them). I think we do have at least the first part of what is needed. A mechanism to generate the RSS feed, including a behind-the-scenes database from which the information is pulled. Once we have that, we can add the second part from your roadmap - the "upload" screens. In my experience most people find it *extremely* difficult to visualize what they are agreeing to unless they see a demonstration. I'm thinking that if we get a proof-of-concept script ready people will find it easier to see what we've missed! Regards, Richard -----Original Message----- From: Tom Kent [mailto:li...@te...]=20 Sent: Friday, January 13, 2006 09:20 To: Foster, Richard - PAL Subject: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? Nope, I was hoping to get some more consensus before we started. I'd really like to flush out all the ideas and make sure the design is up to snuff before we start writing code. It only gets harder to go back and change stuff once you start with code. Tom ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log = files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_idv37&alloc_id=16865&op=3Dick _______________________________________________ Podserver-devel mailing list Pod...@li... https://lists.sourceforge.net/lists/listinfo/podserver-devel |
From: Tom K. <li...@te...> - 2006-01-13 15:54:12
|
Here's the reply I didn't send correctly. -----Original Message----- From: Tom Kent [mailto:li...@te...] Sent: Friday, January 13, 2006 8:20 AM To: 'Foster, Richard - PAL' Subject: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? Nope, I was hoping to get some more consensus before we started. I'd really like to flush out all the ideas and make sure the design is up to snuff before we start writing code. It only gets harder to go back and change stuff once you start with code. Tom ________________________________________ From: pod...@li... [mailto:pod...@li...] On Behalf Of Foster, Richard - PAL Sent: Friday, January 13, 2006 8:10 AM To: pod...@li... Subject: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? Hi there, I'm just curious, has anyone actually started trying to implement the illustrated schema (and/or any scripts related to it)? Regards, Richard |
From: Foster, R. - P. <RF...@qu...> - 2006-01-13 15:31:42
|
Tom, [Side issue... You might want to change the settings on podserver-devel so replies go to the list instead of the original poster.]=20 I was thinking that the first script would just generate the RSS feed. I would not bother importing the binary file into the database at this time, just have the database table include a reference to the file. This would allow the script to replace Fr Roderick's existing manual mechanism without any other modifications to his server. For that reason, I think the first script should be the one that generates the RSS feed from the database entries. The second script I think we should work on would be the one to populate the database when an episode is uploaded. Before deciding if the binary data should be in the database or as a standard URL reference I would like to do a bit more research. I would anticipate that serving the RSS feed will typically take a lot less in the way of resources than serving the episode files. When a podcast gets popular enough it may be desirable to have a single "feed" server and multiple "file" servers. The reason I think that configuration makes sense is that a database server cluster (again in my limited experience) will need more care and maintenance than a web server cluster. Obviously, if a feed gets *really* popular then a database server cluster could eventually become necessary too. Regards, Richard -----Original Message----- From: Tom Kent [mailto:li...@te...]=20 Sent: Friday, January 13, 2006 09:54 To: Foster, Richard - PAL Subject: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? I wouldn't say completely (as to me being a "design it completely" kind of guy). I know that no matter how much stuff we think of, more will come. However, if I start writing without a great idea of what I want to do, I always end up making shortcuts that are a pain to deal with later...and it frustrates me to no end ;-) Anyway, I'd like to get more user input before we have a "first release", but if you want to start working on some demo scripts that'd be cool. What scripts will we need first? My ideas: - Something to generate rss So call rss.sqpn.com/feed.php?Feedname.xml - Something to get files rss.sqpn.com/file.php?FileName.mp3 or rss.sqpn.com/file.php?FileIDNumber - Something to upload show and information production.sqpn.com/upload.php What are your feelings towards having binary data in a database vs. filesystem? I'd still really like to get it in the DB, but I understand the problem of file corruption. I'm thinking we should go the DB route, and add a MD5SUM to the media table. Then we could run periodic checks of file integrity. Would that be satisfactory? What do you think? Tom |
From: Foster, R. - P. <RF...@qu...> - 2006-01-13 14:29:50
|
Hi Tom, Ah! I take it from that you still subscribe to the "design it completely first" school of development. I'm more in line with Martin Fowler's views (see http://martinfowler.com/articles/newMethodology.html for more details if you're unfamiliar with them). I think we do have at least the first part of what is needed. A mechanism to generate the RSS feed, including a behind-the-scenes database from which the information is pulled. Once we have that, we can add the second part from your roadmap - the "upload" screens. In my experience most people find it *extremely* difficult to visualize what they are agreeing to unless they see a demonstration. I'm thinking that if we get a proof-of-concept script ready people will find it easier to see what we've missed! Regards, Richard -----Original Message----- From: Tom Kent [mailto:li...@te...]=20 Sent: Friday, January 13, 2006 09:20 To: Foster, Richard - PAL Subject: RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet? Nope, I was hoping to get some more consensus before we started. I'd really like to flush out all the ideas and make sure the design is up to snuff before we start writing code. It only gets harder to go back and change stuff once you start with code. Tom |
From: Foster, R. - P. <RF...@qu...> - 2006-01-13 14:11:15
|
Hi there, I'm just curious, has anyone actually started trying to implement the illustrated schema (and/or any scripts related to it)? Regards, Richard |
From: Tom K. <te...@ya...> - 2006-01-06 22:04:41
|
Cool. I wish I had thought about this sooner, but we should probably continue this on the mailing list, so that others in the project can see what we've been chewing over. -----Original Message----- From: Foster, Richard - PAL [mailto:RF...@qu...] Sent: Friday, January 06, 2006 3:34 PM To: Tom Kent Subject: RE: Podserver Hi again Tom, Probably the best thing to do is just keep checking what we do against reality and make sure we don't let them get too far apart! It certainly seems like your initial model is a good one, but the only way anyone really finds out for certain that their schema is good is when they use it! :-) Yes, there is going to be a lot of binary data. Like I said, storing it directly in the database may not cause problems (especially not in a non-Microsoft world). The other option would be to store it as simple files. They may need to be renamed (since different people may want to upload files with the same filename), but that could be a simpler option. Yes, there is still potential for corruption, but my (admittedly limited) experience is that anything bad only happens to one set of binary data, rather than many. The bad thing about storing the binary data "by reference" in this manner is that it is much more difficult to do a complete backup. It sounds like we're on the same page as far as limiting the RSS feed is concerned. The archive could then be presented as a set of dynamically generated pages, using CSS to apply appropriate publication-specific "branding". Regards, Richard P.S. How did I hear about this project? Simple, by listening to Fr Roderick and being a member of the SQPN team (I'm rjfoster over there!) :-) -----Original Message----- From: Tom Kent [mailto:te...@ya...] Sent: Friday, January 06, 2006 16:16 To: Foster, Richard - PAL Subject: RE: Podserver Yeah, I does sound like you have more experience than me. I'd be more than open to any criticisms/revisions/completely new models that you may have :-) I was also planning MySQL for at least the first stage. I think that it will be easiest on everyone. Once there's free time, we could add functionality for other DBs. I didn't realize storing binary data this way would cause problems. Could it be alleviated with using an MD5 sum and checking periodically or something of that nature? How would you recommend we store the binary data....there's going to be A LOT of it. I was thinking that limiting the RSS feeds to N number of episodes would probably be the best way to go. Then we could have another (non-rss?) page that would list the archived versions. We could also put in the ability to delete an episode if there's something that people want to get rid of (like someone included copyrighted material that they got busted for). I don't want to make deleting stuff a typical occurrence though. I don't think things in the version control world will get too complicated early on. Hopefully converting to SVN when it comes down will be painless. Tom P.S. Where did you hear about this project? Did you just surf to it? |