podserver-devel

RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet?

[Foster, R. - P. <RF...@qu...>]

Hi Tom,

Ah! I take it from that you still subscribe to the "design it completely
first" school of development. I'm more in line with Martin Fowler's
views (see http://martinfowler.com/articles/newMethodology.html for more
details if you're unfamiliar with them).

I think we do have at least the first part of what is needed. A
mechanism to generate the RSS feed, including a behind-the-scenes
database from which the information is pulled. Once we have that, we can
add the second part from your roadmap - the "upload" screens.

In my experience most people find it *extremely* difficult to visualize
what they are agreeing to unless they see a demonstration. I'm thinking
that if we get a proof-of-concept script ready people will find it
easier to see what we've missed!

Regards,
Richard

-----Original Message-----
From: Tom Kent [mailto:li...@te...]=20
Sent: Friday, January 13, 2006 09:20
To: Foster, Richard - PAL
Subject: RE: [Podserver-devel] Does anyone have a database with the
schema (and or any PHP code to pull the details from it) yet?

Nope, I was hoping to get some more consensus before we started.  I'd
really like to flush out all the ideas and make sure the design is up to
snuff before we start writing code.  It only gets harder to go back and
change stuff once you start with code.
Tom

RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet?

[Foster, R. - P. <RF...@qu...>]

Tom,

[Side issue... You might want to change the settings on podserver-devel
so replies go to the list instead of the original poster.]=20

I was thinking that the first script would just generate the RSS feed. I
would not bother importing the binary file into the database at this
time, just have the database table include a reference to the file. This
would allow the script to replace Fr Roderick's existing manual
mechanism without any other modifications to his server.

For that reason, I think the first script should be the one that
generates the RSS feed from the database entries. The second script I
think we should work on would be the one to populate the database when
an episode is uploaded.

Before deciding if the binary data should be in the database or as a
standard URL reference I would like to do a bit more research. I would
anticipate that serving the RSS feed will typically take a lot less in
the way of resources than serving the episode files. When a podcast gets
popular enough it may be desirable to have a single "feed" server and
multiple "file" servers. The reason I think that configuration makes
sense is that a database server cluster (again in my limited experience)
will need more care and maintenance than a web server cluster.
Obviously, if a feed gets *really* popular then a database server
cluster could eventually become necessary too.

Regards,
Richard

-----Original Message-----
From: Tom Kent [mailto:li...@te...]=20
Sent: Friday, January 13, 2006 09:54
To: Foster, Richard - PAL
Subject: RE: [Podserver-devel] Does anyone have a database with the
schema (and or any PHP code to pull the details from it) yet?

I wouldn't say completely (as to me being a "design it completely" kind
of guy). I know that no matter how much stuff we think of, more will
come.
However, if I start writing without a great idea of what I want to do, I
always end up making shortcuts that are a pain to deal with later...and
it frustrates me to no end ;-)

Anyway, I'd like to get more user input before we have a "first
release", but if you want to start working on some demo scripts that'd
be cool.

What scripts will we need first?  My ideas:
- Something to generate rss
So call rss.sqpn.com/feed.php?Feedname.xml

- Something to get files
rss.sqpn.com/file.php?FileName.mp3
or
rss.sqpn.com/file.php?FileIDNumber

- Something to upload show and information
production.sqpn.com/upload.php

What are your feelings towards having binary data in a database vs.
filesystem?  I'd still really like to get it in the DB, but I understand
the problem of file corruption.  I'm thinking we should go the DB route,
and add a MD5SUM to the media table.  Then we could run periodic checks
of file integrity.  Would that be satisfactory?

What do you think?
Tom

RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet?

[Tom K. <li...@te...>]

Hmmm.

How about if we add a field to the GenericMedia table for a URL.  Then we
could populate that OR the binary?  (Plus add a MD5SUM field if that's the
way we want to go).

I was hoping to get away from the manual upload either way.  No one should
have to FTP into the server (except us to update these scripts).

So this brings up an interesting point.  If we have files on a server out
there, does that pretty much get rid of my script that will use php to get
you back a file when you give it an name or ID?  I guess that's OK, I just
thought it would be nice to have a central way for people to reference files
on the server.

As far as when feeds get popular, that's what the idea of the "Remote
PodServer" was all about.  It would be a mini copy of the database that
could be put on any computer, and we'd do simple file transfers of the MP3s
(from DB to DB) instead of a complete DB mirror, then the feed could
alternately point to a different server based on their availability.  I've
thought about this a bunch and I really like it...but I don't want to worry
about that stuff too soon.

Tom

-----Original Message-----
From: Foster, Richard - PAL [mailto:RF...@qu...] 
Sent: Friday, January 13, 2006 9:31 AM
To: Tom Kent
Cc: pod...@li...
Subject: RE: [Podserver-devel] Does anyone have a database with the schema
(and or any PHP code to pull the details from it) yet?

Tom,

[Side issue... You might want to change the settings on podserver-devel
so replies go to the list instead of the original poster.] 

I was thinking that the first script would just generate the RSS feed. I
would not bother importing the binary file into the database at this
time, just have the database table include a reference to the file. This
would allow the script to replace Fr Roderick's existing manual
mechanism without any other modifications to his server.

For that reason, I think the first script should be the one that
generates the RSS feed from the database entries. The second script I
think we should work on would be the one to populate the database when
an episode is uploaded.

Before deciding if the binary data should be in the database or as a
standard URL reference I would like to do a bit more research. I would
anticipate that serving the RSS feed will typically take a lot less in
the way of resources than serving the episode files. When a podcast gets
popular enough it may be desirable to have a single "feed" server and
multiple "file" servers. The reason I think that configuration makes
sense is that a database server cluster (again in my limited experience)
will need more care and maintenance than a web server cluster.
Obviously, if a feed gets *really* popular then a database server
cluster could eventually become necessary too.

Regards,
Richard

-----Original Message-----
From: Tom Kent [mailto:li...@te...] 
Sent: Friday, January 13, 2006 09:54
To: Foster, Richard - PAL
Subject: RE: [Podserver-devel] Does anyone have a database with the
schema (and or any PHP code to pull the details from it) yet?

I wouldn't say completely (as to me being a "design it completely" kind
of guy). I know that no matter how much stuff we think of, more will
come.
However, if I start writing without a great idea of what I want to do, I
always end up making shortcuts that are a pain to deal with later...and
it frustrates me to no end ;-)

Anyway, I'd like to get more user input before we have a "first
release", but if you want to start working on some demo scripts that'd
be cool.

What scripts will we need first?  My ideas:
- Something to generate rss
So call rss.sqpn.com/feed.php?Feedname.xml

- Something to get files
rss.sqpn.com/file.php?FileName.mp3
or
rss.sqpn.com/file.php?FileIDNumber

- Something to upload show and information
production.sqpn.com/upload.php

What are your feelings towards having binary data in a database vs.
filesystem?  I'd still really like to get it in the DB, but I understand
the problem of file corruption.  I'm thinking we should go the DB route,
and add a MD5SUM to the media table.  Then we could run periodic checks
of file integrity.  Would that be satisfactory?

What do you think?
Tom

RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet?

[Foster, R. - P. <RF...@qu...>]

Tom,

I agree that there should be no need for a manual FTP operation. It was
certainly never my expectation that there should be. Uploading the file
could (relatively) easily be accomplished from the same web page as the
entry of the feed-related information. The original file name, and the
name used after uploading may not (and probably wouldn't) be the same.

Yes, if the files are available as "plain mp3" files out on a server (or
server cluster), there is no requirement for your second script. (That's
another reason for considering it, since it would mean less "code" to
maintain). It would not break the "central reference" concept since
information about the location of all the files would still need to be
in the database so the RSS feeds could be generated. It does have the
downside that if the files are moved manually (as opposed to through any
script we provide) then the references would end up broken.

Why reinvent the wheel with the "Remote Podserver"? MySQL (at least) can
handle having copies of the database in multiple locations natively
through database replication. (Note: I have not investigated if partial
replication is possible, or if it is only possible to generate a full
replica. I would be surprised if partial replication is not possible).
Similarly there are already well designed and widely available
clustering solutions for web servers. I have not investigated them, but
I believe it should even be possible for listeners to "donate" server
space effectively acting as a full or partial mirror of the primary
site. Fr Roderick's server would effectively act as a manager,
redirecting listeners to the appropriate mirror locations.

Regards,
Richard

RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet?

[Tom K. <li...@te...>]

As far as the Remote PodServer idea....
The hope was that it would be simpler to just update remote files than =
to do
database replication or clustering.  Heck, if we wanted it could be a =
simple
application that just parsed the RSS feed (this would have to happen =
before
general release to the public) and grab the files it needs from there.  =
If
we can keep all the junk (feeds, users, keywords, show notes, etc) off =
the
remote systems and just have them deal with the files, I think we'll =
save
ourselves some trouble.

I've never done database replication or clustering, but I've heard that =
its
very difficult to setup and maintain.  Does MySQL replication require =
either
server to go offline?

Tom

-----Original Message-----
From: pod...@li...
[mailto:pod...@li...] On Behalf Of =
Foster,
Richard - PAL
Sent: Friday, January 13, 2006 11:15 AM
To: Tom Kent; pod...@li...
Subject: RE: [Podserver-devel] Does anyone have a database with the =
schema
(and or any PHP code to pull the details from it) yet?

Tom,

I agree that there should be no need for a manual FTP operation. It was
certainly never my expectation that there should be. Uploading the file
could (relatively) easily be accomplished from the same web page as the
entry of the feed-related information. The original file name, and the
name used after uploading may not (and probably wouldn't) be the same.

Yes, if the files are available as "plain mp3" files out on a server (or
server cluster), there is no requirement for your second script. (That's
another reason for considering it, since it would mean less "code" to
maintain). It would not break the "central reference" concept since
information about the location of all the files would still need to be
in the database so the RSS feeds could be generated. It does have the
downside that if the files are moved manually (as opposed to through any
script we provide) then the references would end up broken.

Why reinvent the wheel with the "Remote Podserver"? MySQL (at least) can
handle having copies of the database in multiple locations natively
through database replication. (Note: I have not investigated if partial
replication is possible, or if it is only possible to generate a full
replica. I would be surprised if partial replication is not possible).
Similarly there are already well designed and widely available
clustering solutions for web servers. I have not investigated them, but
I believe it should even be possible for listeners to "donate" server
space effectively acting as a full or partial mirror of the primary
site. Fr Roderick's server would effectively act as a manager,
redirecting listeners to the appropriate mirror locations.

Regards,
Richard


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log =
files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id=16865&op=3Dick
_______________________________________________
Podserver-devel mailing list
Pod...@li...
https://lists.sourceforge.net/lists/listinfo/podserver-devel

RE: [Podserver-devel] Does anyone have a database with the schema (and or any PHP code to pull the details from it) yet?

[Foster, R. - P. <RF...@qu...>]

Tom,

If you want the "remote podserver" to parse the RSS feed, then it would
almost certainly be better to have things as file based. I could be
misunderstanding you completely, but it almost sounds as if the "remote
podserver" would effectively become a mirror of items necessary to
support the active episodes in a particular feed.

That would then lead me to wonder about the propagation mechanism. How
would a feed update on a remote podserver be trigged? From a master
site? Should a direct request to a remote site trigger a request back to
the master site to determine if it is up to date? On every occasion, or
only periodically? If periodically, would it only happen when something
is requested from the remote site, or would there need to be something
like a cron job (which may not be available through many providers).
What happens if the remote site doesn't see a request for several days?
If triggered from the master site, what happens if the remote site is
unavailable? What happens if the master site triggers an update, but
then suffers from a system outage?

Like you, I have not yet needed to deal with them, and understand that
replication and clustering are complicated... but you should be aware
that what you are proposing is effectively the same thing. Replication
in particular is complicated because it provide solutions to questions
like those above.

Regards,
Richard

P.S. No, to the best of my knowledge database replication (when
correctly configured) does not require either server to go offline.
Database backups, on the other hand, often do (or at least require the
database to be placed in a read-only state) to preserve database
integrity.