[Openpvr-devel] Re: [OpenPVR] Re: [Xmltv-devel] XMLTV Query API

[James O. <jf...@fu...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm probably more coherent now. I have put together a *proposed* overview 
that should better communicate what I am thinking, with a diagram and 
everything. You can view/edit it at:

http://www.funktronics.ca/openpvr/wiki/SystemOverview

On November 29, 2001 02:47 am, Jerry Veldhuis wrote:
>   James, your email made me think. I don't like it when
> that happens, now I'll have to share my views, its the
> only way I'll get to sleep tonight. Bare with me.
>
>    XMLTV certainly could use some more user friendly or integration
> friendly tools. I've also thought about this problem quite a bit
> in the past, although my interests were unrelated to pvr. This may
> change soon though I bought a WinTV card :)
>
> Although I find the XML very useful, it's rarely a good substitute
> for a real database.
>
> I see tv listing queries falling into one of two categories:
>   1. Tell me what's on these channels from hour X to hour Y on day Z
>   2. Find me all the programs within "this" time frame that have
>      these kinds of attributes.
>
> I see your application (pytv) as fitting into category 1.
> You can picture a dynamic schedule site (much like, but
> smaller than zap2it) as being in both cateogries.
>
> Certainly your openpvr project also falls into both of
> these as well.

I've been thinking about these comments all day. I have decided to separate 
PyTV from OpenPVR for the most part. Basically, PyTV will be for non-PVR 
applications such as standalone schedule viewers and the like, but will share 
some code with OpenPVR.


> Trying to implement either of these using the XMLTV dtd
> to defined the file format is wrong (IMHO).
>
> Some may see this as a weekness in XMLTV, but I see this as
> something that can be use to "with" XMLTV to achieve goals
> outside that of XMLTV. Let XMLTV do what it does best, get
> the listings in a consistant, reliable fasion.

If I'm reading that paragraph right, we're on the same page, for the most 
part.


> What is missing from XMLTV to support these types
> of queries ?
>
>  - nothing, unless you are worried about slow sequential,
>    possibly memory intensive answers.

I've been scraping small amounts of data (1 or 2 days worth) and storing it 
all in memory. For the amount of data, search speed has been just fine, even 
though I'm using the absolute slowest data structure for it: nested lists. I 
have two sides: "get it working, then profile" on regular computers, and 
"optimise as you go along" on processors like Z80s and PICs. :-)*


>  - dates are 'loosely based on ISO 8601' (xmltv.dtd), so
>    some amount of work is required to do EVERY date comparison.

It's probably best to store all dates in unix time while in memory.


>  - channels have no defined order, so you'll have to reorder
>    the query results (this is probably really minor)

No biggie. I haven't had any speed problems and I don't see *any* of these 
tools trying to handle massive amounts of data anytime soon.


>  - programmes have no defined order, they don't have to be in
>    channel order or date order. (XMLTV does have utilities that
>    sort listings though, but only date ordering, not channel or
>    otherwise)

I was doing that at first, but realised that it doesn't really make a 
difference

>
>  - no enumerations - want a list of categories ? travers the
>    entire listing.

With such a small amount of data, the speed increase from using more advanced 
data structures isn't really worth it... at least not yet.


> There are most certainly others that didn't come to mind. These
> are not necessarily "bugs" in XMLTV, XMLTV's job is to
> deliver the listings, not present them they way you want to use
> them.

That's all I ever wanted.


> There has been talk of expanding the set of tools that XMLTV
> comes with to fill some of these gaps. Like many other
> things in the software world, nobody has the perfect tool
> for every job, there's always a need for glue.
>
> Although not everyone sees the listings data as
> relational, I do. This most certainly comes from the fact
> that what I want isn't what you want, not exactly anyway.
>
> I've thought that for most applications (not all) you
> want the data in something like mysql. It's client/server,
> has interfaces for every language under the sun, and the
> client end could certainly convert the results into whatever
> the application requires. Without a lot of work I think you
> could even get the data into mysql optimized for the most
> common queries. I know of at least one XMLTV user who does
> just this, in support of his pvr project. (he can step forward
> and comment if he likes, I won't force him by mentioning names).

For PyTV, a DB is overkill, IMO, but OpenPVR will have to use one anyway for 
storing details of recorded programs, so if it's there, why not? It's a good 
idea.


> My two cents. I like many others have a lot of things I'd LIKE
> to do, and a minimal amount of time to do ONE of them. That said
> I'm not sure when I might get around to this project.
> Maybe a subproject of XMLTV called tvmysql ?

The core of such a thing is not really software, but suggested 
representations of XMLTV data for various purposes. That's mostly a 
make-it-up-as-you-go-along thing, IMO.

Thanks for the comments,

- -- 
James Oakley
jf...@fu...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8Bul4kCVJEWdfdCgRAsmDAKCG6cjQqIgP3Crpu6mYifdV2BXoSgCgsDOO
EfP5JrUZzbNEV15v92TV+ts=
=a2QL
-----END PGP SIGNATURE-----