Re: [Xmltv-devel] Re: "program holes" at zap2it

[Ed A. <ed...@me...>]

On Fri, 31 Jan 2003, Grant Taylor wrote:

>>>On an unrelated topic, would there be any call for a non-zap2it _na
>>>backend?

>Generate XML?  Surely the interface consists of using some core
>xmltv logic to build up program data and then call the one true
>"write" routine?

Yes, some grabbers work like this and it is the best way IMHO.  But
you can also write out XML by hand, as long as you make sure it is
valid.

>Robust in which way?  Against site changes, or in terms of being
>bugless in the absence of site change?

Against site changes, ideally - though this is more a practical thing,
making the script try to work around the kinds of changes which tend
to happen in practice.  Obviously you cannot foresee all the changes
which might happen, but if you know that some things do often happen,
the script should handle those.  tv_grab_na's gymnastics with changing
provider ids is a good example.

Not bugless, but hopefully not requring us to rush out a new xmltv
bugfix release every time the site changes.

>Yahoo offers a partial-day grid, and per program detail pages. I'm
>not aware of any alternative presentations, although I haven't
>looked in a year.  The grid has almost(tm) enough information, but
>it is inadequate for start/end times,

Your grabber can leave out end times, no problem there.  Failing to
get start times is more serious - perhaps the grabber could resort to
doing extra page fetches just in the cases where this is needed for
start time?

>the titles are sometimes truncated,

Not great but probably a lesser evil than tons of page fetches.

Perhaps the grabber could even detect truncation and fetch the extra
page only in these cases?  I don't know whether this would be
possible.

>and I don't recall descriptions.

Likewise.  As I've mentioned on this list before, there could be some
way to run the grabber in two passes, first getting titles, filtering
on those, then putting the XML back in for another pass where it fills
in descriptions and stuff.  But for the time being, just 'fast' and
'slow' modes will do.  I think many users can live without
descriptions, strange as it sounds.

>On the bright side, program pages are shared across rebroadcasts.

Whether this is much of a saving depends on how repeat-laden the
channels you watch are :-).

>Well, I'll poke at xmltv core this weekend and see what I'd need to
>do besides fix the scraping and calling a few xmltv calls.

The checklist of things for writing a grabber is at
<http://sourceforge.net/mailarchive/message.php?msg_id=2344531>.

>To be honest, if I'm going to have to put much work into xmltv I'd
>rather make it do some sort of distributed caching so that we can
>avoid an us-vs-sites arms race.

But caching involves making copies of files, and so may break
copyright law.  OTOH ordinary web proxies don't usually get sued.  
IANAL and all that.  We could certainly cooperate with sites to cache
their files, if they want to do that.

-- 
Ed Avis <ed...@me...>