Re: [Scid-users] Script to get PGN files

[Ben H. <ben...@bt...>]

On Wed, 2010-02-24 at 15:54 +0100, Alexander Wagner wrote: 
> pgeorges wrote:
> 
> Hi!
> 
> > Some web sites are commercial, and I think it is fair to
> > ask before using them in an automated manner
> 
> Maybe I was easy to missunderstand. I didn't mean "should
> not" but I said I think it would not be necessary. Still it
> is nice to ask anyway.

One thing that might be worth trying is giving site owners the option to
show an advert for their site every time you download from it (or
perhaps the first time only). I don't think any have any legal recourse,
but there may be some jurisdictions where this is not true. And of
course they can make it much harder in a practical sense.

> I could do it, the only thing I don't know about is how to
> handle ZIP in a simple transparent and portable way. I'd
> tend to call the commandline unzip that is freely available,
> but usually it's not there on a certain platform.

Presumably it wouldn't be a big problem to include one of the freely
redistributable unzip programs in the scid package for a certain
platform, or possibly a VB Script? Not ideal but they're small enough
that it shouldn't make much difference.

> >> IMHO it would even be better to set up a webservice
> >> somewhere that collects the games from the various sites
> >> and offers them for download via some
> >> webservice/soap-calls.  Then we could add a simple
> >> inferface to Scid that allows to select certain
> >> collections that get fetched to the local database.
> >> Something similar to RSS for chess games.
> >>
> > I agree. This would be really useful. But this is another
> > project ...
> 
> Sure. I did not mean the Scid project to do it. But if there
> is a willing and skilled web programer arround...

I did wonder about that, but it looks like it would be a pain for
whoever was responsible for keeping it up-to-date. Instead what about
doing something similar, but with screen-scraping instructions rather
than the actual files? E.g. something like:
<GameSource name="TWIC>
<URL>"http://www.chess.co.uk/twic/twic"</URL>
<Regex>"http:\/\/www\.chesscenter\.com\/twic\/zips\/twic\d+g
\.zip"</Regex>
<Type>ZIP</Type>
</GameSource>

It would mean having to do a bit more work in SCID, but should require a
lot less maintenance.

Ben Hague