Menu
▾
▴
xmltv-commit — XMLTV commit messages
You can subscribe to this list here.
| 2001 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(95) |
Dec
(62) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2002 |
Jan
(70) |
Feb
(137) |
Mar
(102) |
Apr
(50) |
May
(62) |
Jun
(6) |
Jul
|
Aug
(12) |
Sep
(146) |
Oct
(108) |
Nov
(49) |
Dec
(84) |
| 2003 |
Jan
(60) |
Feb
(126) |
Mar
(111) |
Apr
(170) |
May
(162) |
Jun
(87) |
Jul
(88) |
Aug
(51) |
Sep
(50) |
Oct
(24) |
Nov
(95) |
Dec
(62) |
| 2004 |
Jan
(93) |
Feb
(81) |
Mar
(106) |
Apr
(94) |
May
(109) |
Jun
(8) |
Jul
(22) |
Aug
(15) |
Sep
(82) |
Oct
(26) |
Nov
(44) |
Dec
(28) |
| 2005 |
Jan
(25) |
Feb
(17) |
Mar
(27) |
Apr
(7) |
May
(25) |
Jun
(22) |
Jul
(21) |
Aug
(22) |
Sep
(11) |
Oct
(15) |
Nov
(20) |
Dec
(42) |
| 2006 |
Jan
(200) |
Feb
(42) |
Mar
(56) |
Apr
(91) |
May
(37) |
Jun
(54) |
Jul
(33) |
Aug
(15) |
Sep
(26) |
Oct
(30) |
Nov
(71) |
Dec
(102) |
| 2007 |
Jan
(48) |
Feb
(40) |
Mar
(24) |
Apr
(25) |
May
(7) |
Jun
(14) |
Jul
(35) |
Aug
(32) |
Sep
(28) |
Oct
(55) |
Nov
(57) |
Dec
(14) |
| 2008 |
Jan
(40) |
Feb
(32) |
Mar
(25) |
Apr
(29) |
May
(16) |
Jun
(49) |
Jul
(26) |
Aug
(31) |
Sep
(7) |
Oct
(16) |
Nov
(11) |
Dec
(13) |
| 2009 |
Jan
(13) |
Feb
(30) |
Mar
(38) |
Apr
(8) |
May
(23) |
Jun
(19) |
Jul
(24) |
Aug
(46) |
Sep
(65) |
Oct
(30) |
Nov
(28) |
Dec
(37) |
| 2010 |
Jan
(31) |
Feb
(41) |
Mar
(43) |
Apr
(82) |
May
(39) |
Jun
(40) |
Jul
(31) |
Aug
(46) |
Sep
(104) |
Oct
(55) |
Nov
(67) |
Dec
(32) |
| 2011 |
Jan
(26) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2012 |
Jan
|
Feb
(17) |
Mar
(32) |
Apr
(34) |
May
(15) |
Jun
(50) |
Jul
(19) |
Aug
(8) |
Sep
(16) |
Oct
(18) |
Nov
(12) |
Dec
(3) |
| 2013 |
Jan
(7) |
Feb
(11) |
Mar
(11) |
Apr
(13) |
May
(17) |
Jun
(5) |
Jul
(4) |
Aug
(2) |
Sep
(4) |
Oct
(44) |
Nov
(14) |
Dec
(57) |
| 2014 |
Jan
(42) |
Feb
(11) |
Mar
(13) |
Apr
(33) |
May
(48) |
Jun
(22) |
Jul
(7) |
Aug
(19) |
Sep
(11) |
Oct
(34) |
Nov
(12) |
Dec
(18) |
| 2015 |
Jan
(17) |
Feb
(17) |
Mar
(15) |
Apr
(23) |
May
(2) |
Jun
(41) |
Jul
(31) |
Aug
(33) |
Sep
(6) |
Oct
(5) |
Nov
(8) |
Dec
(23) |
| 2016 |
Jan
(9) |
Feb
(6) |
Mar
(26) |
Apr
(2) |
May
(24) |
Jun
(20) |
Jul
(34) |
Aug
(4) |
Sep
(1) |
Oct
|
Nov
(2) |
Dec
(1) |
| 2017 |
Jan
(17) |
Feb
(6) |
Mar
(8) |
Apr
|
May
(1) |
Jun
(1) |
Jul
(4) |
Aug
(4) |
Sep
(14) |
Oct
(1) |
Nov
(8) |
Dec
|
|
From: Robert E. <rm...@us...> - 2017-01-24 00:38:30
|
Update of /cvsroot/xmltv/xmltv/grab/zz_sdjson In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv16047/zz_sdjson Log Message: Directory /cvsroot/xmltv/xmltv/grab/zz_sdjson added to the repository |
|
From: Jan S. <yu...@us...> - 2017-01-23 14:59:16
|
Update of /cvsroot/xmltv/xmltv/grab/eu_epgdata In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv21649/grab/eu_epgdata Modified Files: tv_grab_eu_epgdata Log Message: Tweak channel name order to put the more common name first. Index: tv_grab_eu_epgdata =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/eu_epgdata/tv_grab_eu_epgdata,v retrieving revision 1.44 retrieving revision 1.45 diff -C2 -d -r1.44 -r1.45 *** tv_grab_eu_epgdata 9 Oct 2014 20:24:06 -0000 1.44 --- tv_grab_eu_epgdata 23 Jan 2017 14:59:14 -0000 1.45 *************** *** 587,596 **** } my @names; - if ($channel->first_child('ch11')->text) { - push(@names, [ $channel->first_child('ch11')->text ]); - } if ($channel->first_child('ch0')->text) { push(@names, [ $channel->first_child('ch0')->text ]); } if ($channel->first_child('ch1')->text) { push(@names, [ $channel->first_child('ch1')->text ]); --- 587,596 ---- } my @names; if ($channel->first_child('ch0')->text) { push(@names, [ $channel->first_child('ch0')->text ]); } + if ($channel->first_child('ch11')->text) { + push(@names, [ $channel->first_child('ch11')->text ]); + } if ($channel->first_child('ch1')->text) { push(@names, [ $channel->first_child('ch1')->text ]); |
|
From: Geoff <bil...@us...> - 2017-01-22 22:06:10
|
Update of /cvsroot/xmltv/xmltv In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv13987 Modified Files: Makefile.PL Log Message: Add HTML::TreeBuilder to tv_grab_uk_tvguide Index: Makefile.PL =================================================================== RCS file: /cvsroot/xmltv/xmltv/Makefile.PL,v retrieving revision 1.465 retrieving revision 1.466 diff -C2 -d -r1.465 -r1.466 *** Makefile.PL 22 Jan 2017 21:07:34 -0000 1.465 --- Makefile.PL 22 Jan 2017 22:06:08 -0000 1.466 *************** *** 897,906 **** }, prereqs => { 'HTTP::Cache::Transparent' => 1.0, ! 'DateTime' => 0, ! 'Date::Parse' => 0, ! 'Encode' => 0, ! 'URI::Escape' => 0, ! 'File::Path' => 0, ! 'Data::Dumper' => 0, }, }, --- 897,907 ---- }, prereqs => { 'HTTP::Cache::Transparent' => 1.0, ! 'HTML::TreeBuilder' => 0, ! 'DateTime' => 0, ! 'Date::Parse' => 0, ! 'Encode' => 0, ! 'URI::Escape' => 0, ! 'File::Path' => 0, ! 'Data::Dumper' => 0, }, }, |
|
From: Geoff <bil...@us...> - 2017-01-22 22:01:28
|
Update of /cvsroot/xmltv/xmltv/grab/uk_tvguide In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv13720 Modified Files: tv_grab_uk_tvguide Log Message: Retry web page fetch 5 times before ignoring page (bugs #514 #518) Index: tv_grab_uk_tvguide =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/uk_tvguide/tv_grab_uk_tvguide,v retrieving revision 1.20 retrieving revision 1.21 diff -C2 -d -r1.20 -r1.21 *** tv_grab_uk_tvguide 1 Jan 2017 13:01:03 -0000 1.20 --- tv_grab_uk_tvguide 22 Jan 2017 22:01:26 -0000 1.21 *************** *** 23,27 **** use XMLTV::Configure::Writer; use XMLTV::Supplement 0.005065 qw/SetSupplementRoot GetSupplementDir GetSupplementLines/; ! use XMLTV::Get_nice qw/get_nice_tree/; use File::Path; --- 23,27 ---- use XMLTV::Configure::Writer; use XMLTV::Supplement 0.005065 qw/SetSupplementRoot GetSupplementDir GetSupplementLines/; ! use XMLTV::Get_nice 0.005065 qw/get_nice_tree/; use File::Path; *************** *** 32,35 **** --- 32,36 ---- use Encode; use URI::Escape; + use HTML::TreeBuilder; use HTTP::Cache::Transparent; *************** *** 200,204 **** # Fetch the page ! my $tree = XMLTV::Get_nice::get_nice_tree($url); # $tree->dump; exit; --- 201,206 ---- # Fetch the page ! # my $tree = XMLTV::Get_nice::get_nice_tree($url); ! my $tree = fetch_url($url); # $tree->dump; exit; *************** *** 271,275 **** # Fetch the page ! my $treedetail = XMLTV::Get_nice::get_nice_tree($url); my $showdetail = $treedetail; --- 273,278 ---- # Fetch the page ! # my $treedetail = XMLTV::Get_nice::get_nice_tree($url); ! my $treedetail = fetch_url($url); my $showdetail = $treedetail; *************** *** 434,439 **** # <span class="season">6:00 am </span> # but this just doesn't sound right to me (i.e. I think it might change again), so let's try both ways ! # @ 2016-08-05 looks like this is permanent ! # my $showstart = $show->look_down('_tag' => 'span', 'class' => 'tvchannel'); my $showstart = $show->look_down('_tag' => 'span', 'class' => 'season'); if (!$prog{'start'}) { --- 437,443 ---- # <span class="season">6:00 am </span> # but this just doesn't sound right to me (i.e. I think it might change again), so let's try both ways ! # ! # @2016-08-05 looks like this is permanent ! # my $showstart = $show->look_down('_tag' => 'span', 'class' => 'tvchannel'); my $showstart = $show->look_down('_tag' => 'span', 'class' => 'season'); if (!$prog{'start'}) { *************** *** 838,841 **** --- 842,866 ---- } + sub fetch_url { + # fetch a url with up to 5 retries + my ($url) = @_; + $XMLTV::Get_nice::FailOnError = 0; + my $content; + my $maxretry = 5; + my $retry = 0; + while ( (not defined($content = XMLTV::Get_nice::get_nice($url))) || (length($content) == 0) ) { + my $r = $XMLTV::Get_nice::Response; + print STDERR "HTTP error: ".$r->status_line."\n"; + $retry++; + return undef if $retry > $maxretry; + print STDERR "Retrying URL: $url (attempt $retry of $maxretry) \n"; + } + $content = decode('UTF-8', $content); + my $t = HTML::TreeBuilder->new(); + $t->parse($content) or die "cannot parse content of $url\n"; + $t->eof; + return $t; + } + sub get_default_dir { my $winhome = $ENV{HOMEDRIVE} . $ENV{HOMEPATH} |
|
From: Robert E. <rm...@us...> - 2017-01-22 21:07:36
|
Update of /cvsroot/xmltv/xmltv In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv10338 Modified Files: Makefile.PL Log Message: remove broken tv_grab_hr, tv_grab_huro, tv_grab_pt Index: Makefile.PL =================================================================== RCS file: /cvsroot/xmltv/xmltv/Makefile.PL,v retrieving revision 1.464 retrieving revision 1.465 diff -C2 -d -r1.464 -r1.465 *** Makefile.PL 22 Jan 2017 20:43:23 -0000 1.464 --- Makefile.PL 22 Jan 2017 21:07:34 -0000 1.465 *************** *** 511,530 **** }, ! { name => 'tv_grab_hr', ! blurb => 'Grabber for Croatia', ! exes => [ 'grab/hr/tv_grab_hr' ], ! pl_files => { 'grab/hr/tv_grab_hr.PL' ! => 'grab/hr/tv_grab_hr' }, ! to_clean => [ 'grab/hr/tv_grab_hr' ], ! deps => [ 'grab/hr/tv_grab_hr' ! => [ 'grab/se_swedb/tv_grab_se_swedb.in' ] ], ! prereqs => { 'XML::LibXML' => 0, ! 'Compress::Zlib' => 0, ! 'IO::Stringy' => 0, ! 'HTTP::Cache::Transparent' => 0, }, ! }, { name => 'tv_grab_huro', ! blurb => 'Grabber for Hungary, Romania, Slovakia, Czech Republic', exes => [ 'grab/huro/tv_grab_huro' ], pl_files => { 'grab/huro/tv_grab_huro.PL' --- 511,533 ---- }, ! # ! # 2017-01-22 - source site (www.gonix.net/ gone. ! # ! # { name => 'tv_grab_hr', ! # blurb => 'Grabber for Croatia', ! # exes => [ 'grab/hr/tv_grab_hr' ], ! # pl_files => { 'grab/hr/tv_grab_hr.PL' ! # => 'grab/hr/tv_grab_hr' }, ! # to_clean => [ 'grab/hr/tv_grab_hr' ], ! # deps => [ 'grab/hr/tv_grab_hr' ! # => [ 'grab/se_swedb/tv_grab_se_swedb.in' ] ], ! # prereqs => { 'XML::LibXML' => 0, ! # 'Compress::Zlib' => 0, ! # 'IO::Stringy' => 0, ! # 'HTTP::Cache::Transparent' => 0, }, ! # }, { name => 'tv_grab_huro', ! blurb => 'Grabber for Slovakia (Hungary, Romania broken)', exes => [ 'grab/huro/tv_grab_huro' ], pl_files => { 'grab/huro/tv_grab_huro.PL' *************** *** 695,709 **** # }, ! { name => 'tv_grab_pt', ! blurb => 'Grabber for Portugal', ! exes => [ 'grab/pt/tv_grab_pt' ], ! prereqs => { 'Getopt::Long' => 0, ! 'HTML::TreeBuilder' => 0, ! 'HTML::Entities' => 1.27, ! 'HTTP::Cache::Transparent' => 0, ! 'DateTime' => 0, ! 'Encode' => 0, ! 'JSON::PP' => 0, }, ! }, { name => 'tv_grab_pt_meo', --- 698,715 ---- # }, ! # ! # 2017-01-22 Broken due to site changes ! # ! # { name => 'tv_grab_pt', ! # blurb => 'Grabber for Portugal', ! # exes => [ 'grab/pt/tv_grab_pt' ], ! # prereqs => { 'Getopt::Long' => 0, ! # 'HTML::TreeBuilder' => 0, ! # 'HTML::Entities' => 1.27, ! # 'HTTP::Cache::Transparent' => 0, ! # 'DateTime' => 0, ! # 'Encode' => 0, ! # 'JSON::PP' => 0, }, ! # }, { name => 'tv_grab_pt_meo', |
|
From: Robert E. <rm...@us...> - 2017-01-22 20:43:26
|
Update of /cvsroot/xmltv/xmltv In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv8901 Modified Files: Makefile.PL Log Message: remove tv_grab_uk_atlas due to site rule changes Index: Makefile.PL =================================================================== RCS file: /cvsroot/xmltv/xmltv/Makefile.PL,v retrieving revision 1.463 retrieving revision 1.464 diff -C2 -d -r1.463 -r1.464 *** Makefile.PL 22 Jan 2017 20:38:18 -0000 1.463 --- Makefile.PL 22 Jan 2017 20:43:23 -0000 1.464 *************** *** 763,792 **** }, ! { name => 'tv_grab_uk_atlas', ! blurb => 'Fast grabber for UK and Ireland using Atlas database', ! exes => [ 'grab/uk_atlas/tv_grab_uk_atlas' ], ! share_files => { ! 'grab/uk_atlas/tv_grab_uk_atlas.user.map.conf' ! => 'tv_grab_uk_atlas/tv_grab_uk_atlas.user.map.conf', ! 'grab/uk_atlas/tv_grab_uk_atlas.pa.genres.conf' ! => 'tv_grab_uk_atlas/tv_grab_uk_atlas.pa.genres.conf', ! 'grab/uk_atlas/tv_grab_uk_atlas.map.genres.conf' ! => 'tv_grab_uk_atlas/tv_grab_uk_atlas.map.genres.conf', ! 'grab/uk_atlas/tv_grab_uk_atlas.map.channels.conf' ! => 'tv_grab_uk_atlas/tv_grab_uk_atlas.map.channels.conf', ! 'grab/uk_atlas/cgi-bin/getatlas.pl' ! => 'tv_grab_uk_atlas/cgi-bin/getatlas.pl', }, ! prereqs => { 'HTTP::Cache::Transparent' => 1.0, ! 'LWP::UserAgent' => 0, ! 'HTTP::Cookies' => 0, ! 'JSON::PP' => 0, ! 'DateTime' => 0, ! 'Date::Parse' => 0, ! 'Encode' => 0, ! 'URI::Escape' => 0, ! 'File::Path' => 0, ! 'Data::Dumper' => 0, ! 'Pod::Usage' => 0, }, ! }, { name => 'tv_grab_uk_bleb', --- 763,795 ---- }, ! # ! # 2017-01-22 tv_grab_uk_atlas removed due to site rule changes ! # ! # { name => 'tv_grab_uk_atlas', ! # blurb => 'Fast grabber for UK and Ireland using Atlas database', ! # exes => [ 'grab/uk_atlas/tv_grab_uk_atlas' ], ! # share_files => { ! # 'grab/uk_atlas/tv_grab_uk_atlas.user.map.conf' ! # => 'tv_grab_uk_atlas/tv_grab_uk_atlas.user.map.conf', ! # 'grab/uk_atlas/tv_grab_uk_atlas.pa.genres.conf' ! # => 'tv_grab_uk_atlas/tv_grab_uk_atlas.pa.genres.conf', ! # 'grab/uk_atlas/tv_grab_uk_atlas.map.genres.conf' ! # => 'tv_grab_uk_atlas/tv_grab_uk_atlas.map.genres.conf', ! # 'grab/uk_atlas/tv_grab_uk_atlas.map.channels.conf' ! # => 'tv_grab_uk_atlas/tv_grab_uk_atlas.map.channels.conf', ! # 'grab/uk_atlas/cgi-bin/getatlas.pl' ! # => 'tv_grab_uk_atlas/cgi-bin/getatlas.pl', }, ! # prereqs => { 'HTTP::Cache::Transparent' => 1.0, ! # 'LWP::UserAgent' => 0, ! # 'HTTP::Cookies' => 0, ! # 'JSON::PP' => 0, ! # 'DateTime' => 0, ! # 'Date::Parse' => 0, ! # 'Encode' => 0, ! # 'URI::Escape' => 0, ! # 'File::Path' => 0, ! # 'Data::Dumper' => 0, ! # 'Pod::Usage' => 0, }, ! # }, { name => 'tv_grab_uk_bleb', |
|
From: Robert E. <rm...@us...> - 2017-01-22 20:38:20
|
Update of /cvsroot/xmltv/xmltv In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv8650 Modified Files: MANIFEST Makefile.PL Log Message: copy tv_grab_sd_json to tv_grab_zz_sdjson in prep for rename, and add warning Index: Makefile.PL =================================================================== RCS file: /cvsroot/xmltv/xmltv/Makefile.PL,v retrieving revision 1.462 retrieving revision 1.463 diff -C2 -d -r1.462 -r1.463 *** Makefile.PL 22 Jan 2017 02:17:34 -0000 1.462 --- Makefile.PL 22 Jan 2017 20:38:18 -0000 1.463 *************** *** 908,911 **** --- 908,923 ---- }, + { name => 'tv_grab_zz_sdjson', + blurb => '$$ Grabber for schedulesDirect.org SD-JSON service (many countries)', + exes => [ 'grab/zz_sdjson/tv_grab_zz_sdjson' ], + prereqs => { 'DateTime' => 0, + 'Digest::SHA' => 0, + 'JSON' => 0, + 'LWP::Protocol::https' => 0, + 'LWP::UserAgent' => 0, + 'Storable' => 0, + 'Try::Tiny' => 0, }, + }, + { name => 'tv_grab_zz_sdjson_sqlite', blurb => '$$ Grabber for schedulesDirect.org SD-JSON service (many countries)', Index: MANIFEST =================================================================== RCS file: /cvsroot/xmltv/xmltv/MANIFEST,v retrieving revision 1.203 retrieving revision 1.204 diff -C2 -d -r1.203 -r1.204 *** MANIFEST 29 Jul 2016 06:36:24 -0000 1.203 --- MANIFEST 22 Jan 2017 20:38:18 -0000 1.204 *************** *** 246,249 **** --- 246,250 ---- grab/za/test.conf grab/za/tv_grab_za + grab/zz_sdjson/tv_grab_sd_json grab/zz_sdjson_sqlite/tv_grab_zz_sdjson_sqlite lib/.cvsignore |
|
From: Nick M. <kno...@us...> - 2017-01-22 02:17:37
|
Update of /cvsroot/xmltv/xmltv In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv13365 Modified Files: Makefile.PL Log Message: Whitespace/alignment. No functional changes Index: Makefile.PL =================================================================== RCS file: /cvsroot/xmltv/xmltv/Makefile.PL,v retrieving revision 1.461 retrieving revision 1.462 diff -C2 -d -r1.461 -r1.462 *** Makefile.PL 22 Jan 2017 01:55:47 -0000 1.461 --- Makefile.PL 22 Jan 2017 02:17:34 -0000 1.462 *************** *** 483,489 **** pl_files => { 'grab/fi/merge.PL' => 'grab/fi/tv_grab_fi' }, to_clean => [ 'grab/fi/tv_grab_fi' ], ! prereqs => { ! 'HTML::TreeBuilder' => 0, ! 'Time::Local' => 0, }, }, --- 483,488 ---- pl_files => { 'grab/fi/merge.PL' => 'grab/fi/tv_grab_fi' }, to_clean => [ 'grab/fi/tv_grab_fi' ], ! prereqs => { 'HTML::TreeBuilder' => 0, ! 'Time::Local' => 0, }, }, *************** *** 704,708 **** 'HTTP::Cache::Transparent' => 0, 'DateTime' => 0, ! 'Encode' => 0, 'JSON::PP' => 0, }, }, --- 703,707 ---- 'HTTP::Cache::Transparent' => 0, 'DateTime' => 0, ! 'Encode' => 0, 'JSON::PP' => 0, }, }, *************** *** 909,932 **** }, ! { name => 'tv_grab_zz_sdjson_sqlite', blurb => '$$ Grabber for schedulesDirect.org SD-JSON service (many countries)', exes => [ 'grab/zz_sdjson_sqlite/tv_grab_zz_sdjson_sqlite' ], ! prereqs => { ! 'Encode' => 0, ! 'JSON' => 0, ! 'Digest::SHA' => 0, ! 'File::Basename'=> 0, ! 'File::Which' => 0, ! 'File::HomeDir' => 0, ! 'File::Path' => 0, ! 'DateTime' => 0, ! 'DateTime::TimeZone' => 0, ! 'DateTime::Format::ISO8601' => 0, ! 'DateTime::Format::SQLite' => 0, ! 'List::MoreUtils' => 0, ! 'DBI' => 0, ! 'DBD::SQLite' => 0, ! 'Scalar::Util' => 0, ! 'Data::Dumper' => 0,} }, --- 908,930 ---- }, ! { name => 'tv_grab_zz_sdjson_sqlite', blurb => '$$ Grabber for schedulesDirect.org SD-JSON service (many countries)', exes => [ 'grab/zz_sdjson_sqlite/tv_grab_zz_sdjson_sqlite' ], ! prereqs => { 'Encode' => 0, ! 'JSON' => 0, ! 'Digest::SHA' => 0, ! 'File::Basename' => 0, ! 'File::Which' => 0, ! 'File::HomeDir' => 0, ! 'File::Path' => 0, ! 'DateTime' => 0, ! 'DateTime::TimeZone' => 0, ! 'DateTime::Format::ISO8601' => 0, ! 'DateTime::Format::SQLite' => 0, ! 'List::MoreUtils' => 0, ! 'DBI' => 0, ! 'DBD::SQLite' => 0, ! 'Scalar::Util' => 0, ! 'Data::Dumper' => 0, }, }, |
|
From: Nick M. <kno...@us...> - 2017-01-22 01:55:50
|
Update of /cvsroot/xmltv/xmltv In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv12232 Modified Files: Makefile.PL Log Message: Update dependency: Digest::SHA1 to Digest::SHA (core Perl module) This issue has already been addressed in the relevant grabber script. Index: Makefile.PL =================================================================== RCS file: /cvsroot/xmltv/xmltv/Makefile.PL,v retrieving revision 1.460 retrieving revision 1.461 diff -C2 -d -r1.460 -r1.461 *** Makefile.PL 6 Jan 2017 01:16:49 -0000 1.460 --- Makefile.PL 22 Jan 2017 01:55:47 -0000 1.461 *************** *** 915,919 **** 'Encode' => 0, 'JSON' => 0, ! 'Digest::SHA1' => 0, 'File::Basename'=> 0, 'File::Which' => 0, --- 915,919 ---- 'Encode' => 0, 'JSON' => 0, ! 'Digest::SHA' => 0, 'File::Basename'=> 0, 'File::Which' => 0, |
|
From: Geoff <bil...@us...> - 2017-01-14 11:42:23
|
Update of /cvsroot/xmltv/xmltv/grab/fr In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv21643 Modified Files: tv_grab_fr Log Message: Fix broken get_channels() (bug #516) - credit Jean-Georges Index: tv_grab_fr =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/fr/tv_grab_fr,v retrieving revision 1.107 retrieving revision 1.108 diff -C2 -d -r1.107 -r1.108 *** tv_grab_fr 15 Aug 2015 06:09:48 -0000 1.107 --- tv_grab_fr 14 Jan 2017 11:42:20 -0000 1.108 *************** *** 1018,1019 **** --- 1018,1024 ---- return get_date_today() + $offset; } + + # Return 'now' as dd-mm-yyyy string + sub get_date_ddmmyyyy { + return get_date_today->strftime("%d-%m-%Y"); + } |
|
From: Jan S. <yu...@us...> - 2017-01-06 08:00:48
|
Update of /cvsroot/xmltv/xmltv/grab/eu_epgdata In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv7576 Modified Files: channel_ids Log Message: Update channel list. Index: channel_ids =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/eu_epgdata/channel_ids,v retrieving revision 1.13 retrieving revision 1.14 diff -C2 -d -r1.13 -r1.14 *** channel_ids 10 Feb 2016 15:14:55 -0000 1.13 --- channel_ids 6 Jan 2017 08:00:46 -0000 1.14 *************** *** 36,42 **** 57;kika.de;KiKA (KiKa) 100;einsextra.ard.de;tagesschau24 ! 146;einsfestival.ard.de;EinsFestival 104;alpha.br-online.de;ARD alpha - 275;theater.zdf.de;ZDFkultur (ZDFtheaterkanal) 276;info.zdf.de;ZDFinfo (ZDFinfokanal) 159;hse24.de;HSE24 (HSE24 Digital|HSE24) --- 36,41 ---- 57;kika.de;KiKA (KiKa) 100;einsextra.ard.de;tagesschau24 ! 146;einsfestival.ard.de;One 104;alpha.br-online.de;ARD alpha 276;info.zdf.de;ZDFinfo (ZDFinfokanal) 159;hse24.de;HSE24 (HSE24 Digital|HSE24) *************** *** 72,76 **** 152;goldstar-tv.de;GoldStar TV (GOLDSTAR TV) 123;beate-uhse.tv;Beate-Uhse.TV (BEATE-UHSE.TV) - 1196;mgm.premiere.de;MGM 396;mezzo.tv;Mezzo 154;heimatkanal.de;Heimatkanal (HEIMATKANAL) --- 71,74 ---- *************** *** 98,102 **** 471;13thstreet.de;13TH STREET (13 TH STREET) 472;scifi.de;Syfy (SCI FI) - 475;einsplus.de;EinsPlus 756;spiegel.de;Spiegel TV Wissen 625;geschichte.spiegel.de;Spiegel Geschichte --- 96,99 ---- *************** *** 110,116 **** 763;rtl-nitro.rtl.de;RTL NITRO 529;passion-tv.de;Passion (PASSION) ! 537;anixehd.tv;Anixe HD 538;paneurope.cnbc.com;CNBC Paneurope - 539;animaxtv.de;Animax (ANIMAX (a/P)) 544;3plus.tv;3plus (3+) 564;sportdigital.tv;sportdigital --- 107,112 ---- 763;rtl-nitro.rtl.de;RTL NITRO 529;passion-tv.de;Passion (PASSION) ! 537;anixehd.tv;Anixe HD Serie (Anixe HD) 538;paneurope.cnbc.com;CNBC Paneurope 544;3plus.tv;3plus (3+) 564;sportdigital.tv;sportdigital *************** *** 123,133 **** 629;hd.cinema.sky.de;Sky Cinema HD 767;atlantic.sky.de;Sky Atlantic HD ! 613;action.sky.de;Sky Action ! 614;nostalgie.sky.de;Sky Nostalgie 615;krimi.sky.de;Sky Krimi ! 616;emotion.sky.de;Sky Emotion ! 617;comedy.sky.de;Sky Comedy ! 618;hits.cinema.sky.de;Sky Hits ! 10059;hd.hits.sky.de;Sky Hits HD 619;1.sport.sky.de;Sky Sport 1 620;2.sport.sky.de;Sky Sport 2 --- 119,129 ---- 629;hd.cinema.sky.de;Sky Cinema HD 767;atlantic.sky.de;Sky Atlantic HD ! 613;action.sky.de;Sky Cinema Action (Sky Action) ! 614;nostalgie.sky.de;Sky Cinema Nostalgie (Sky Nostalgie) 615;krimi.sky.de;Sky Krimi ! 616;emotion.sky.de;Sky Cinema Emotion (Sky Emotion) ! 617;comedy.sky.de;Sky Cinema Comedy (Sky Comedy) ! 618;hits.cinema.sky.de;Sky Cinema Hits (Sky Hits) ! 10059;hd.hits.sky.de;Sky Cinema Hits HD (Sky Hits HD) 619;1.sport.sky.de;Sky Sport 1 620;2.sport.sky.de;Sky Sport 2 *************** *** 153,157 **** 247;telebern.ch;Tele Bärn 250;telezueri.ch;Tele Züri ! 10058;hd.action.sky.de;Sky Action HD 701;3d.sky.de;Sky 3D 601;livehd.mtv.tv;MTV Live HD --- 149,153 ---- 247;telebern.ch;Tele Bärn 250;telezueri.ch;Tele Züri ! 10058;hd.action.sky.de;Sky Cinema Action HD (Sky Action HD) 701;3d.sky.de;Sky 3D 601;livehd.mtv.tv;MTV Live HD *************** *** 173,176 **** 485;nick.de;Nickelodeon (NICK Germany) 136066;viva-comedycentral.tv;Viva / Comedy Central - 764;tnt-glitz.tv;TNT Glitz 666;swr.de;SWR --- 169,175 ---- 485;nick.de;Nickelodeon (NICK Germany) 136066;viva-comedycentral.tv;Viva / Comedy Central 666;swr.de;SWR + 12035;comedy.tnt-tv.de;TNT Comedy + 12036;hd.comedy.tnt-tv.de;TNT Comedy HD + 12033;rtlplus.de;RTL Plus + 136012;tlc.de;TLC HD |
|
From: Kevin G. <kgr...@us...> - 2017-01-06 01:16:52
|
Update of /cvsroot/xmltv/xmltv/grab/sd_json In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv19704/grab/sd_json Modified Files: tv_grab_sd_json Log Message: tv_grab_sd_json: Sort output data. Not only was the data not sorted the ordering was somewhat random do to the way perl handles hashes. This makes it difficult to even compare the output results from two back to back runs of the grabber. And having the output sorted is just generally nicer. Index: tv_grab_sd_json =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/sd_json/tv_grab_sd_json,v retrieving revision 1.21 retrieving revision 1.22 diff -C2 -d -r1.21 -r1.22 *** tv_grab_sd_json 15 Jul 2016 01:26:44 -0000 1.21 --- tv_grab_sd_json 6 Jan 2017 01:16:50 -0000 1.22 *************** *** 60,64 **** use Digest::SHA qw(sha1_hex); use DateTime; - use List::MoreUtils qw(uniq); use Scalar::Util qw(looks_like_number); use Try::Tiny; --- 60,63 ---- *************** *** 967,973 **** sub get_channel_list { my ($conf) = @_; if($conf->{'mode'}->[0] eq 'lineup') { - my %hash; for my $lineup (@{$conf->{'lineup'}}) { if(ref $cache_lineups->{$lineup}->{'stations'} ne 'ARRAY') { --- 966,972 ---- sub get_channel_list { my ($conf) = @_; + my %hash; if($conf->{'mode'}->[0] eq 'lineup') { for my $lineup (@{$conf->{'lineup'}}) { if(ref $cache_lineups->{$lineup}->{'stations'} ne 'ARRAY') { *************** *** 984,997 **** } } - - my @list = keys %hash; - return \@list; } else { ! my @list; ! ! for my $channel (uniq(@{$conf->{'channel'}})) { if(exists $channel_index{$channel}) { ! push(@list, $channel); } else { --- 983,991 ---- } } } else { ! for my $channel (@{$conf->{'channel'}}) { if(exists $channel_index{$channel}) { ! $hash{$channel} = 1; } else { *************** *** 999,1005 **** } } - - return \@list; } } --- 993,1000 ---- } } } + + my @list = sort(keys %hash); + return \@list; } *************** *** 1505,1510 **** # write programs for my $channel (@{$channels}) { ! for my $schedule (values %{$cache_schedules->{$channel}}) { ! for my $program (@{$schedule->{'programs'}}) { write_programme($w, $channel_index{$channel}, $program, $cache_programs->{$program->{'programID'}}); } --- 1500,1506 ---- # write programs for my $channel (@{$channels}) { ! my $schedules = $cache_schedules->{$channel}; ! for my $day (sort(keys %{$schedules})) { ! for my $program (@{$schedules->{$day}->{'programs'}}) { write_programme($w, $channel_index{$channel}, $program, $cache_programs->{$program->{'programID'}}); } |
|
From: Geoff <bil...@us...> - 2017-01-01 13:01:05
|
Update of /cvsroot/xmltv/xmltv/grab/uk_tvguide In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv23698 Modified Files: tv_grab_uk_tvguide Log Message: Try to avoid break when no response from tv_guide website (bug #514) Index: tv_grab_uk_tvguide =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/uk_tvguide/tv_grab_uk_tvguide,v retrieving revision 1.19 retrieving revision 1.20 diff -C2 -d -r1.19 -r1.20 *** tv_grab_uk_tvguide 1 Jan 2017 12:56:39 -0000 1.19 --- tv_grab_uk_tvguide 1 Jan 2017 13:01:03 -0000 1.20 *************** *** 206,209 **** --- 206,216 ---- if ($tree) { my $channelname = $tree->look_down('_tag' => 'option', 'value' => $channel_id); + + # tvguide website can be very slow - try to avoid barfing when no response + if (!defined $channelname) { + print STDERR "Unable to retrieve web page for $channel_id \n"; + next; + } + $channelname = $channelname->as_text; |
|
From: Geoff <bil...@us...> - 2017-01-01 12:56:41
|
Update of /cvsroot/xmltv/xmltv/grab/uk_tvguide In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv23487 Modified Files: tv_grab_uk_tvguide Log Message: Fix stop time and film classification which broke after website changed Index: tv_grab_uk_tvguide =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/uk_tvguide/tv_grab_uk_tvguide,v retrieving revision 1.18 retrieving revision 1.19 diff -C2 -d -r1.18 -r1.19 *** tv_grab_uk_tvguide 5 Aug 2016 17:26:56 -0000 1.18 --- tv_grab_uk_tvguide 1 Jan 2017 12:56:39 -0000 1.19 *************** *** 214,218 **** if (@shows) { foreach my $show (@shows) { ! # $show->dump; # are we processing yesterday's schedule? (see above) --- 214,218 ---- if (@shows) { foreach my $show (@shows) { ! # $show->dump; # are we processing yesterday's schedule? (see above) *************** *** 319,339 **** ! # removed after site changed 2014-11-26 ! if (0) { my ($treespans); ! # rating i.e. "Certificate" ! my $showcertificate = $treespans->look_down('_tag' => 'span', 'class' => 'tvchannel', sub { $_[0]->as_text =~ /Certificate/ } ); ! if ($showcertificate) { ! $showcertificate = $showcertificate->right; ! if ($showcertificate->tag eq 'span' && $showcertificate->attr('class') eq 'programmetext') { ! if ($showcertificate->as_text) { ! $prog{'rating'} = [[ $showcertificate->as_text, 'BBFC' ]] } } - $showcertificate->detach; } ! # start and stop time (actually an optional DTD element) # <span class="datetime">10:00am-11:50am <span class=programmetext> (1 hour 50 minutes)</span> Wed 20 Mar</span> # (use the Date provided to avoid issues with the site running from 06:00-06:00) --- 319,341 ---- ! # Get the "Left Panel" which contains the programme times and attributes ! my $lhs = $showdetail->look_down('_tag' => 'div', 'class' => qr/divLHS-section-2/); ! ! # Get the programme's "attributes" e.g. "Certificate" ! if ($lhs) { ! my @attrs = $lhs->look_down('_tag' => 'span', 'class' => 'LHS-attribute'); ! if (@attrs) { ! foreach my $attr (@attrs) { ! # $attr->dump; ! if ( my $showattr = $attr->as_text() ) { ! if ( $showattr =~ /^Certificate\s:\s(.*)\s*$/ ) { $prog{'rating'} = [[ $1, 'BBFC' ]]; } ! } } } } ! # start time, and stop time (actually an optional DTD element) # <span class="datetime">10:00am-11:50am <span class=programmetext> (1 hour 50 minutes)</span> Wed 20 Mar</span> # (use the Date provided to avoid issues with the site running from 06:00-06:00) *************** *** 344,350 **** # this makes $showtime->set barf on "invalid local time for date in timezone" # ! my $showtimes = $treedetail->look_down('_tag' => 'span', 'class' => 'datetime'); ! if ($showtimes) { ! my ($h, $i, $a, $h2, $i2, $a2, $dt) = $showtimes->as_text =~ /(\d*):(\d*)(am|pm)(?:-(\d*):(\d*)(am|pm))?\s*(?:\(.*\))?\s*(.*)$/; if ($dt && $dt !~ /\D\D\D\s\d\d?\s\D\D\D/) { my @thedt = localtime(time); # ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) --- 346,359 ---- # this makes $showtime->set barf on "invalid local time for date in timezone" # ! # 1/Jan/17 times are now in the left panel. 'datetime' is used for user comments! ! # my $showtimes = $showdetail->look_down('_tag' => 'span', 'class' => 'datetime'); ! # ! if ($lhs) { ! ! # Unfortunately the div with the date doesn't have any safe identifier. There are several ways we could remove the ! # cruft from the container but the following, although clunky, is probably the safest ! my ($dt, $h, $i, $a, $h2, $i2, $a2) = $lhs->as_text =~ /((?:Mon|Tue|Wed|Thu|Fri|Sat|Sun|Christmas|Boxing Day|New Year).*?)(\d*):(\d*)(am|pm)(?:-(\d*):(\d*)(am|pm))?/; ! # print STDERR $dt."\n"; ! if ($dt && $dt !~ /\D\D\D\s\d\d?\s\D\D\D/) { my @thedt = localtime(time); # ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) *************** *** 356,361 **** $dt =~ 'Christmas\s+Day' && do { $dt = '25 Dec '.$yr1; last SWITCH; }; $dt =~ 'Boxing\s+Day' && do { $dt = '26 Dec '.$yr1; last SWITCH; }; ! $dt =~ 'New\s+Years\s+Eve' && do { $dt = '31 Dec '.$yr1; last SWITCH; }; ! $dt =~ 'New\s+Years\s+Day' && do { $dt = '1 Jan ' .$yr2; last SWITCH; }; undef $dt; } --- 365,370 ---- $dt =~ 'Christmas\s+Day' && do { $dt = '25 Dec '.$yr1; last SWITCH; }; $dt =~ 'Boxing\s+Day' && do { $dt = '26 Dec '.$yr1; last SWITCH; }; ! $dt =~ 'New\s+Years\s+Eve' && do { $dt = '31 Dec '.$yr1; last SWITCH; }; ! $dt =~ 'New\s+Years\s+Day' && do { $dt = '1 Jan ' .$yr2; last SWITCH; }; undef $dt; } *************** *** 391,397 **** } } ! ! } ! undef @lis; --- 400,404 ---- } } ! undef @lis; |
|
From: Geoff <bil...@us...> - 2016-12-11 09:43:09
|
Update of /cvsroot/xmltv/xmltv/lib In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv20276 Modified Files: Augment.pm Log Message: bug #503 : don't remove trailing period from title or sub-title if it could be an abbreviation, e.g. 'M.I.A.' Index: Augment.pm =================================================================== RCS file: /cvsroot/xmltv/xmltv/lib/Augment.pm,v retrieving revision 1.20 retrieving revision 1.21 diff -C2 -d -r1.20 -r1.21 *** Augment.pm 12 Jul 2016 01:30:08 -0000 1.20 --- Augment.pm 11 Dec 2016 09:43:07 -0000 1.21 *************** *** 335,339 **** # remove trailing character if any of .,:;-| and not ellipsis ! $prog->{'title'}[$i][0] =~ s/[|\.,:;-]$// if $prog->{'title'}[$i][0] !~ m/\.{3}$/; } } --- 335,340 ---- # remove trailing character if any of .,:;-| and not ellipsis ! # bug #503 : don't remove trailing period if it could be an abbreviation, e.g. 'M.I.A.' ! $prog->{'title'}[$i][0] =~ s/[|\.,:;-]$// if $prog->{'title'}[$i][0] !~ m/\.{3}$/ && $prog->{'title'}[$i][0] !~ m/\..\.$/; } } *************** *** 353,357 **** # remove trailing character if any of .,:;-| and not ellipsis ! $prog->{'sub-title'}[$i][0] =~ s/[|\.,:;-]$//g if $prog->{'sub-title'}[$i][0] !~ m/\.{3}$/; } } --- 354,359 ---- # remove trailing character if any of .,:;-| and not ellipsis ! # bug #503 : don't remove trailing period if it could be an abbreviation, e.g. 'In the U.S.' ! $prog->{'sub-title'}[$i][0] =~ s/[|\.,:;-]$//g if $prog->{'sub-title'}[$i][0] !~ m/\.{3}$/ && $prog->{'sub-title'}[$i][0] !~ m/\..\.$/; } } |
|
From: Nick M. <kno...@us...> - 2016-11-23 19:41:39
|
Update of /cvsroot/xmltv/xmltv/grab/pt_meo In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv21886/grab/pt_meo Modified Files: tv_grab_pt_meo Log Message: Fix typos in POD detected by lintian. (Lintian is a Debian package/source linter) Index: tv_grab_pt_meo =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/pt_meo/tv_grab_pt_meo,v retrieving revision 1.12 retrieving revision 1.13 diff -C2 -d -r1.12 -r1.13 *** tv_grab_pt_meo 15 Dec 2015 14:40:08 -0000 1.12 --- tv_grab_pt_meo 23 Nov 2016 19:41:36 -0000 1.13 *************** *** 30,34 **** See L<http://seguranca.sapo.pt/termosdeutilizacao/apis_rss_webservices.html> for their terms of service. (automatic translation suggests it's free ! for personal use, verfication appreciated) First you must run B<tv_grab_pt_meo --configure> to choose which stations --- 30,34 ---- See L<http://seguranca.sapo.pt/termosdeutilizacao/apis_rss_webservices.html> for their terms of service. (automatic translation suggests it's free ! for personal use, verification appreciated) First you must run B<tv_grab_pt_meo --configure> to choose which stations |
|
From: Nick M. <kno...@us...> - 2016-11-23 19:41:38
|
Update of /cvsroot/xmltv/xmltv/grab/fr_kazer In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv21886/grab/fr_kazer Modified Files: tv_grab_fr_kazer Log Message: Fix typos in POD detected by lintian. (Lintian is a Debian package/source linter) Index: tv_grab_fr_kazer =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/fr_kazer/tv_grab_fr_kazer,v retrieving revision 1.7 retrieving revision 1.8 diff -C2 -d -r1.7 -r1.8 *** tv_grab_fr_kazer 27 Jun 2015 00:30:54 -0000 1.7 --- tv_grab_fr_kazer 23 Nov 2016 19:41:36 -0000 1.8 *************** *** 28,32 **** See TODO INSERTLINKHERE for their terms of service. (automatic translation suggests it's free ! for personal use, verfication appreciated) First you must register an account at kazer.org and choose which stations --- 28,32 ---- See TODO INSERTLINKHERE for their terms of service. (automatic translation suggests it's free ! for personal use, verification appreciated) First you must register an account at kazer.org and choose which stations *************** *** 45,49 **** to your account it can take up to one hour until they are returned from the grabber. You can remove the downloaded guide to force an immediate refresh ! by deleteing the guide from the supplementry file cache. On Unix like system it is by default at ~/.xmltv/supplement/tvguide.zip?u=<your userhash>. --- 45,49 ---- to your account it can take up to one hour until they are returned from the grabber. You can remove the downloaded guide to force an immediate refresh ! by deleting the guide from the supplementry file cache. On Unix like system it is by default at ~/.xmltv/supplement/tvguide.zip?u=<your userhash>. |
|
From: Gary B. <gt...@us...> - 2016-09-26 15:39:21
|
Update of /cvsroot/xmltv/xmltv/grab/zz_sdjson_sqlite In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv13224 Modified Files: tv_grab_zz_sdjson_sqlite Log Message: Update to master version v1.24 after soak Includes fixes: Improve (correct) cast mappings per @jpoet Remove experimental decl Index: tv_grab_zz_sdjson_sqlite =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/zz_sdjson_sqlite/tv_grab_zz_sdjson_sqlite,v retrieving revision 1.2 retrieving revision 1.3 diff -C2 -d -r1.2 -r1.3 *** tv_grab_zz_sdjson_sqlite 28 Aug 2016 23:48:19 -0000 1.2 --- tv_grab_zz_sdjson_sqlite 26 Sep 2016 15:39:19 -0000 1.3 *************** *** 45,48 **** --- 45,50 ---- # Version history: # + # 2016/09/10 - 1.24 - change (improve) cast mapping + # 2016/09/10 - 1.23 - remove use warning nonfatal experimental decl # 2016/08/25 - 1.22 - no warning messages for malformed SD data if quiet # 2016/08/25 - 1.21 - additional error checking of SD data *************** *** 77,81 **** use strict; use warnings FATAL => 'all'; ! use warnings NONFATAL => qw(exec recursion internal malloc newline experimental deprecated portable); no warnings 'once'; --- 79,83 ---- use strict; use warnings FATAL => 'all'; ! use warnings NONFATAL => qw(exec recursion internal malloc newline deprecated portable); no warnings 'once'; *************** *** 893,898 **** { 'Actor' => 'actor', ! 'Guest Star' => 'actor', ! 'Guest Voice' => 'actor', 'Voice' => 'actor', 'Correspondent' => 'guest', --- 895,900 ---- { 'Actor' => 'actor', ! 'Guest Star' => 'guest', ! 'Guest Voice' => 'guest', 'Voice' => 'actor', 'Correspondent' => 'guest', |
|
From: Gary B. <gt...@us...> - 2016-08-28 23:48:21
|
Update of /cvsroot/xmltv/xmltv/grab/zz_sdjson_sqlite In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv32006 Modified Files: tv_grab_zz_sdjson_sqlite Log Message: Update to current master Index: tv_grab_zz_sdjson_sqlite =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/zz_sdjson_sqlite/tv_grab_zz_sdjson_sqlite,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** tv_grab_zz_sdjson_sqlite 29 Jul 2016 06:36:24 -0000 1.1 --- tv_grab_zz_sdjson_sqlite 28 Aug 2016 23:48:19 -0000 1.2 *************** *** 22,26 **** # # For extended help information run ! # tv_grab_zz_sdjson_sqlite --info # # --- 22,26 ---- # # For extended help information run ! # tv_grab_zz_sdjson_sqlite --info # # *************** *** 45,48 **** --- 45,63 ---- # Version history: # + # 2016/08/25 - 1.22 - no warning messages for malformed SD data if quiet + # 2016/08/25 - 1.21 - additional error checking of SD data + # 2016/08/24 - 1.20 - correct sql error reporting + # 2016/08/03 - 1.19 - reflect multinational capability (and fix docs) + # 2016/08/03 - 1.18 - rename grabber based on xmltv agreed convention + # 2016/07/30 - 1.17 - don't report radio stations as tvshow category + # 2016/07/30 - 1.16 - eliminate XML:Writer validation for performance + # 2016/07/17 - 1.15 - use Digest::SHA rather than Digest::SHA1 + # 2016/06/07 - 1.14 - support multipart episodes + # 2016/06/07 - 1.13 - improved season/episode value checks + # 2016/05/28 - 1.12 - add support for episodeImage + # 2016/05/26 - 1.11 - use program duration as length + # 2016/05/26 - 1.10 - hack for tv_find_grabbers source parsing of desc + # 2016/05/25 - 1.9 - Support total seasons, and more robust validation + # 2016/05/24 - 1.8 - retry limit updates and get-lineup improvements # 2016/05/21 - 1.7 - protect against bad json returned by server # 2016/05/21 - 1.6 - correct (mis)use of global variable in package *************** *** 77,81 **** use Encode qw/decode encode/; use JSON; ! use Digest::SHA1 qw(sha1 sha1_hex sha1_base64); use File::Basename; use File::Which; --- 92,96 ---- use Encode qw/decode encode/; use JSON; ! use Digest::SHA qw(sha1 sha1_hex sha1_base64); use File::Basename; use File::Which; *************** *** 88,91 **** --- 103,107 ---- use POSIX qw(strftime); use List::MoreUtils qw(natatime); + use List::Util qw/min max/; use DBI; use DBI qw(:sql_types); *************** *** 98,103 **** my $SCRIPT_VERSION = '$Id$'; ! my $SCRIPT_DESC = 'North America Schedules Direct XMLTV grabber'; ! my $SCRIPT_URL = 'https://github.com/garybuhrmaster/tv_grab_na_sd'; my $SCRIPT_NAME = basename("$0"); my $SCRIPT_NAME_DIR = dirname("$0"); --- 114,118 ---- my $SCRIPT_VERSION = '$Id$'; ! my $SCRIPT_URL = 'https://github.com/garybuhrmaster/tv_grab_zz_sdjson_sqlite'; my $SCRIPT_NAME = basename("$0"); my $SCRIPT_NAME_DIR = dirname("$0"); *************** *** 147,151 **** preferredmethod => 'allatonce', version => "$SCRIPT_VERSION", ! description => "$SCRIPT_DESC", extra_options => [qw/manage-lineups force-download download-only no-download passwordhash=s/], defaults => { days => 30 }, --- 162,166 ---- preferredmethod => 'allatonce', version => "$SCRIPT_VERSION", ! description => 'Multinational (Schedules Direct JSON web services with SQLite DB)', extra_options => [qw/manage-lineups force-download download-only no-download passwordhash=s/], defaults => { days => 30 }, *************** *** 489,493 **** # feed to our DB # ! for (my $retry = 0; $retry < 4; $retry++) { --- 504,508 ---- # feed to our DB # ! for (my $retry = 0; $retry < 7; $retry++) { *************** *** 554,558 **** print (STDERR " downloading " . scalar(@{$stationsSchedulesList}) . " new, updated, or missing daily schedules" . (($retry == 0) ? "" : " (retry $retry)") . "\n") if (!$quiet); ! sleep(10 * $retry); $sql1 = "delete from schedules where station = ? and day = ?"; --- 569,573 ---- print (STDERR " downloading " . scalar(@{$stationsSchedulesList}) . " new, updated, or missing daily schedules" . (($retry == 0) ? "" : " (retry $retry)") . "\n") if (!$quiet); ! sleep(min(30, (10 * $retry))); $sql1 = "delete from schedules where station = ? and day = ?"; *************** *** 561,565 **** $sth1 = $DBH->prepare_cached($sql1); ! if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql1): " . $DBH->errstr . "\n"); --- 576,580 ---- $sth1 = $DBH->prepare_cached($sql1); ! if (!defined($sth1)) { print (STDERR "Unexpected error when preparing statement ($sql1): " . $DBH->errstr . "\n"); *************** *** 567,571 **** } $sth2 = $DBH->prepare_cached($sql2); ! if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql2): " . $DBH->errstr . "\n"); --- 582,586 ---- } $sth2 = $DBH->prepare_cached($sql2); ! if (!defined($sth2)) { print (STDERR "Unexpected error when preparing statement ($sql2): " . $DBH->errstr . "\n"); *************** *** 573,579 **** } $sth3 = $DBH->prepare_cached($sql3); ! if (!defined($sth)) { ! print (STDERR "Unexpected error when preparing statement ($sql2): " . $DBH->errstr . "\n"); exit(1); } --- 588,594 ---- } $sth3 = $DBH->prepare_cached($sql3); ! if (!defined($sth3)) { ! print (STDERR "Unexpected error when preparing statement ($sql3): " . $DBH->errstr . "\n"); exit(1); } *************** *** 652,656 **** if ($sth1->err) { ! print (STDERR "Unexpected error when executing statement ($sql): " . $sth1->errstr . "\n"); exit(1); } --- 667,671 ---- if ($sth1->err) { ! print (STDERR "Unexpected error when executing statement ($sql1): " . $sth1->errstr . "\n"); exit(1); } *************** *** 658,665 **** { my $pID = $program->{'programID'}; ! my $starttime = DateTime::Format::ISO8601->parse_datetime($program->{'airDateTime'}); my $duration = $program->{'duration'}; my $phash = $program->{'md5'}; my $details = $JSON->utf8->encode($program); $sth2->bind_param( 1, $sID, SQL_VARCHAR ); $sth2->bind_param( 2, DateTime::Format::SQLite->format_date($dayDateTime), SQL_DATE ); --- 673,686 ---- { my $pID = $program->{'programID'}; ! my $airDateTime = $program->{'airDateTime'}; my $duration = $program->{'duration'}; my $phash = $program->{'md5'}; my $details = $JSON->utf8->encode($program); + if ((!defined($duration)) || (!defined($phash)) || (!defined($pID)) || (!defined($airDateTime))) + { + print (STDERR "Unexpected parsing error in program (data malformed) in schedule for $sID on " . $meta->{'startDate'} . ", skipping\n") if (!$quiet); + next; + } + my $starttime = DateTime::Format::ISO8601->parse_datetime($airDateTime); $sth2->bind_param( 1, $sID, SQL_VARCHAR ); $sth2->bind_param( 2, DateTime::Format::SQLite->format_date($dayDateTime), SQL_DATE ); *************** *** 672,676 **** if ($sth2->err) { ! print (STDERR "Unexpected error when executing statement ($sql): " . $sth2->errstr . "\n"); exit(1); } --- 693,697 ---- if ($sth2->err) { ! print (STDERR "Unexpected error when executing statement ($sql2): " . $sth2->errstr . "\n"); exit(1); } *************** *** 682,686 **** if ($sth3->err) { ! print (STDERR "Unexpected error when executing statement ($sql): " . $sth3->errstr . "\n"); exit(1); } --- 703,707 ---- if ($sth3->err) { ! print (STDERR "Unexpected error when executing statement ($sql3): " . $sth3->errstr . "\n"); exit(1); } *************** *** 700,704 **** # ! for (my $retry = 0; $retry < 4; $retry++) { --- 721,725 ---- # ! for (my $retry = 0; $retry < 7; $retry++) { *************** *** 763,767 **** print (STDERR " downloading " . scalar(@{$programsList}) . " new, updated, or missing programs" . (($retry == 0) ? "" : " (retry $retry)") . "\n") if (!$quiet); ! sleep(10 * $retry); $sql1 = "replace into programs (program, hash, details) values (?, ?, ?)"; --- 784,788 ---- print (STDERR " downloading " . scalar(@{$programsList}) . " new, updated, or missing programs" . (($retry == 0) ? "" : " (retry $retry)") . "\n") if (!$quiet); ! sleep(min(30, (10 * $retry))); $sql1 = "replace into programs (program, hash, details) values (?, ?, ?)"; *************** *** 832,836 **** if ($sth1->err) { ! print (STDERR "Unexpected error when executing statement ($sql): " . $sth1->errstr . "\n"); exit(1); } --- 853,857 ---- if ($sth1->err) { ! print (STDERR "Unexpected error when executing statement ($sql1): " . $sth1->errstr . "\n"); exit(1); } *************** *** 991,995 **** my $w = XML::Writer->new( 'ENCODING' => 'UTF-8', 'DATA_MODE' => 1, ! 'DATA_INDENT' => 1 ); $w->xmlDecl('UTF-8'); --- 1012,1017 ---- my $w = XML::Writer->new( 'ENCODING' => 'UTF-8', 'DATA_MODE' => 1, ! 'DATA_INDENT' => 1, ! 'UNSAFE' => (!$debug) ); $w->xmlDecl('UTF-8'); *************** *** 1016,1020 **** # it is a necessary compromise # ! $sql = "select schedules.station, schedules.starttime, schedules.duration, schedules.program, schedules.details, programs.details, strftime('%Y%m%d%H%M%S', schedules.starttime), strftime('%Y%m%d%H%M%S', datetime(schedules.starttime, '+' || schedules.duration || ' seconds')) from schedules as schedules left join programs as programs on programs.program = schedules.program where schedules.station in (select distinct stations.station from stations as stations where stations.station in ( select distinct channels.station from channels as channels where channels.lineup = ? and channels.selected = 1)) AND schedules.day >= ? and schedules.day < ? order by schedules.station, schedules.starttime"; $sth = $DBH->prepare_cached($sql); --- 1038,1042 ---- # it is a necessary compromise # ! $sql = "select schedules.station, schedules.starttime, schedules.duration, schedules.program, schedules.details, programs.details, strftime('%Y%m%d%H%M%S', schedules.starttime), strftime('%Y%m%d%H%M%S', datetime(schedules.starttime, '+' || schedules.duration || ' seconds')), stations.details from schedules as schedules left join programs as programs on programs.program = schedules.program left join stations as stations on stations.station = schedules.station where schedules.station in (select distinct stations.station from stations as stations where stations.station in ( select distinct channels.station from channels as channels where channels.lineup = ? and channels.selected = 1)) AND schedules.day >= ? and schedules.day < ? order by schedules.station, schedules.starttime"; $sth = $DBH->prepare_cached($sql); *************** *** 1051,1054 **** --- 1073,1077 ---- $sth->bind_col(7, undef, SQL_VARCHAR ); $sth->bind_col(8, undef, SQL_VARCHAR ); + $sth->bind_col(9, undef, SQL_VARCHAR ); my $programsWritten = 0; *************** *** 1068,1071 **** --- 1091,1099 ---- $programDetails = $JSON->decode($r->[5]); } + my $stationDetails; + if (defined($r->[8])) + { + $stationDetails = $JSON->decode($r->[8]); + } $w->startTag('programme', 'channel' => generateRFC2838($sID), *************** *** 1201,1205 **** # For MythTV, we need to specify the first category # in the xmltv file as one of movie, series, sports, ! # or tvshow. We can derive that from the entityType if (defined($programDetails->{'entityType'})) { --- 1229,1239 ---- # For MythTV, we need to specify the first category # in the xmltv file as one of movie, series, sports, ! # or tvshow. We can derive that from the entityType. ! # If the station is a radio station, do not add tvshow ! my $radioStation = 0; ! if (defined($stationDetails->{'isRadioStation'})) ! { ! $radioStation = $stationDetails->{'isRadioStation'}; ! } if (defined($programDetails->{'entityType'})) { *************** *** 1217,1228 **** $w->dataElement('category', 'series'); } ! else # Should be Show, but if anything else, show { ! $w->dataElement('category', 'tvshow'); } } else # entityType is supposed to be manditory, but.... ! { ! $w->dataElement('category', 'tvshow'); } } --- 1251,1268 ---- $w->dataElement('category', 'series'); } ! else # Should be Show { ! if (!$radioStation) ! { ! $w->dataElement('category', 'tvshow'); ! } } } else # entityType is supposed to be manditory, but.... ! { ! if (!$radioStation) ! { ! $w->dataElement('category', 'tvshow'); ! } } } *************** *** 1268,1274 **** } ! if (defined($programDetails->{'movie'}->{'duration'})) { ! $w->dataElement('length', $programDetails->{'movie'}->{'duration'}, 'units' => 'seconds'); } --- 1308,1331 ---- } ! if (defined($programDetails->{'duration'})) { ! $w->dataElement('length', $programDetails->{'duration'}, 'units' => 'seconds'); ! } ! ! if (defined($programDetails->{'episodeImage'}) && ! defined($programDetails->{'episodeImage'}->{'uri'})) ! { ! my $url = $SD->uriResolve($programDetails->{'episodeImage'}->{'uri'}, '/image'); ! if (defined($programDetails->{'episodeImage'}->{'width'}) && ! defined($programDetails->{'episodeImage'}->{'height'})) ! { ! $w->emptyTag('icon', 'src' => $url, ! 'width' => $programDetails->{'episodeImage'}->{'width'}, ! 'height' => $programDetails->{'episodeImage'}->{'height'}); ! } ! else ! { ! $w->emptyTag('icon', 'src' => $url); ! } } *************** *** 1285,1288 **** --- 1342,1348 ---- } + my $season = ''; + my $episode = ''; + my $part = ''; if (defined($programDetails->{'metadata'})) { *************** *** 1292,1306 **** if (defined($meta->{'Gracenote'})) { ! my $season = $meta->{'Gracenote'}->{'season'}; ! my $episode = $meta->{'Gracenote'}->{'episode'} || ''; ! my $total = $meta->{'Gracenote'}->{'totalEpisodes'} || ''; ! next if (!looks_like_number($season)); $season = $season - 1; ! $episode = $episode - 1 if looks_like_number($episode); ! $episode = "$episode / $total" if ($total ne ''); ! $w->dataElement('episode-num', " $season . $episode . ", 'system' => 'xmltv_ns'); } } } if (defined($scheduleDetails->{'videoProperties'})) --- 1352,1388 ---- if (defined($meta->{'Gracenote'})) { ! $season = $meta->{'Gracenote'}->{'season'}; ! my $totalSeasons = $meta->{'Gracenote'}->{'totalSeasons'}; ! $episode = $meta->{'Gracenote'}->{'episode'}; ! my $totalEpisodes = $meta->{'Gracenote'}->{'totalEpisodes'}; ! next if (!defined($season)); ! $season = '0' if (!looks_like_number($season)); ! $totalSeasons = '' if (!looks_like_number($totalSeasons)); ! $episode = '0' if (!looks_like_number($episode)); ! $totalEpisodes = '' if (!looks_like_number($totalEpisodes)); $season = $season - 1; ! $season = '' if ($season < 0); ! $season = "$season / $totalSeasons" if ($totalSeasons ne ''); ! $episode = $episode - 1; ! $episode = '' if ($episode < 0); ! $episode = "$episode / $totalEpisodes" if ($totalEpisodes ne ''); ! last; } } } + if (defined($scheduleDetails->{'multipart'})) + { + $part = $scheduleDetails->{'multipart'}->{'partNumber'}; + my $totalParts = $scheduleDetails->{'multipart'}->{'totalParts'}; + $part = 0 if (!looks_like_number($part)); + $part = $part - 1; + $part = '' if ($part < 0); + $totalParts = '' if (!looks_like_number($totalParts)); + $part = "$part / $totalParts" if ($totalParts ne ''); + } + if (($season ne '') || ($episode ne '') || ($part ne '')) + { + $w->dataElement('episode-num', " $season . $episode . $part ", 'system' => 'xmltv_ns'); + } if (defined($scheduleDetails->{'videoProperties'})) *************** *** 2280,2288 **** } ! if ($SDtype eq 'DVB-T') { $w->startTag('dvb-channel'); ! my $networkID = $c->{'networkID'} || 0; ! if (looks_like_number($networkID)) { $networkID = 0 + $networkID; --- 2362,2376 ---- } ! if (($SDtype eq 'DVB-T') || ($SDtype eq 'DVB-S') | ($SDtype eq 'DVB-C')) { $w->startTag('dvb-channel'); ! my $freq = $c->{'frequencyHz'}; ! if (defined($freq) && looks_like_number($freq)) ! { ! $freq = 0 + $freq; ! $w->dataElement('frequency', $freq); ! } ! my $networkID = $c->{'networkID'}; ! if (defined($networkID) && looks_like_number($networkID)) { $networkID = 0 + $networkID; *************** *** 2295,2321 **** $w->dataElement('transport-id', $transportID); } ! my $serviceID = $c->{'serviceID'} || 0; ! if (looks_like_number($serviceID)) { $serviceID = 0 + $serviceID; $w->dataElement('service-id', $serviceID); } ! if (defined($c->{'channel'}) && looks_like_number($c->{'channel'})) { ! my $preset = $c->{'channel'}; ! $preset = 0 + $preset; ! $w->dataElement('lcn', $preset); } $w->endTag('dvb-channel'); } - if ($SDtype eq 'DVB-S') - { - } - - if ($SDtype eq 'DVB-C') - { - } - $w->endTag('lineup-entry'); } --- 2383,2401 ---- $w->dataElement('transport-id', $transportID); } ! my $serviceID = $c->{'serviceID'}; ! if (defined($serviceID) && looks_like_number($serviceID)) { $serviceID = 0 + $serviceID; $w->dataElement('service-id', $serviceID); } ! my $lcn = $c->{'channel'}; ! if (defined($lcn) && looks_like_number($lcn)) { ! $lcn = 0 + $lcn; ! $w->dataElement('lcn', $lcn); } $w->endTag('dvb-channel'); } $w->endTag('lineup-entry'); } *************** *** 2614,2618 **** $sql1 = "update lineups set name = ?, location = ?, transport = ?, details = ? where lineup = ?"; $sth1 = $DBH->prepare_cached($sql1); ! if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql1): " . $DBH->errstr . "\n"); --- 2694,2698 ---- $sql1 = "update lineups set name = ?, location = ?, transport = ?, details = ? where lineup = ?"; $sth1 = $DBH->prepare_cached($sql1); ! if (!defined($sth1)) { print (STDERR "Unexpected error when preparing statement ($sql1): " . $DBH->errstr . "\n"); *************** *** 2647,2651 **** if ($sth1->err) { ! print (STDERR "Unexpected error when executing statement ($sql): " . $sth1->errstr . "\n"); exit(1); } --- 2727,2731 ---- if ($sth1->err) { ! print (STDERR "Unexpected error when executing statement ($sql1): " . $sth1->errstr . "\n"); exit(1); } *************** *** 4321,4325 **** my $transportTypeMap = # Map for Schedules Direct transport to XMLTV type { ! 'DVB-C' => 'STB', # DVB-C (most use a STB?) 'DVB-T' => 'DTV', # DVB-T 'DVB-S' => 'DTV', # DVB-S (should be STB?) --- 4401,4405 ---- my $transportTypeMap = # Map for Schedules Direct transport to XMLTV type { ! 'DVB-C' => 'DTV', # DVB-C 'DVB-T' => 'DTV', # DVB-T 'DVB-S' => 'DTV', # DVB-S (should be STB?) *************** *** 4374,4392 **** 'Saskatchewan Film and Video Classification Board' => 'SFVCB', 'Medietilsynet' => 'NMA', ! 'Departamento de Justiça, Classificação, Títulos e Qualificação' => 'ClassInd', 'Alberta\'s Film Classification Board' => 'AFR', ! 'The Régie du cinéma' => 'RCQ', 'Ontario Film Review Board' => 'OFRB', 'Maritime Film Classification Board' => 'MFC', 'Canadian Parental Rating' => 'CHVRS', ! 'Conseil Supérieur de l\'Audiovisuel' => 'CSA', ! 'Dirección General de Radio, Televisión y Cinematografía' => 'RTC', ! 'Instituto de Cinematografía y de las Artes Visuales' => 'ICAA', ! 'Mediakasvatus- ja kuvaohjelmayksikkö' => 'MEKU', 'UK Content Provider' => 'UK', ! 'Centre national du cinéma et de l\'image animée' => 'CNC', 'Irish Film Classification Office' => 'IFCO', # Guess ! 'Statens filmgranskningsbyrå' => 'VET', # Guess ! 'Nemzeti Média- és Hirközlési Hatóság' => 'NMHH', # Guess 'Nederlands Instituut voor de Classificatie van Audiovisuele Media' => 'NICAM', # Guess 'Office of Film and Literature Classification' => 'OFLC', # Guess --- 4454,4472 ---- 'Saskatchewan Film and Video Classification Board' => 'SFVCB', 'Medietilsynet' => 'NMA', ! 'Departamento de Justiça, Classificação, TÃtulos e Qualificação' => 'ClassInd', 'Alberta\'s Film Classification Board' => 'AFR', ! 'The Régie du cinéma' => 'RCQ', 'Ontario Film Review Board' => 'OFRB', 'Maritime Film Classification Board' => 'MFC', 'Canadian Parental Rating' => 'CHVRS', ! 'Conseil Supérieur de l\'Audiovisuel' => 'CSA', ! 'Dirección General de Radio, Televisión y CinematografÃa' => 'RTC', ! 'Instituto de CinematografÃa y de las Artes Visuales' => 'ICAA', ! 'Mediakasvatus- ja kuvaohjelmayksikkö' => 'MEKU', 'UK Content Provider' => 'UK', ! 'Centre national du cinéma et de l\'image animée' => 'CNC', 'Irish Film Classification Office' => 'IFCO', # Guess ! 'Statens filmgranskningsbyrÃ¥' => 'VET', # Guess ! 'Nemzeti Média- és Hirközlési Hatóság' => 'NMHH', # Guess 'Nederlands Instituut voor de Classificatie van Audiovisuele Media' => 'NICAM', # Guess 'Office of Film and Literature Classification' => 'OFLC', # Guess *************** *** 4546,4552 **** Output TV listings in XMLTV format for many locations available in ! North America (US/CA). The data come from ! L<http://www.schedulesdirect.org> and an account must be created ! on the Schedules Direct site in order to grab data. This grabber uses a shared local database which allows for --- 4626,4634 ---- Output TV listings in XMLTV format for many locations available in ! North America (US/CA) and other selected countries internationally. ! The data comes from L<http://www.schedulesdirect.org> and an account ! must be created on the Schedules Direct site in order to grab data. ! Refer to the Schedules Direct site for signup requirements and ! supported locations. This grabber uses a shared local database which allows for *************** *** 4557,4565 **** sources in the same location). ! First, you must run B<tv_grab_zz_sdjson_sqlite --manage-lineups> to manage ! the lineups available to your grabber configuration. ! Second, you must run B<tv_grab_zz_sdjson_sqlite --configure> to choose ! which lineup this configuration will grab (this grabber will share the downloaded information for multiple lineups, and can substantially reduce the royal overheads in those --- 4639,4648 ---- sources in the same location). ! First, you must run B<tv_grab_zz_sdjson_sqlite --manage-lineups> ! to manage the lineups available to your grabber configuration ! at the Schedules Direct service. ! Second, you must run B<tv_grab_zz_sdjson_sqlite --configure> to ! choose which lineup this configuration will grab (this grabber will share the downloaded information for multiple lineups, and can substantially reduce the royal overheads in those *************** *** 4582,4587 **** B<--config-file FILE> Set the name of the configuration file, the ! default is B<~/.xmltv/tv_grab_zz_sdjson_sqlite.conf>. This is the file written by ! B<--configure> and read when grabbing. B<--output FILE> When grabbing, write output to FILE rather than --- 4665,4670 ---- B<--config-file FILE> Set the name of the configuration file, the ! default is B<~/.xmltv/tv_grab_zz_sdjson_sqlite.conf>. This is ! the file written by B<--configure> and read when grabbing. B<--output FILE> When grabbing, write output to FILE rather than *************** *** 4639,4644 **** First you must signup for an account at Schedules Direct. This is a paid service providing EPG data for North America ! (and some other) locations. See L<http://www.schedulesdirect.org> ! for signup requirements. 2. --- 4722,4727 ---- First you must signup for an account at Schedules Direct. This is a paid service providing EPG data for North America ! and other selected countries. See L<http://www.schedulesdirect.org> ! for signup requirements, and the countries served. 2. *************** *** 4821,4824 **** --- 4904,4908 ---- # obtainAvailable - return available counties/satellites # deleteMessage - delete message + # uriResolve - convert uri to absolute # *************** *** 4832,4836 **** use Carp; ! use Digest::SHA1 qw(sha1 sha1_hex sha1_base64); use URI::Escape; use Compress::Zlib; --- 4916,4921 ---- use Carp; ! use Digest::SHA qw(sha1 sha1_hex sha1_base64); ! use URI; use URI::Escape; use Compress::Zlib; *************** *** 4872,4876 **** $self->{'_LWP'}->default_header('Accept-Encoding' => scalar HTTP::Message::decodable(), 'Accept' => 'application/json', ! 'Content_Type' => 'application/json'); bless($self, $class); --- 4957,4963 ---- $self->{'_LWP'}->default_header('Accept-Encoding' => scalar HTTP::Message::decodable(), 'Accept' => 'application/json', ! 'Content_Type' => 'application/json', ! 'Pragma' => 'no-cache', ! 'Cache-Control' => 'no-cache'); bless($self, $class); *************** *** 5160,5163 **** --- 5247,5276 ---- # + # Resolve a possible relative uri to absolute URL + # + sub uriResolve + { + my $self = shift; + + print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); + + my $uri = shift; + + my $path = shift || ''; + + my $return; + + # strip off leading / + # add trailing / if needed + #$rel = '/' . $rel . '/' if (defined($rel) && ($rel ne '')); + + $return = URI->new_abs( $uri, "$self->{'RESTUrl'}" . $path . "/" )->as_string(); + + print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); + + return $return; + } + + # # Delete a message # *************** *** 6695,6697 **** } ! 1; \ No newline at end of file --- 6808,6810 ---- } ! 1; |
|
From: Stefan B. <ste...@us...> - 2016-08-20 16:55:15
|
Update of /cvsroot/xmltv/xmltv/grab/fi/fi/source In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv7076/grab/fi/fi/source Modified Files: telkku.pm Log Message: - telkku: rewritten for update web page contents (bug #510) - NOTE: identifiers have changed; you need to run configure again! - test.conf: updated with new telkku.com identifiers - programme: updated one season/episode extractor regex Index: telkku.pm =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/fi/fi/source/telkku.pm,v retrieving revision 2.05 retrieving revision 2.06 diff -C2 -d -r2.05 -r2.06 *** telkku.pm 21 Jun 2014 16:36:15 -0000 2.05 --- telkku.pm 20 Aug 2016 16:55:13 -0000 2.06 *************** *** 13,16 **** --- 13,18 ---- use strict; use warnings; + use Date::Manip qw(UnixDate); + use JSON qw(); BEGIN { *************** *** 25,73 **** sub description { 'telkku.com' } ! # Grab channel list ! sub channels { ! # Fetch & parse HTML ! my $root = fetchTree("http://www.telkku.com/channel"); ! if ($root) { ! my %channels; # ! # Channel list can be found from the left sidebar # ! # <div class="l-wrap l-grid--16" id="channelContainer"> ! # ... ! # <ul> ! # <li><a href="http://www.telkku.com/channel/list/8/20101218">4 Sport</a></li> ! # <li><a href="http://www.telkku.com/channel/list/24/20101218">4 Sport Pro</a></li> ! # ... ! # <li><a href="http://www.telkku.com/channel/list/87/20101218">Viron ETV</a></li> ! # <li><a href="http://www.telkku.com/channel/list/10/20101218">YLE Teema</a></li> ! # </ul> ! # </div> # ! if (my $container = $root->look_down("id" => "channelContainer")) { ! if (my @list = $container->find("li")) { ! debug(2, "Source telkku.com found " . scalar(@list) . " channels"); ! foreach my $list_entry (@list) { ! if (my $link = $list_entry->find("a")) { ! my $href = $link->attr("href"); ! my $name = $link->as_text(); ! if (defined($href) && length($name) && ! (my($channel_no) = ($href =~ m,channel/list/(\d+)/,))) { ! debug(3, "channel '$name' ($channel_no)"); ! $channels{"${channel_no}.telkku.com"} = "fi $name"; ! } ! } } } } - - # Done with the HTML tree - $root->delete(); - - debug(2, "Source telkku.com parsed " . scalar(keys %channels) . " channels"); - return(\%channels); } --- 27,72 ---- sub description { 'telkku.com' } ! my %categories = ( ! SPORTS => "urheilu", ! MOVIE => "elokuvat", ! ); ! # Fetch raw HTML and extract & parse JSON ! sub _getJSON($$$) { ! my($date, $page, $keys) = @_; + # Fetch raw text + my $text = fetchRaw("http://www.telkku.com/tv-ohjelmat/$date/patch/koko-paiva"); + if ($text) { # ! # All data is encoded in JSON in a script node # ! # <script> ! # window.__INITIAL_STATE__ = {...}; ! # </script> # ! my($match) = ($text =~ /window.__INITIAL_STATE__ = ({.+});/); ! if ($match) { ! my $decoded = JSON->new->decode($match); ! ! if (ref($decoded) eq "HASH") { ! my $data = $decoded; ! ! #debug(5, JSON->new->pretty->encode($decoded)); ! ! # step through hashes using key sequence ! foreach my $key (@{$keys}) { ! debug(5, "Looking for JSON key $key"); ! return unless exists $data->{$key}; ! $data = $data->{$key}; } + debug(5, "Found JSON data"); + + #debug(5, JSON->new->pretty->encode($data)); + #debug(5, "KEYS: ", join(", ", sort keys %{$data})); + return($data); } } } *************** *** 75,138 **** } ! # ! # http://www.telkku.com/movie contains information about (all?) movies for ! # today and the next 7 days, i.e. offsets 0 to 7. We extract the URL to the ! # detailed programme information (http://www.telkku.com/program/show/......) ! # that can then be used to identify movies when processing programme entries. ! # ! { ! my %ids; ! sub _getMovieIDsForOffset($) { ! my($offset) = @_; ! # There is only data for the next 7 days ! return({}) if $offset > 7; ! # Reuse cached data ! return(\%ids) if %ids; ! # In order to reduce website traffic, we only try this once ! $ids{__DUMMY_ID_THAT_NEVER_MATCHES__}++; ! # Fetch & parse HTML (do not ignore HTML5 <section>) ! # This is entirely optional, so please don't abort on failure... ! my $root = fetchTree("http://www.telkku.com/movie", undef, 1, 1); ! if ($root) { ! my $test; ! # ! # Document structure for movie entries: ! # ! # <li class="l-embed theme-hr program"> ! # ... ! # <section class="l-embed__bd program__content"> ! # <a href="http://www.telkku.com/program/show/2014061910151"> ! # ... ! # </a> ! # </section> ! # </li> ! # ! debug(2, "Source telkku.com provided movie data"); ! if (my @list = $root->look_down("class" => qr/program__content/)) { ! debug(2, "Source telkku.com found " . scalar(@list) . " movies"); ! foreach my $list_entry (@list) { ! if (my $link = $list_entry->find("a")) { ! my $href = $link->attr("href"); ! if (defined($href) && length($href)) { ! debug(3, "movie ID: " . $href); ! $ids{$href}++; } } } } - - # Done with the HTML tree - $root->delete(); } ! debug(2, "Source telkku.com parsed " . (scalar(keys %ids) - 1) . " movies"); ! return(\%ids); } } --- 74,145 ---- } ! # Grab channel list ! sub channels { ! # Fetch & extract JSON sub-part ! my $data = _getJSON("tanaan", "peruskanavat", ! ["channelGroups", ! "channelGroupsArray"]); ! # ! # Channels data has the following structure ! # ! # [ ! # { ! # slug => "peruskanavat", ! # channels => [ ! # { ! # id => "yle-tv1", ! # name => "Yle TV1", ! # ... ! # }, ! # ... ! # ], ! # ... ! # }, ! # ... ! # ] ! # ! if (ref($data) eq "ARRAY") { ! my %channels; ! my %duplicates; ! foreach my $item (@{$data}) { ! if ((ref($item) eq "HASH") && ! (exists $item->{slug}) && ! (exists $item->{channels}) && ! (ref($item->{channels}) eq "ARRAY")) { ! my $group = $item->{slug}; ! my $channels = $item->{channels}; ! if (defined($group) && length($group) && ! (ref($channels) eq "ARRAY")) { ! debug(2, "Source telkku.com found group '$group' with " . scalar(@{$channels}) . " channels"); ! foreach my $channel (@{$channels}) { ! if (ref($channel) eq "HASH") { ! my $id = $channel->{id}; ! my $name = $channel->{name}; ! if (defined($id) && length($id) && ! (not exists $duplicates{$id}) && ! length($name)) { ! debug(3, "channel '$name' ($id)"); ! $channels{"${id}.${group}.telkku.com"} = "fi $name"; ! ! # Same ID can appear in multiple groups - avoid duplicates ! $duplicates{$id}++; ! } } } } } } ! debug(2, "Source telkku.com parsed " . scalar(keys %channels) . " channels"); ! return(\%channels); } + + return; } *************** *** 142,229 **** # Get channel number from XMLTV id ! return unless my($channel) = ($id =~ /^(\d+)\.telkku\.com$/); ! # Fetch & parse HTML ! my $root = fetchTree("http://www.telkku.com/channel/list/$channel/$today"); ! if ($root) { ! my $movie_ids = _getMovieIDsForOffset($offset); ! # ! # All program info is contained within a unsorted list with class "programList" ! # ! # <ul class="l-stack programList"> ! # <li> ! # <a class="program" href="http://www.telkku.com/program/show/2012100920451"> ! # <div class="theme-hr program__content"> ! # <div class="program__desc"> ! # <div class="h4 program__title">23:45 Uutisikkuna</div> ! # <div class="progrram__desc">...</div> ! # </div> ! # </div> ! # </a> ! # </li> ! # ... ! # </ul> ! # ! my $opaque = startProgrammeList($id, "fi"); ! if (my $container = $root->look_down("class" => "l-stack programList")) { ! if (my @list = $container->find("li")) { ! foreach my $list_entry (@list) { ! my $link = $list_entry->look_down("class", "program"); ! my $title = $list_entry->look_down("class", "h4 program__title"); ! my $desc = $list_entry->look_down("class", "progrram__desc"); ! if ($link && $title && $desc) { ! my $date; ! # Extract texts from HTML elements. Entities are already decoded. ! $date = $title->as_text(); ! $desc = $desc->as_text(); ! # Use "." to match character (it's not included in \s?) ! if (my($hour, $minute, $title) = ! $date =~ /^(\d{2}):(\d{2}).(.+)/) { ! my $href = $link->attr("href"); ! my $category = (defined($href) && exists($movie_ids->{$href})) ? ! "elokuvat" : undef; ! debug(3, "List entry $channel ($hour:$minute) $title"); ! debug(4, $desc); ! debug(4, $category) if defined $category; ! # Only record entry if title isn't empty ! if (length($title) > 0) { ! my $object = appendProgramme($opaque, $hour, $minute, $title); ! $object->category($category); ! $object->description($desc); ! } ! } ! } } } } ! # Done with the HTML tree ! $root->delete(); ! # Each page on telkku.com contains the program information ! # for one channel for one whole day. ! # ! # Example (compiled from several pages for illustration): ! # ! # /- start time (day) ! # | /- program title ! # | | ! # [23:45 Uutisikkuna (yesterday)] ! # 00:10 Uutisikkuna (today ) ! # ... ! # 23:31 Uusi päivä (today ) ! # 00:00 Kova laki (tomorrow ) ! # [00:40 Piilosana (tomorrow )] ! # [01:00 Tellus-tietovisa (tomorrow )] ! # ! # The lines in [] don't appear on every page. ! # ! # Convert list to program objects ! return(convertProgrammeList($opaque, $yesterday, $today, $tomorrow)); } --- 149,227 ---- # Get channel number from XMLTV id ! return unless my($channel, $group) = ($id =~ /^([\w-]+)\.(\w+)\.telkku\.com$/); ! # Fetch & extract JSON sub-part ! my $data = _getJSON($today, $group, ! ["offeringByChannelGroup", ! $group, ! "offering", ! "publicationsByChannel"]); ! # ! # Programme data has the following structure ! # ! # [ ! # { ! # channel => { ! # id => "yle-tv1", ! # ... ! # }, ! # publications => [ ! # { ! # startTime => "2016-08-18T06:25:00.000+03:00", ! # endTime => "2016-08-18T06:55:00.000+03:00", ! # title => "Helil kyläs", ! # description => "Osa 9/10. Asiaohjelma, mikä ...", ! # programFormat => "MOVIE", ! # ... ! # }, ! # ... ! # ] ! # }, ! # ... ! # ] ! # ! if (ref($data) eq "ARRAY") { ! my @objects; ! foreach my $item (@{$data}) { ! if ((ref($item) eq "HASH") && ! (ref($item->{channel}) eq "HASH") && ! (ref($item->{publications}) eq "ARRAY") && ! ($item->{channel}->{id} eq $channel)) { ! foreach my $programme (@{$item->{publications}}) { ! my($start, $end, $title, $desc) = ! @{$programme}{qw(startTime endTime title description)}; ! #debug(5, JSON->new->pretty->encode($programme)); ! if ($start && $end && $title && $desc) { ! $start = UnixDate($start, "%s"); ! $end = UnixDate($end, "%s"); ! ! # NOTE: entries with same start and end time are invalid ! if ($start && $end && ($start != $end)) { ! my $category = $categories{$programme->{programFormat}}; ! ! debug(3, "List entry $channel.$group ($start -> $end) $title"); ! debug(4, $desc); ! debug(4, $category) if defined $category; ! ! # Create program object ! my $object = fi::programme->new($id, "fi", $title, $start, $end); ! $object->category($category); ! $object->description($desc); ! push(@objects, $object); ! } ! } } } } ! # Fix overlapping programmes ! fi::programme->fixOverlaps(\@objects); ! return(\@objects); } |
|
From: Geoff <bil...@us...> - 2016-08-05 17:26:58
|
Update of /cvsroot/xmltv/xmltv/grab/uk_tvguide In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv8140 Modified Files: tv_grab_uk_tvguide Log Message: Fix break when episode title is "Category ..." [bug #509] Index: tv_grab_uk_tvguide =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/uk_tvguide/tv_grab_uk_tvguide,v retrieving revision 1.17 retrieving revision 1.18 diff -C2 -d -r1.17 -r1.18 *** tv_grab_uk_tvguide 30 Jun 2016 19:46:57 -0000 1.17 --- tv_grab_uk_tvguide 5 Aug 2016 17:26:56 -0000 1.18 *************** *** 420,425 **** # <span class="season">6:00 am </span> # but this just doesn't sound right to me (i.e. I think it might change again), so let's try both ways ! # ! my $showstart = $show->look_down('_tag' => 'span', 'class' => 'tvchannel'); if (!$prog{'start'}) { my ($h, $i, $a) = $showstart->as_text =~ /(\d*):(\d*)\s*(am|pm)/; --- 420,426 ---- # <span class="season">6:00 am </span> # but this just doesn't sound right to me (i.e. I think it might change again), so let's try both ways ! # @ 2016-08-05 looks like this is permanent ! # my $showstart = $show->look_down('_tag' => 'span', 'class' => 'tvchannel'); ! my $showstart = $show->look_down('_tag' => 'span', 'class' => 'season'); if (!$prog{'start'}) { my ($h, $i, $a) = $showstart->as_text =~ /(\d*):(\d*)\s*(am|pm)/; *************** *** 440,444 **** # category # <span class="tvchannel">Category </span><span class="programmetext">General Movie/Drama</span> ! my $showcategory = $show->look_down('_tag' => 'span', 'class' => 'tvchannel', sub { $_[0]->as_text =~ /Category/ } ); if ($showcategory) { $showcategory = $showcategory->right; --- 441,445 ---- # category # <span class="tvchannel">Category </span><span class="programmetext">General Movie/Drama</span> ! my $showcategory = $show->look_down('_tag' => 'span', 'class' => 'tvchannel', sub { $_[0]->as_text =~ /^Category\s*$/ } ); if ($showcategory) { $showcategory = $showcategory->right; |
|
From: Gary B. <gt...@us...> - 2016-08-02 03:21:59
|
Update of /cvsroot/xmltv/xmltv In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv5151 Modified Files: authors.txt Log Message: Add Gary Buhrmaster to authors map Index: authors.txt =================================================================== RCS file: /cvsroot/xmltv/xmltv/authors.txt,v retrieving revision 1.3 retrieving revision 1.4 diff -C2 -d -r1.3 -r1.4 *** authors.txt 3 Jun 2016 00:41:04 -0000 1.3 --- authors.txt 2 Aug 2016 03:21:57 -0000 1.4 *************** *** 22,25 **** --- 22,26 ---- fgouget=Francois Gouget <fg...@fr...> gawen=Bruno Tavares <ga...@us...> + gtb=Gary Buhrmaster <gar...@gm...> igitur=Francois Botha <ig...@gm...> jskov=Jesper Skov <js...@zo...> |
|
From: Robert E. <rm...@us...> - 2016-07-29 06:36:26
|
Update of /cvsroot/xmltv/xmltv/grab/zz_sdjson_sqlite In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv19246/grab/zz_sdjson_sqlite Added Files: tv_grab_zz_sdjson_sqlite Log Message: add Gary Buhrmaster's tv_grab_zz_sdjson_sqlite grabber! --- NEW FILE: tv_grab_zz_sdjson_sqlite --- #!/usr/bin/perl -w # # tv_grab_zz_sdjson_sqlite # # Copyright (c) 2016 Gary Buhrmaster <gar...@gm...> # # This code is distributed under the GNU General Public License v2 (GPLv2) # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # version 2 as published by the Free Software Foundation. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software [...6658 lines suppressed...] print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); if ($self->{'_Error'}) { if ($self->{'RaiseError'}) { Carp::croak($self->{'_ErrorString'}); } if ($self->{'PrintError'}) { Carp::carp($self->{'_ErrorString'}); } } print (STDERR "DEBUG: Returning from " . (caller(0))[3] . "\n") if ($self->{'Debug'}); return; } 1; |
|
From: Robert E. <rm...@us...> - 2016-07-29 06:34:49
|
Update of /cvsroot/xmltv/xmltv/grab/zz_sdjson_sqlite In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv19169/zz_sdjson_sqlite Log Message: Directory /cvsroot/xmltv/xmltv/grab/zz_sdjson_sqlite added to the repository |
|
From: Patric M. <bh...@us...> - 2016-07-15 10:23:27
|
Update of /cvsroot/xmltv/xmltv/grab/ch_search In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv14749 Modified Files: tv_grab_ch_search.in Log Message: tv_grab_ch_search: use https instead of http The website started redirecting http to https. This resulted in channel selection not working anymore. Index: tv_grab_ch_search.in =================================================================== RCS file: /cvsroot/xmltv/xmltv/grab/ch_search/tv_grab_ch_search.in,v retrieving revision 1.21 retrieving revision 1.22 diff -C2 -d -r1.21 -r1.22 *** tv_grab_ch_search.in 13 Apr 2016 10:09:36 -0000 1.21 --- tv_grab_ch_search.in 15 Jul 2016 10:23:24 -0000 1.22 *************** *** 140,145 **** ## attributes of xmltv root element my $head = { ! 'source-data-url' => 'http://tv.search.ch/channels', ! 'source-info-url' => 'http://tv.search.ch/', 'generator-info-name' => 'XMLTV', 'generator-info-url' => 'http://xmltv.org/', --- 140,145 ---- ## attributes of xmltv root element my $head = { ! 'source-data-url' => 'https://tv.search.ch/channels', ! 'source-info-url' => 'https://tv.search.ch/', 'generator-info-name' => 'XMLTV', 'generator-info-url' => 'http://xmltv.org/', |
203 messages has been excluded from this view by a project administrator.
