#486 Error obtaining data for more than 20 channels

0.5.65
closed-fixed
5
2014-04-14
2013-05-04
Joel Snape
No

The site used to obtain data for the tv_grab_pt_meo command returns an error when more than 20 channels are requested. This is what the tv_grab_pt_meo execution returns:

requesting http://services.sapo.pt/EPG/GetChannelListByDateInterval?channelSiglas=RTP1,RTP2,SIC,TVI,SICN,SICM,SICR,RTPM,RTPA,EURSP,SLB,ESPN,CART,MOTOR,RMTV,MUTV,PANDA,HOLLW,TCM,AXN,FOX,FOXCR,MTV,FUEL,FASH,TVREC,DISCV,ODISS,NGC,HIST,BIOG,TRAV,MCM%20T,MCM%20P,TRACE,VH1,AFRO,MEZZO,EURN,CNN,BBC%20W,ALJAZ,FR24I,ARTV,BLOOM,TVEI,TV5,CHELS,DISNY,JJAM,SKYN,Q,ESPNA,FLIFE,BARCA,SYFY,FOOD,TVI24,PORTO,BBC%20E,AXNBL,FOXM,TLC,AXNWH,BIGGS,ABOLA,TVIFIC,SICK,CMTV,RTPIN&startDate=2013-05-04 00:00:00&endDate=2013-05-11 23:59:59
could not fetch http://services.sapo.pt/EPG/GetChannelListByDateInterval?channelSiglas=RTP1,RTP2,SIC,TVI,SICN,SICM,SICR,RTPM,RTPA,EURSP,SLB,ESPN,CART,MOTOR,RMTV,MUTV,PANDA,HOLLW,TCM,AXN,FOX,FOXCR,MTV,FUEL,FASH,TVREC,DISCV,ODISS,NGC,HIST,BIOG,TRAV,MCM%20T,MCM%20P,TRACE,VH1,AFRO,MEZZO,EURN,CNN,BBC%20W,ALJAZ,FR24I,ARTV,BLOOM,TVEI,TV5,CHELS,DISNY,JJAM,SKYN,Q,ESPNA,FLIFE,BARCA,SYFY,FOOD,TVI24,PORTO,BBC%20E,AXNBL,FOXM,TLC,AXNWH,BIGGS,ABOLA,TVIFIC,SICK,CMTV,RTPIN&startDate=2013-05-04 00:00:00&endDate=2013-05-11 23:59:59, error: 500 Internal Server Error, aborting

I did some tinkering and changed the script in order to split the request into several fetch commands (patch attached). I have no perl experience so the code might not be very good but it works for me so it might help someone until a fix is made in the official code.

Discussion

  • Joel Snape

    Joel Snape - 2013-05-04

    Patch splitting the requests into parts

     
  • Joel Snape

    Joel Snape - 2013-05-04

    The tv_grab_pt_meo configuration file

     
  • Joel Snape

    Joel Snape - 2013-05-13

    Apparently, even 15 channels can be too much for the service... I recently had to change the script to obtain 5 channels in each request in order to make it work... Just change "my $maxChannelListSize = 15;" to "my $maxChannelListSize = 5;" and it should work. If someone has this problem and can reduce the data gathering interval to less than one week that might help too.

     
  • Geoff

    Geoff - 2014-04-14

    Thanks for the patch.

    v 1.7
    Changed grabber to fetch data in batches of channels.
    Added config option to adjust batch size ('maxchannels', default=5)

     
  • Geoff

    Geoff - 2014-04-14
    • status: open --> closed-fixed
    • Group: --> 0.5.65
     

Log in to post a comment.