Help save net neutrality! Learn more.
Close

#73 Weather download slow

Freevo 1.x series
closed
nobody
Plugins (215)
5
2007-01-18
2006-11-26
Frank Naude
No

Here is a patch I've applied to see why the Weather download is so slow. It would be great if the patch can be applied to SVN. It will help people to understand that the weather download is actually quick, but downloading the radar map takes a lot of time.

Sample output:

download http://www.msnbc.com/m/chnk/d/weather_d_src.asp?acid=SFXX0044 - 1.0 sec
download http://w3.weather.com/weather/map/SFXX0044?from=LAPmaps - 5.9 sec
download http://w3.weather.com//maps/local/local/sf/1a/index_large.html - 2.9 sec
download http://image.weather.com/images/sat/africasat_720x486.jpg - 3.1 sec

So, it takes 1 seconds for the weather download, and 5.9 + 2.9 + 3.1 = 11.9 seconds for the radar map.

Possible suggestions to speed-up the radar map download:

1) Hard code a maplink URL
2) Use threads to download in parallel

Discussion

  • Frank Naude

    Frank Naude - 2006-11-26
     
  • ryan

    ryan - 2006-12-22

    Logged In: YES
    user_id=1196812
    Originator: NO

    Save the following as weather_helper.py:

    """
    BEGINING
    """
    import os
    import urllib
    import config

    location = []
    dataurl = []
    WEATHER_DIR = os.path.join(config.SHARE_DIR, 'images', 'weather')
    for local in config.PLUGIN_WEATHER_LOCATIONS:
    location = local[0]
    dataurl = "http://www.msnbc.com/m/chnk/d/weather_d_src.asp?acid=%s" % location
    mapurl = "http://w3.weather.com/weather/map/%s?from=LAPmaps" % location
    cacheDir = '%s/weather_%s' % (config.FREEVO_CACHEDIR, location)
    cacheFile = '%s/data' % (cacheDir)
    mapFile = '%s/map' % (cacheDir)

    os.system ('wget %s -O %s' % (dataurl, cacheFile ))
    os.system ('wget %s -O %s' % (mapurl, mapFile ))
    """
    END
    """

    Then setup cron to run the following command however often you want, I suggest once an hour:

    freevo execute /full/path/to/weather_helper.py

     
  • ryan

    ryan - 2006-12-22

    Logged In: YES
    user_id=1196812
    Originator: NO

    New patch sent out on mailing list.

     
  • ryan

    ryan - 2006-12-31

    Logged In: YES
    user_id=1196812
    Originator: NO

    Does the weather update helper solve this for you?

     
  • Duncan Webb

    Duncan Webb - 2007-01-03
    • status: open --> pending
     
  • Duncan Webb

    Duncan Webb - 2007-01-03

    Logged In: YES
    user_id=104395
    Originator: NO

    The weather helper should really be a daemon plugin type.

     
  • ryan

    ryan - 2007-01-03

    Logged In: YES
    user_id=1196812
    Originator: NO

    THe newest version no longer uses cron. It goes into memory and sleeps

     
  • SourceForge Robot

    Logged In: YES
    user_id=1312539
    Originator: NO

    This Tracker item was closed automatically by the system. It was
    previously set to a Pending status, and the original submitter
    did not respond within 14 days (the time period specified by
    the administrator of this Tracker).

     
  • SourceForge Robot

    • status: pending --> closed
     

Log in to post a comment.