Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#73 Weather download slow

Freevo 1.x series
closed
nobody
Plugins (215)
5
2007-01-18
2006-11-26
Frank Naude
No

Here is a patch I've applied to see why the Weather download is so slow. It would be great if the patch can be applied to SVN. It will help people to understand that the weather download is actually quick, but downloading the radar map takes a lot of time.

Sample output:

download http://www.msnbc.com/m/chnk/d/weather_d_src.asp?acid=SFXX0044 - 1.0 sec
download http://w3.weather.com/weather/map/SFXX0044?from=LAPmaps - 5.9 sec
download http://w3.weather.com//maps/local/local/sf/1a/index_large.html - 2.9 sec
download http://image.weather.com/images/sat/africasat_720x486.jpg - 3.1 sec

So, it takes 1 seconds for the weather download, and 5.9 + 2.9 + 3.1 = 11.9 seconds for the radar map.

Possible suggestions to speed-up the radar map download:

1) Hard code a maplink URL
2) Use threads to download in parallel

Discussion

  • Frank Naude
    Frank Naude
    2006-11-26

     
    Attachments
  • ryan
    ryan
    2006-12-22

    Logged In: YES
    user_id=1196812
    Originator: NO

    Save the following as weather_helper.py:

    """
    BEGINING
    """
    import os
    import urllib
    import config

    location = []
    dataurl = []
    WEATHER_DIR = os.path.join(config.SHARE_DIR, 'images', 'weather')
    for local in config.PLUGIN_WEATHER_LOCATIONS:
    location = local[0]
    dataurl = "http://www.msnbc.com/m/chnk/d/weather_d_src.asp?acid=%s" % location
    mapurl = "http://w3.weather.com/weather/map/%s?from=LAPmaps" % location
    cacheDir = '%s/weather_%s' % (config.FREEVO_CACHEDIR, location)
    cacheFile = '%s/data' % (cacheDir)
    mapFile = '%s/map' % (cacheDir)

    os.system ('wget %s -O %s' % (dataurl, cacheFile ))
    os.system ('wget %s -O %s' % (mapurl, mapFile ))
    """
    END
    """

    Then setup cron to run the following command however often you want, I suggest once an hour:

    freevo execute /full/path/to/weather_helper.py

     
  • ryan
    ryan
    2006-12-22

    Logged In: YES
    user_id=1196812
    Originator: NO

    New patch sent out on mailing list.

     
  • ryan
    ryan
    2006-12-31

    Logged In: YES
    user_id=1196812
    Originator: NO

    Does the weather update helper solve this for you?

     
  • Duncan Webb
    Duncan Webb
    2007-01-03

    • status: open --> pending
     
  • Duncan Webb
    Duncan Webb
    2007-01-03

    Logged In: YES
    user_id=104395
    Originator: NO

    The weather helper should really be a daemon plugin type.

     
  • ryan
    ryan
    2007-01-03

    Logged In: YES
    user_id=1196812
    Originator: NO

    THe newest version no longer uses cron. It goes into memory and sleeps

     
  • Logged In: YES
    user_id=1312539
    Originator: NO

    This Tracker item was closed automatically by the system. It was
    previously set to a Pending status, and the original submitter
    did not respond within 14 days (the time period specified by
    the administrator of this Tracker).

     
    • status: pending --> closed