Viking GPS data editor and analyzer

Hi Matthias,

Thanks for taking time to keep us informed of this matter.

2011/10/30 Matthias Meißer <matthias.meisser@...>:
> During the past weeks OSM had massive server problems, cause a lot
> of Apps embedded our webmap. For that reason we had to block the most
> popular applications [1] cause they simply violate our tiles usage
> policy [2].
> Now I did some visualisations/compare with the most current logs and
> it turns out that Viking is again under the Top 30 Apps.
> http://www.openstreetmap.org/user/!i!/diary/15190
>
> You know our project is about creating a free worldmap (so collecting
> geoinformations) and not to deploy a full featured webmap directly for
> everyone  as Google Maps does. All our server infrastructure is
> sponsored and administrated by just a few volunteers, so we have a
> lack of every kind of resources to act as a big service.
>
> So can you please use MapQuest or Cloudmade tiles instead and check
> again your tool against the policies?
>
> [1] http://wiki.openstreetmap.org/wiki/Blocked_applications
> [2] http://wiki.openstreetmap.org/wiki/Tile_usage_policy

Yes, due to my own wrong understanding, current release of viking do
not follow policies. But, a next release should change the default and
heavily reduce the load over servers. See commit
7965485d76adfecbbec710d6e9a3ea8717934ec4
http://viking.git.sourceforge.net/git/gitweb.cgi?p=viking/viking;a=commit;h=7965485d76adfecbbec710d6e9a3ea8717934ec4

Technically, current viking try to use an If-modified HTTP header
flag. But the check can be done many times per minutes. We plan to
extend this timeout. As suggested, we will use the 7 days timeout you
require for OpenStreetMap servers.

Please, note that viking already supports MapQuest. We can try to push
MapQuest on top, but user will still be free to use OSM tiles. Do you
have any suggestion about a comment on the label for OSM related
servers? Perhaps "OpenStreetMap (for contributors)" ?


As a conlusion, please, do not filter viking. I know it is a tool used
by many OSM contributors. If you really need to apply restrictions to
viking, please, use an evolved filter that match on version before
1.2.1 in order to allow full acces for next release.
-- 
Guilhem BONNEFILLE
-=- JID: guyou@... MSN: guilhem_bonnefille@...
-=- mailto:guilhem.bonnefille@...
-=- http://nathguil.free.fr/

> Date: Sun, 30 Oct 2011 13:24:56 +0100
> From: matthias.meisser@...
> To: viking-devel@...
> Subject: [Viking-devel] OpenStreetMap Tile usage policy
>
> Hi folks.
>
> During the past weeks OSM had massive server problems, cause a lot
> of Apps embedded our webmap. For that reason we had to block the most
> popular applications [1] cause they simply violate our tiles usage
> policy [2].
> Now I did some visualisations/compare with the most current logs and
> it turns out that Viking is again under the Top 30 Apps.
> http://www.openstreetmap.org/user/!i!/diary/15190
>
> You know our project is about creating a free worldmap (so collecting
> geoinformations) and not to deploy a full featured webmap directly for
> everyone as Google Maps does. All our server infrastructure is
> sponsored and administrated by just a few volunteers, so we have a
> lack of every kind of resources to act as a big service.
>
> So can you please use MapQuest or Cloudmade tiles instead and check
> again your tool against the policies?
>
> [1] http://wiki.openstreetmap.org/wiki/Blocked_applications
> [2] http://wiki.openstreetmap.org/wiki/Tile_usage_policy
>
> bye
> Matthias (user:!i!)

Dear users,
If you use any of OSM tile sources, then please ensure you are following the Tile Usage Policy by setting your tile checking period to 7 days:

Viking->Edit->Preferences->Tile Age = 604800

Unfortunately due to an implementation misunderstanding, this is set by default to 30 seconds. (In all currently released versions)

I know every user wishes OSM to remain fully usable, so by increasing your tile age setting you can do your bit.

Alternatively, as suggested above - MapQuest can be enabled and used in preference see: http://sourceforge.net/apps/mediawiki/viking/index.php?title=Maps

Thank you for your corporation.

So probably we should:

  drop osm tiles from default menu

  (probably) in the new version, make config files with the current
  mapnik layer not get mapnik tiles

  (maybe) make sure the user can configure a map layer with a tile
  server URL easily

  make sure the user-agent is recognizably different post this change

  make a release

That way people who are actually mappers can still have mapnik tiles (to
look at where they went to see what needs fixing, which is part of what
I do), but random users who aren't into OSM are not that likely to use
tiles.

Then, it would make sense to see about other tile sources, but I don't
think we should delay a new release w/o mapnik by default.

Am 31.10.2011 19:27, schrieb Greg Troxel:

> Then, it would make sense to see about other tile sources, but I don't
> think we should delay a new release w/o mapnik by default.
Well even if I understand your interest Greg, I wouldn't do it that fast.

Currently the OSM servers aren't on fire, so we all have the time to 
think before we do something. My personal opinion is, that with simply 
dropping OSM Mapnik support, the users will get angry, cause they feel 
that they lost something useful. We don't want to prevent them to make 
use of OSM, so (my opinion) the best would be to simply change the tiles 
provider e.g. to MapQuest and to make the original OSM tiles available 
to OSM Mappers (that need all details).

What you say?

bye
Matthias
>
>
>
>
>
> ------------------------------------------------------------------------------
> Get your Android app more play: Bring it to the BlackBerry PlayBook
> in minutes. BlackBerry App World™ now supports Android™ Apps
> for the BlackBerry® PlayBook™. Discover just how easy and simple
> it is! http://p.sf.net/sfu/android-dev2dev
>
>
>
> _______________________________________________
> Viking-devel mailing list
> Viking-devel@...
> https://lists.sourceforge.net/lists/listinfo/viking-devel
> Viking home page: http://viking.sf.net/

> Date: Mon, 31 Oct 2011 19:33:02 +0100
> From: matthias.meisser@...
> To: viking-devel@...
> Subject: Re: [Viking-devel] OpenStreetMap Tile usage policy
>
> Am 31.10.2011 19:27, schrieb Greg Troxel:
>
> > Then, it would make sense to see about other tile sources, but I don't
> > think we should delay a new release w/o mapnik by default.

> Well even if I understand your interest Greg, I wouldn't do it that fast.

Agreed.

> Currently the OSM servers aren't on fire, so we all have the time to
> think before we do something. My personal opinion is, that with simply
> dropping OSM Mapnik support, the users will get angry, cause they feel
> that they lost something useful. We don't want to prevent them to make
> use of OSM, so (my opinion) the best would be to simply change the tiles
> provider e.g. to MapQuest and to make the original OSM tiles available
> to OSM Mappers (that need all details).
>
> What you say?

It is very possible to make MapQuest the default tile source (albeit needs adding MapQuest in code not just the configurable xml), however there seems little point in removing OSM server sources as an easy selectable option - after all Viking is not that high up on the offenders list ;)

Even with a new release, as a Desktop program it will not see that much immediate end user adoption:

. Until Linux distribution start shipping it and users upgrade.
. Until there is a Windows version and users upgrade

Also note that, even with new default values of preferences and map types will not effect existing user's preferences or map usage from saved .vik files.


> bye
> Matthias
> >

2011/10/31 Greg Troxel <gdt@...>:
>
> So probably we should:
>
> [...]
>
>  make sure the user-agent is recognizably different post this change

On this aspect I'm really confident as viking's user-agent includes
viking's version number.

-- 
Guilhem BONNEFILLE
-=- JID: guyou@... MSN: guilhem_bonnefille@...
-=- mailto:guilhem.bonnefille@...
-=- http://nathguil.free.fr/

Hi,

Le 30 octobre 2011, Guilhem Bonnefille a écrit :
> Technically, current viking try to use an If-modified HTTP header
> flag. But the check can be done many times per minutes. We plan to
> extend this timeout. As suggested, we will use the 7 days timeout you
> require for OpenStreetMap servers.

Note that openstreetmap.org doesn't use If-modified HTTP header, but
Etag header. Etag headers support was added to viking only a year ago,
which might explain why there are too much traffic from viking.

I have just looked at git log, and the first version which has Etag
support is 0.9.96. According to the summary available from 
http://ikaria.informatik.uni-rostock.de/mm337/osm/tile_stats/, versions
before 0.9.96 account to 1/4 of all viking requests.

It is not possible from the summary to know if there are some viking
clients that generate too much traffic, or if it is evenly spreaded
between the clients.


Anyway, I've looked at the modifications from my initial commit to the
latest version, and it looks like Etag should still work as intended.
But I didn't check if the network traffic sends the correct Etag value
to the server when checking if there is a new tile.
Can anybody check this ?


Thanks,
Jocelyn

Hi,

2011/11/5 Jocelyn Jaubert <jocelyn.jaubert@...>:
> Le 30 octobre 2011, Guilhem Bonnefille a écrit :
>> Technically, current viking try to use an If-modified HTTP header
>> flag. But the check can be done many times per minutes. We plan to
>> extend this timeout. As suggested, we will use the 7 days timeout you
>> require for OpenStreetMap servers.
>
> Note that openstreetmap.org doesn't use If-modified HTTP header, but
> Etag header. Etag headers support was added to viking only a year ago,
> which might explain why there are too much traffic from viking.

Oops... you're right, it's ETag feature.

> I have just looked at git log, and the first version which has Etag
> support is 0.9.96. According to the summary available from
> http://ikaria.informatik.uni-rostock.de/mm337/osm/tile_stats/, versions
> before 0.9.96 account to 1/4 of all viking requests.
>
> It is not possible from the summary to know if there are some viking
> clients that generate too much traffic, or if it is evenly spreaded
> between the clients.

Furthermore, I'm not able to understand if the data report number of
hits or amount of data. With ETag feature, viking regularly request
tiles, but no data are sent back. This certainly produce a load on
server, but a lightweight traffic on network.
Perhaps Matthias can give us more details.


> Anyway, I've looked at the modifications from my initial commit to the
> latest version, and it looks like Etag should still work as intended.
> But I didn't check if the network traffic sends the correct Etag value
> to the server when checking if there is a new tile.
> Can anybody check this ?

I did the following test:
1) launch viking in debug and verbose (viking -d -V)
2) download tiles
3) right click on map entry and "Download New Onscreen Map"
In the log, I can see the ETag was sent and the response from the
server indicates that Etag was considered and no data was transmitted.
All seems OK.

-- 
Guilhem BONNEFILLE
-=- JID: guyou@... MSN: guilhem_bonnefille@...
-=- mailto:guilhem.bonnefille@...
-=- http://nathguil.free.fr/