Pre-bundled maps regenerated

2012-04-06
2013-05-29
  • Markus Bäurle

    Markus Bäurle - 2012-04-06

    Hello everybody.

    As some of you may already have noticed, I have regenerated the pre-bundled maps, which we supply for many areas and countries, last weekend and fixed some problems on Monday evening.
    Map version is still 69 and Midlet version 0.7.7.

    Two important shortcomings have to be mentioned:
    1) Milano/Bergamo, Rhein-Main and Italy, North couldn't be updated because Osm2GpsMid aborts with an error (IndexOutOfBoundsException in code called from RouteData.resolveViaWays). If I remember correctly, this bug has been fixed already, so updates for these areas should be working again once we have release 0.7.8. Using a head revision of Osm2GpsMid is no option as this would generate map version 70, which the released version 0.7.7. can't read.

    2) Believe it or not, but Cloudmade has last updated their extract files on December 13th! So areas/countries which have to use extracts from them (because Geofabrik doesn't offer extracts for all countries) unfortunately have data which is almost 4 months old! Affected countries are:
    zimbabwe, ethiopia, nigeria, egypt, morocco, tunisia, democratic_republic_of_the_congo, kenya, seychelles, namibia, jamaica, mexico, venezuela, puerto_rico, trinidad_and_tobago, malaysia, thailand, afghanistan, bangladesh, iran, sri_lanka, saudi_arabia, kuwait, south_korea, lebanon, singapore, jordan, syria, bahrain, brunei, uae, georgia, armenia, myanmar
    There are certainly ways to work around this such as creating our own extracts or using the extract for the whole continent, but both require extra work for which I don't have enough time.
    Nine of these countries are actually available from Geofabrik and I have already updated the download script to use them, but not the generation script until now. Maybe I will update these during the next days. If I do, I will let you know.
    If the missing update is a big problem for you and you can't create the data yourself then please determine a suitable bounding box for the corresponding bundle file (e.g. Jamaica.properties, found here: http://gpsmid.git.sourceforge.net/git/gitweb.cgi?p=gpsmid/GpsMid;a=tree;f=Osm2GpsMid/prebundledBuildScripts;hb=HEAD) and get in touch with me.

    I hope most of you will enjoy the update despite the shortcomings.

    So long,
    Markus

     
  • Markus Bäurle

    Markus Bäurle - 2012-04-08

    Hello again.

    I have changed as many countries as possible to use up-to-date map extracts from Geofabrik, which affects these countries:
    ethiopia, nigeria, morocco, democratic_republic_of_the_congo, mexico, malaysia, iran, singapore and brunei.

    There is now one single bundle for Malaysia, Singapore and Brunei because Geofabrik offers one single extract for them. I think this is no problem because the JAR/ZIP is only about 12 MB in size.

    So the countries which still have old map data from December (and where we might want to make sure newer extracts are provided by someone) are:
    zimbabwe, egypt, tunisia, kenya, seychelles, namibia, jamaica, venezuela, puerto_rico, trinidad_and_tobago, thailand, afghanistan, bangladesh, sri_lanka, saudi_arabia, kuwait, south_korea, lebanon, jordan, syria, bahrain, uae, georgia, armenia, myanmar

    Generation of Japan is currently running. Seems it was disabled some time ago, now I'm trying if generation works again.
    If it won't show up on the web page then there's obviously still a problem, maybe due to the size.

    So long,
    Markus

     
  • omo683

    omo683 - 2012-06-13

    Two important shortcomings have to be mentioned: 1) Milano/Bergamo, Rhein-Main and Italy, North couldn't be updated because Osm2GpsMid aborts with an error (IndexOutOfBoundsException in code called from RouteData.resolveViaWays). If I remember correctly, this bug has been fixed already, so updates for these areas should be working again once we have release 0.7.8. Using a head revision of Osm2GpsMid is no option as this would generate map version 70, which the released version 0.7.7. can't read.

    I tried to generate a ZIP map of Germany with Osm2GpsMid-0.7.7-map69.jar, but ran into the error mentioned:

    java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
            at java.util.ArrayList.rangeCheck(ArrayList.java:571)
            at java.util.ArrayList.get(ArrayList.java:349)
            at de.ueller.osmToGpsMid.RouteData.resolveViaWays(RouteData.java:232)
            at de.ueller.osmToGpsMid.RouteData.calculateTurnRestrictions(RouteData.java:101)
            at de.ueller.osmToGpsMid.RouteData.create(RouteData.java:93)
            at de.ueller.osmToGpsMid.BundleGpsMid.run(BundleGpsMid.java:568)
            at java.lang.Thread.run(Thread.java:679)
    

    The same occurs with Osm2GpsMid-0.7.71-map69.jar, so there's no way to create a map69 of Germany.

    The good news: I was able to build this map with Osm2GpsMid-0.7.71-map71.jar with the following conditions and settings:

    • I used the OSM extract from Geofabrik: http://download.geofabrik.de/osm/europe/germany.osm.bz2, Size 1955544518 bytes, dated 2012-06-08 16:56

    • I used a machine with 96GiB RAM, running CentOS 6.2 64bit (RHEL 6 clone), Kernel  2.6.32-220.17.1.el6.x86_64

    • java-1.6.0-openjdk-1.6.0.0-1.43.1.10.6.el6_2.x86_64 was installed from the standard CentOS repository

    • command line:

      java -Xmx32600M -XX:+UseCompressedOops -jar Osm2GpsMid-0.7.71-map71-debug.jar --properties=De.properties --mapzip > osm2gpsmid_De_07.log 2>osm2gpsmid_De_07.err
      
    • contents of De.properties:

      mapSource = /anw/tmp/osm2gpsmid/germany.osm.bz2
      cellIDnoLAC = false
      generateSea = false
      app = GpsMid-Generic-full
      dontCompress = wav,amr
      enableEditing = false
      useHouseNumbers = false
      useRouting = motorcar, bicycle, foot
      maxDictDepth = 12
      routing.maxTileSize = 6000
      maxTileSize = 20000
      maxTileWays0 = 1000
      maxTileWays1 = 1000
      maxTileWays2 = 1000
      maxTileWays3 = 1000
      style-file = OSM2GpsMid/cachestyle120508.xml
      useSounds = mp3
      lang = de,en,*
      langName = Deutsch(4),English(5),*
      useSoundFilesWithSyntax = sound
      useIcons = true
      midlet.name = GpsMidDe
      
    • The style file is the geocache style from Billi25, because it contains good german sounds. For map69, I had to remove all <toll rule/> lines from the style file.

    • In addition, for the style file to work under Linux (where case is important), I had to change the filename of one sound file: um-auf-die-Autobahn-aufzufahren.mp3 to um-auf-die-autobahn-aufzufahren.mp3 (changed the line in GoogleSpeachDownload.bat, so the file will be downloaded with lowercase filename.)

    • Osm2GpsMid-0.7.71-map71.jar complained about

      camera_alert.mp3(ERROR: file not found)
      

    This seems to be the new speed trap warning, but currently the used sound file is not included in the jar file.

    One additonal hint: I got the following error on my first try:

    Unhandled FileNotFoundException: /anw/tmp/osm2gpsmid/temp4030649193255528713/map/t3/22426.d (No space left on device)
    java.io.FileNotFoundException: /anw/tmp/osm2gpsmid/temp4030649193255528713/map/t3/22426.d (No space left on device)
            at java.io.FileOutputStream.open(Native Method)
            at java.io.FileOutputStream.<init>(FileOutputStream.java:209)
            at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
            at de.ueller.osmToGpsMid.tools.FileTools.createFileOutputStream(FileTools.java:99)
            at de.ueller.osmToGpsMid.CreateGpsMidData.writeRenderTile(CreateGpsMidData.java:1358)
            at de.ueller.osmToGpsMid.CreateGpsMidData.exportTile(CreateGpsMidData.java:1280)
            at de.ueller.osmToGpsMid.CreateGpsMidData.exportMapToMid(CreateGpsMidData.java:1011)
            at de.ueller.osmToGpsMid.CreateGpsMidData.exportMapToMid(CreateGpsMidData.java:167)
            at de.ueller.osmToGpsMid.BundleGpsMid.run(BundleGpsMid.java:594)
            at java.lang.Thread.run(Thread.java:679)
    Unhandled FileNotFoundException: /anw/tmp/osm2gpsmid/temp4030649193255528713/map/dat/dict-4.dat (No space left on device)
    java.io.FileNotFoundException: /anw/tmp/osm2gpsmid/temp4030649193255528713/map/dat/dict-4.dat (No space left on device)
            at java.io.FileOutputStream.open(Native Method)
            at java.io.FileOutputStream.<init>(FileOutputStream.java:209)
            at java.io.FileOutputStream.<init>(FileOutputStream.java:99)
            at de.ueller.osmToGpsMid.CreateGpsMidData.exportMapToMid(CreateGpsMidData.java:977)
            at de.ueller.osmToGpsMid.CreateGpsMidData.exportMapToMid(CreateGpsMidData.java:179)
            at de.ueller.osmToGpsMid.BundleGpsMid.run(BundleGpsMid.java:594)
            at java.lang.Thread.run(Thread.java:679)
    java.lang.NullPointerException
            at de.ueller.osmToGpsMid.CreateGpsMidData.exportMapToMid(CreateGpsMidData.java:1073)
            at de.ueller.osmToGpsMid.CreateGpsMidData.exportMapToMid(CreateGpsMidData.java:179)
            at de.ueller.osmToGpsMid.BundleGpsMid.run(BundleGpsMid.java:594)
            at java.lang.Thread.run(Thread.java:679)
    

    But there was much free space in the filesystem…
    The reason was: I created the filesystem with too few inodes (mkfs -T largefile). There were no free inodes, because Osm2GpsMid creates a lot of small files. So better create the filesystem with standard settings.

    OK, now I have GpsMidDe-0.7.71-map71.zip (1,15GB) and even GpsMidDe_Strm-0.7.71-map71.zip (1,30GB, with elevation lines). I could upload this files if I had access to one of the servers.

     

Log in to post a comment.