Identifying Unrecoverable Errors
Brought to you by:
r_x
Menu
▾
▴
Is there any way to easily identify download errors on the map?
I have run atlas creation twice and the second run clearly didn't manage to download whatever is missing from the tile store. (The multiple of 10 from transient to permanent is because I changed the number of tries to 10.)
However, when I show coverage I can't see any obviously missing tiles, let alone 2000+...
Have a look in log file.
You get it from "Debug" menu (in top right).
It should (at least, and among other) contain all failing tiles download, with the HTTP error code.
Yes, it does.
The error code is always 404 and in a browser the URLs return:
Ah. I've maybe solved the problem of seeing them, at least. You have to click the "Show coverage" button again when you change the zoom level: just selecting from the drop-down isn't sufficient. Confusing!
Some of it is tiles off the edge of one of the maps for the selection area that are invalid rather than white at higher zoom levels; and some valid tiles that didn't download from the topography. Will have to run it again.
It's hard to be sure you've got everything in a scenario where there will always be errors you aren't bothered about, as opposed to one where you're aiming for zero unrecoverable errors.
An alternative visualisation might be helpful, so you can see something like the normal red X squares from an overview, regardless of what level they occur at. Then if there are any red (except not red because that's the selection colour) blots where you care about them it's quick and easy to see that you need to re-run the atlas creation.
Yeah, I ran another selection area overnight and ended up with 3685 unrecoverable errors, but I just cannot see where those missing tiles are, and thus whether I need to bother about them or not. Is the job done, or will there end up being a hole somewhere: impossible to say.
The stats say 594 unrecoverable errors for a WMS map, but if I search the log for WMS there's nothing saying "Retry limit reached" as there is with the WMTS one.
The other issue I've encountered is that once you lose the selection area, it's even harder to see what might be going on. Is there a way to save and reload it? Saving a Profile doesn't appear to do that (presumably because you could have different selection areas for different maps in that atlas).
Yes, saving a profile is the solution to your need.
When you restore a profile, the corresponding zone is not "selected", but you can easily display for each layer included in the profile the corresponding selection, right clicking on it, and selecting "Display selected area".
On the other hand, it is not a good practice to have in a given atlas different layers with different selected zone. This may generate a lot of issues when using the atlas. It's A LOT preferable to have all layers of a given atlas based on the same geographical zone.
Last question : you say you have a lot of download errors. But do you have some tiles correctly downloaded (I say "downloaded", not retrieved in the tile sore). Because if it is not the case, and if no tile is correctly downloaded, this probably mean that your mapsource file is wrong (or at least obsolete...).
Which mapsource file are you using ? Did you verify it with MapEvaluator ?
Last edit: Laurent Grenet 2024-07-16
Ah-ha! Thank you so much, Laurent, that's incredibly helpful :)
I had completely missed the "Display selected areas" feature.
Being able to save and load selections would still be helpful for other things, but the fact that the area is displayed in yellow and then you can overlay the coverage in green almost achieves what I was wanting. You have to look at zoom levels individually, and it is tile store coverage rather than errors as such, but the contrast is easily visible. So that works.
The selected zone is the same for both maps and all the layers of each. The uncertainty about errors arises from a slight difference in the map coverage between the IGN SCAN25 and SCAN100 maps at the edges, which often isn't evident until the higher zoom levels, since the lower zoom levels are always filled with white. I'm not concerned about those tiles that don't actually exist in one of the the maps, though, only about tiles which do exist and haven't been correctly downloaded for whatever reason (which happens all the time if you leave the retry count at 2).
That said, I still don't know where the 594 SCAN100 errors are coming from, because everything looks perfect on the coverage vs selected area comparison, and what I can identify for SCAN25 doesn't look like 3000+ tiles to me... I'm inclined to trust that the download worked as intended... but there doesn't seem to be an easy way to be sure within MOBAC...
But yes, the mapsource files are working correctly: that isn't an issue at least.
If your maps are IGN Scan maps, in the border, some tiles are present but blank at low zoom levels, but missing (annd then generating errors) at high zoom level. This may explain.
Nevertheless, I do not understand why you need to set a specific "retry" parameter for these sources. I never had problem with them, with standard parameter.
Are you using them in WMS ? Personnally, I prefer to use the WMTS server (in which tiles are pre-set, and do not need to be "on demand generated", and this may explain the difference...)
And also, I do not use "specifically" Scan100 and Scan25, but rather the "multiscale" that adapt automatically the output with zoom level (ie. Layer GEOGRAPHICALGRIDSYSTEMS.MAPS)
The IGN "multiscale" map is another excellent tip. Thank you very much, Laurent. I hadn't realised that that work had been done for us. It's very nicely thought out, actually. The zoom levels effectively correspond to:
2 - 6 = World
7 -8 = Roads outline
9 - 10 = Long distance road atlas
11 - 12 = More detailed road atlas
13 - 14 = SCAN100
15 - 16 = SCAN25
17 - 18 = Plan Vector Map
Since it has the same underlying data, though, it does have the same bizarre missing tile problem as the SCAN maps. It's not a big deal, you only encounter it at the edges of the coverage, but it does make identifying real errors harder in MOBAC.
The SCAN maps are better for overlaying LIMITES_ADMINISTRATIVES_EXPRESS.LATEST at the lower zoom levels I find.
There would be no way to reload previous selections though, would there, if I did want to go back and use the layers of that multiscale map for an area previously downloaded?
The retry count isn't a big deal either. 10 is excessive, certainly. I think a maximum of 3 or 4 tries has been sufficient, and that was probably before I rate limited the downloads, and only with a few tiles, but the logs aren't retained so I can't check just now. The excess doesn't matter much if it is never needed, of course. And it's a time saver if it is needed, because otherwise you have to re-run the whole atlas creation just because of a glitch.
SCAN100 is WMS, but that wasn't the source of most errors.
Last edit: Calabrese 2024-07-17
Anyway, thank you again for all your help. I think I'm sorted, in that I just did another download and all of the transient errors were converted into permanent ones, which indicates that there was no difficulty in downloading tiles: some are simply missing. I know what to expect from the process now. Perhaps the information MOBAC provides could be expanded in a future version, however.
If you currently have a profile with desired selection on another layer, it's very easy to apply it to this multicale map layer.
Open the corresponding xml file (in your profiles folder) with any text editor, then change everywhere it appears the parameter 'mapSource="xxxxxx"' with the correct source, then save under a new profile name.
PS : retry parameter in my settings is only 1, and it works fine....
Last edit: Laurent Grenet 2024-07-18
Ohhh, cunning! I'd looked at the profiles xml but just hadn't thought of doing that! Yet again, huge thanks for your experienced advice.
I'm sure you're right about the retries too. The IGN servers are very good. I was probably just getting flustered with the errors and the uncertainty inherent in the application when I changed that. It should be irrelevant when there are no missing tiles though, and x10 does have the advantage of making calculations unnecessary.
Ah, I just saw this:
Fair enough. I shall myself stop hinting at UX improvements I cannot implement! :)
It's a fantastic tool that I greatly appreciate: just a little quirky to get to grips with. And clearly capabilities are being added collaboratively, such as EPSG:3857 in WMS custom map sources.