Menu
▾
▴
#159 Can No Longer Download Digital Elevation Data (DEM)
Hello,
since a recent reinstallation, my viking no longer downloads elevation data (DEM). I get the following message: DEM download failure for "my details". When I view the file information, I get: Source: https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Eurasia/N45E005.hgt.zip No DEM File!
I copied the url into my browser and came across an error page like:
A 404 error occurred
Page not found.
The requested URL could not be matched by routing.
I am under ubuntu 20.04.
I tried several viking relocations, but no improvement. I tried the same operation on two other machines and got the same result.
Is there a problem with the server?
Please help me.
Regards,
Jacques Dumas.
I can confirm this issue. I did not previously recognise it because I have been using DEM data of my area downloaded elsewhere for a couple of years.
Stripping back the URL it appears to me that the services require a login now? However, unfortunately I'm not familiar with the USGS site.
Cheers,
r.
For a manual workaround, you can download the tiles for your area here
https://dwtkns.com/srtm30m/
and then try to load them into viking using the DEM layer properties. However, it appears this requires a valid NASA login of which I don't know if it is free or which conditions apply.
r.
Hello,
Tthank you for your quick response. Thanks to her, I was able to
download the DEM that I'm interested in. It remains for me to try to
exploit them, but of that, I make my business. I hope that in the near
future the viking downloads links will be operational again.
Thanks again.
Regards,
Jacques Dumas.
Last edit: Rob Norris 2021-03-05
NB if you know of an alternative server (sorry I don't) to get DEM files from this can be configured in your ~/viking.ini with srtm_http_base_url=XYZ
Available in Viking 1.7 onwards (as mentioned in the manual).
I will investigate if it's possible to include one's login details in Viking's download request - as potentially the NASA site might except some kind of OAuth mechanism...
Hi @robbieonsea
I think I just discovered that the tile server in fact appears to be up, but the tile naming convention / folders have been altered. By default, my Viking 1.8 appears to be looking for
https://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11//Eurasia/N50E015.hgt.zip
(is that double slash // necessary, or is it an error?).
When I enter the URL in my web browser, the URL is valid as far as
https://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11
Beyond that, the directory structure is a different on, see by yourself. So it seems it might be possible to fix this by implementing a different naming scheme in viking. Am I right?
Cheers,
r.
Last edit: rooots 2021-03-05
Addendum:
Looking positive: I can confirm being able to download an example file without the need for authentication using:
Not so fast! :(
Unfortunately that example (or others like it), may appear to download a large zip file, however (I think it is just the redirected to the index.html) it's just an HTML file with a large list of links to the real hgt zip files.
If you run the command line tool 'file' it can tell you what type of file it thinks it is, e.g.
answers --> N21E043.SRTMGL1.hgt.zip: HTML document, ASCII text
(Unzipping the file also doesn't work).
You're right, it's a HTML file :-| When I try to d/l the ZIP file via the web browser, I'm getting an auth dialog and/or "access denied", so obviously with wget we're getting redirected. However, I'm wondering if the 3601x3601 px JPG files in the same folder are only previews, or if they could be used as DEM data as well?
Anyway, your idea about adding an option for providing auth credentials in Viking sounds like the way to go.
If one signs up with the Earth Login https://urs.earthdata.nasa.gov
Then you can download files using http basic authenication e.g.
Obviously you'd want to use a throw away account / non criticial password.
Thus I can readily put those options into Viking and sort out the DEM file (re)naming structure.
So it should work seemlessly as before.
However a proper OAuth method will take longer to sort out
(ATM I don't think I'd delay a new Viking release waiting on this)
Hello,
I was able to download DEMs by opening an account on EOSDIS Earthdata,
then clicking on the link: https://search.earthdata.nasa.gov/search
https://search.earthdata.nasa.gov/search obtained in the welcome email.
Next, I opened the Data Format tab and checked the HGT box which
gave me a choice of three data collections. I opted for: NASA Shuttle
Radar Topography Mission Global 3 arc second V003 which in the end
gives me the following link:
https://search.earthdata.nasa.gov/search/granules?p=C204582034-LPDAAC_ECS&pg[0][gsk]=-start_date&gdf=HGT&tl=1599121967 !
4
https://search.earthdata.nasa.gov/search/granules?p=C204582034-LPDAAC_ECS&pg[0][gsk]=-start_date&gdf=HGT&tl=1599121967!4 !!
Note that when I click on this link, I go directly to the collection I
have chosen, without having to log into EOSDIS Eathdata
I wrote this URLin my Viking.inifile, but I don't understand how to
implement the wget https: // command that you quote in your last post.
Please explain the procedure to me.
Regards,
Jacques Dumas.
Last edit: Rob Norris 2021-03-06
Corrigendum:
Hello,
I was able to download DEMs by opening an account on EOSDIS Earthdata,
then clicking on the link: https://search.earthdata.nasa.gov/search
https://search.earthdata.nasa.gov/search obtained in the welcome email.
Next, I opened the Data Format tab and checked the HGT box which
gave me a choice of three data collections. I opted for: NASA Shuttle
Radar Topography Mission Global 3 arc second V003 which in the end
gives me the following link:
https://search.earthdata.nasa.gov/search/granules?p=C204582034-LPDAAC_ECS&pg[0][gsk]=-start_date&gdf=HGT&tl=1599121967 !
4
https://search.earthdata.nasa.gov/search/granules?p=C204582034-LPDAAC_ECS&pg[0][gsk]=-start_date&gdf=HGT&tl=1599121967!4 !!
Note that when I click on this link, I go directly to the collection I
have chosen, without having to log into EOSDIS Eathdata, however I
have to do this if I want to download a tile.
I wrote this URLin my Viking.inifile, but I don't understand how to
implement the wget https: // command that you quote in your last post.
Please explain the procedure to me.
Regards,
Jacques Dumas.
Last edit: Rob Norris 2021-03-05
@Jacques Dumas:
One more suggestion: It appears you're still using an old version of viking - it might be a good idea to upgrade to the latest version (1.7 or 1.8).
@Jacques Dumas:
First of, please be so kind as to only post your current question without citing your previous posts and our replies - this will make reading and navigating this thread a lot easier. You might want to edit your previous posts and delete all duplicate information. Thanks.
Now to your question: Currently it seems the only way to use the DEM data is to manually implement them in your .VIK file. To do so, right-click on the DEM layer, open properties and add the corresponding DEM files you manually downloaded.
The wget .... command that Rob Norris mentioned is a GNU/linux command line command to download files from the web, so it will do the same as if you're downloading the files using your web browser. Editing the viking.ini file currently will not work, because we currently don't know any valid server which is hosting the DEM tiles. But - we might be close, see my next post.
Good find @rooots
Yes the file & directory layout (no extra directories anymore[1]) has changed on the server end.
So since I'm having to change the DEM layer, I might as well put in the capability to specify username / password in order to be more future proof.
And an easier way to specify the base url, whether the server layout uses continents (i.e. the old style) and then file name pattern as well....
I think internally Viking will keep the nameing and continents layout - as then anything you have already downloaded should still work.
[1] Probably was split up in the past as I think there are approximately 14 thousand files covering the world landscape; so either at that time circa year 2001 that many files in a directory was not possible on the system or it would at least make filesystem operations extremely slow. This is not so much of a problem these days, or only an issue if orders of magnitude bigger number of files.
Ticket moved from /p/viking/support-requests/44/
Also see:
https://wiki.earthdata.nasa.gov/display/EL/How+To+Access+Data+With+cURL+And+Wget
However at the time of writing, the cURL example in the wiki above does not work unless one specifies '--location-trusted' as well.
I've modified Viking so it uses the new USGS location by default - you'll need an Earth Login account in order for this to work and store those in a new section in the preferences.
In a quite timely coincedence a user has made a comment in the Wiki about an alternative DEM data source - https://bailu.ch/dem3/ - which currently does not need any login, although uses it's own directory layout scheme.
https://sourceforge.net/p/viking/wikiallura/Data/
I've thus made it possible to use this source, although since it's only made available as 'Fair Use', the Viking user can't access it by default, they have to do a small but intentional change of configuration of inputting the URL and a couple of other drop down GUI settings in order for it to work.
These changes will be uploaded to the main code repository in the next few days
Thanks, very much appreciated! I will test as soon as possible. Will the changes also be implemented in GTK3-WIP?
Cheers,
r.
It will be on the main line.
I intend to promote the GTK3 branch into the mainline very soon too.