I want to download 17,613 tiles from a shapefile that has links to download each DEM.
What would be the best way to do this?
- 1Build a text or CSV file from your data and use uGET to retrieve the data. Reduce file sizes? GDAL shell script or ArcGIS Pro modelbuilder with iterators come to mind. Rebuild your question to contain one question or this will get closed.GBG– GBG2025-03-06 16:01:26 +00:00Commented Mar 6 at 16:01
- 2or point a csv file in wget... like wget.exe --input-file="batch_list.csv" but this is more a file/IT question than a GIS question.Mapperz– Mapperz ♦2025-03-06 16:21:13 +00:00Commented Mar 6 at 16:21
- Thank you. For anyone reading this in the future I right clicked on my layer,, save as,, csv. It would be neat to have a plug-in that can do the downloading thouhghSylvino– Sylvino2025-03-06 16:26:13 +00:00Commented Mar 6 at 16:26
- Search stackoverflowBera– Bera2025-03-07 07:03:19 +00:00Commented Mar 7 at 7:03
Add a comment |
1 Answer
My solution requires a little Python knowledge, so if you're comfortable with coding this problem becomes much easier to solve. There are several ways of achieving the same thing, with different packages, so I will offer stepwise instructions of what you can try rather than specific code, and you can experiment with packages you're comfortable with.
Export the attribute table of the shapefile to a csv or some other text file format you can easily access in Python.
In a Python script, read in the new file you've created, probably with something like Pandas, and extract the column of download links as a list.
Loop over each download link in the list and use one of Python's built-in libraries such as urrlib or requests to download each DEM to a directory in an automated fashion.
In terms of compression, I guess you will have to check and see if the DEMs are already in a compressed format or not. Try downloading just one and use QGIS layer properties to see if it has compression applied. If it doesn't have compression applied, you can use GDAL to apply a compression to each file after it downloads, which will shrink the file size. If it already has compression applied, you might be out of luck on the file size.
