2

I want to merge 2 large raster files (7GB+8GB) I got error messages, with a final

Unable to allocate 14.2 GiB for an array with shape (53701, 35412) and data type int64

I already manage to generate a 13GB file with the merge command.

Is 14.2 GB an absolute limit (for merging) of the QGIS software? Is thère another way to merge 2 large raster file ?

(My computer has 8GB RAM + 1TB Disk)

GDAL command and log:

gdal_merge.bat -a_nodata 0.0 -ot Float32 -of GTiff -o "C:/Users/Asus/Desktop/FALL test.tif" --optfile C:/Users/Asus/AppData/Local/Temp/processing_PdBtfS/4ffcc760e4124a3f9c847586ff134e2e/mergeInputFiles.txt Sortie de commande GDAL : Traceback (most recent call last): File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Scripts\gdal_merge.py", line 12, in <module> sys.exit(main(sys.argv)) ^^^^^^^^^^^^^^ File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 657, in main return gdal_merge(argv) ^^^^^^^^^^^^^^^^ File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 397, in gdal_merge return _gdal_merge(argv=argv) ^^^^^^^^^^^^^^^^^^^^^^ File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 642, in _gdal_merge fi.copy_into(t_fh, band, band, nodata, verbose) File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 348, in copy_into return raster_copy( ^^^^^^^^^^^^ File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 95, in raster_copy return raster_copy_with_mask( ^^^^^^^^^^^^^^^^^^^^^^ File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 190, in raster_copy_with_mask to_write = np.choose(mask_test, (data_src, data_dst)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\numpy\core\fromnumeric.py", line 416, in choose return _wrapfunc(a, 'choose', choices, out=out, mode=mode) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\numpy\core\fromnumeric.py", line 59, in _wrapfunc return bound(*args, **kwds) ^^^^^^^^^^^^^^^^^^^^ numpy.core._exceptions._ArrayMemoryError: Unable to allocate 14.2 GiB for an array with shape (53701, 35412) and data type int64 Le processus a renvoyé le code erreur 1
Improve this question
2
  • you will need more physical RAM (14.2GB +operating system (4GB). 18.2 you would be getting towards 24GB required - 32GB is recommended from experience. Commented May 13 at 20:22
  • 2
    At least create the output tiff as tiled so it can be written into file as smaller chunks. And probably it would be good to apply lossless compression like LZW or deflate as well. GDAL is a 24 years old program and it does not need huge amounts of memory if used in a right way. That time typical PC had 64 MB of memory. But it would be better to use gdalbuildvrt gdalbuildvrt output.vrt *.tif and then gdal_translate -co tiled=yes -co compress=LZW output.vrt output.tif instead of gdal_merge. Commented May 13 at 21:14

0

Reset to default

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.