13

I am building a QGIS plugin that connects to a MySQL database in the local network, and then adds a subset of one of the tables to an in-memory layer; the subset is based on data currency (only taking the most recent observation for each location where measurements are made). This memory layer is successfully created.

However I then want to run some geoprocessing algorithms, and I'm having trouble using the in-memory layer in any of them.

 self.stationuri = "point?crs=epsg:4326&field=id:integer&field={}:double&index=yes".format(self.cb_field.currentText()) self.vlayer = QgsVectorLayer(self.stationuri,"scratch","memory") if not self.vlayer.isValid(): raise Exception("Failed to create in-memory layer") self.vlayer.startEditing() for i,r in enumerate(result): # Result is row-by-row result of SQL query # Add features ... self.vlayer.commitChanges() self.vlayer.updateExtents() # Add layer to map QgsMapLayerRegistry.instance().addMapLayer(self.vlayer) # Layer is successfully added to map with all features and geometry # BELOW IS WHERE IT FALLS APART try: processing.runandload("gdalogr:gridinvdist",self.vlayer,self.cb_field.currentText(),2,0,0,0,0,0,0,0,'Float32',None) # None = in-memory output; I get the same error if I specify a string path and filename. except Exception, e: raise e

No exception is raised, but no output is produced or added to the TOC, but the following log is made in processing.log:

INFO|Mon May 04 2015 11:28:23|GDAL execution console output|/bin/sh: 1: /tmp/processing/bbebe7599c83446d9c2b03a251879657/OUTPUT.tif: not found|/bin/sh: 1: -zfield: not found||FAILURE: Source datasource is not specified.|Usage: gdal_grid [--help-general] [--formats]| [-ot {Byte/Int16/UInt16/UInt32/Int32/Float32/Float64/| CInt16/CInt32/CFloat32/CFloat64}]| [-of format] [-co "NAME=VALUE"]| [-zfield field_name] [-z_increase increase_value] [-z_multiply multiply_value]| [-a_srs srs_def] [-spat xmin ymin xmax ymax]| [-clipsrc <xmin ymin xmax ymax>|WKT|datasource|spat_extent]| [-clipsrcsql sql_statement] [-clipsrclayer layer]| [-clipsrcwhere expression]| [-l layername]* [-where expression] [-sql select_statement]| [-txe xmin xmax] [-tye ymin ymax] [-outsize xsize ysize]| [-a algorithm[:parameter1=value1]*] [-q]| <src_datasource> <dst_filename>||Available algorithms and parameters with their's defaults:| Inverse distance to a power (default)| invdist:power=2.0:smoothing=0.0:radius1=0.0:radius2=0.0:angle=0.0:max_points=0:min_points=0:nodata=0.0| Moving average| average:radius1=0.0:radius2=0.0:angle=0.0:min_points=0:nodata=0.0| Nearest neighbor| nearest:radius1=0.0:radius2=0.0:angle=0.0:nodata=0.0| Various data metrics| <metric name>:radius1=0.0:radius2=0.0:angle=0.0:min_points=0:nodata=0.0| possible metrics are:| minimum| maximum| range| count| average_distance| average_distance_pts|

The important part seems to be FAILURE: Source datasource is not specified. However self.vlayer.isValid() == True, so I don't see what is wrong with my input. I have tried substituting self.vlayer with 'memory:scratch' in the call to processing.runandload, but then I get the following error printed to the console (but not raised): Error: Wrong parameter value: memory:scratch.

I get the same issue when running this through the QGIS GUI, and using the dropdown menu to select my scratch layer which is in the TOC. This occurs whether I specify the output raster as in-memory or specify a location on disk.

This question seems similar, but their solution was to add the memory layer to the TOC before using it. I am already doing that and yet the error persists.

I thought that this was a general issue with memory layers and the QGIS geoprocessing algorithms, but the following works without issue:

processing.runandload("qgis:fixeddistancebuffer",self.vlayer, 500, 5, True, "output_buffer.shp")

What am I doing wrong? Why can't my memory source dataset "be specified" in some of the processing algorithms?

EDIT: here's the source code of gdalogr:gridinvdist if that's useful.

Improve this question

3 Answers 3

Reset to default
4

It seems like memory layers cannot be used as input for GDAL/OGR processing scripts because Processing fails to properly prepare the data for use with ogr2ogr. That's why, for example, the QGIS buffer tool works but the GDAL/OGR buffer tool fails:

Algorithm Buffer vectors starting... GDAL command: cmd.exe /C ogr2ogr.exe "C:\Users\anita\AppData\Local\Temp\processing70e5e0852cb9456ba2e3780f8386122e\86d237c8f41443f58a230a8133172047\OUTPUTLAYER.shp" point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea} point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea} -dialect sqlite -sql "SELECT ST_Buffer( geometry , 1000 ),* FROM 'point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea}' " GDAL command output: FAILURE: Unable to open datasource `point?crs=EPSG:4326' with the following drivers. -> JP2ECW -> OCI -> SOSI ...

Processing would have to somehow prepare the data (save it to a file) and then feed it to the GDAL/OGR tool.

I opened a ticket: Cannot use memory layers with OGR tools

Improve this answer
4

That is the correct way, as explained in the documentation http://docs.qgis.org/2.14/es/docs/user_manual/processing/console.html

the next code work with in memory all except the last that it is load

MDT=path/mdt.tif drain=processing.runalg("grass:r.drain",MDT,"",(pun),False,False,False,"%f,%f,%f,%f"% (xmin, xmax, ymin, ymax),0,-1,0.00100,None) vect=processing.runalg("grass:r.to.vect",drain['output'],0,False,"%f,%f,%f,%f"% (xmin, xmax, ymin, ymax),0,None) bu=processing.runalg("qgis:fixeddistancebuffer",vect['output'],Metros_afecta,1,False,None) buf=bu['OUTPUT'] bufe= QgsVectorLayer(buf,"area", "ogr") #the last load the layer QgsMapLayerRegistry.instance().addMapLayers([bufe])

the processing.runalg return a dictionary in this case bu['OUTPUT'] OUTPUT IS THE KEY, and the value is a temp path you can see the key with processeing.alghelp("name processing") as processing,alghelp("grass:r.drain")

return

processing.alghelp("grass:r.drain") ALGORITHM: r.drain - Traces a flow through an elevation model on a raster map. input <ParameterRaster> coordinate <ParameterString> vector_points <ParameterMultipleInput> -c <ParameterBoolean> -a <ParameterBoolean> -n <ParameterBoolean> GRASS_REGION_PARAMETER <ParameterExtent> GRASS_REGION_CELLSIZE_PARAMETER <ParameterNumber> GRASS_SNAP_TOLERANCE_PARAMETER <ParameterNumber> GRASS_MIN_AREA_PARAMETER <ParameterNumber> output <OutputRaster>

in this case the key is output , take care with capital letter you must writte in capital or with out capital, in this case not capital

Improve this answer
1
  • Memory layers are a special thing and I think you are talking about TEMPORARY_OUTPUT files here, so files written to a temporary directory as files. Memory layers do not exist as files. Commented Feb 9, 2023 at 11:41
1

And what about to look at GDAL Virtual File Systems (http://www.gdal.org/gdal_virtual_file_systems.html)

There is vsimem prefix for in-memory file-like paths...

Improve this answer

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.