5

I am trying to output the result of a QGIS 3 model as a CSV file. At first I came across How to export csv/excel in QGIS3.0 model processor? but the accepted answer is not satisfying to me. Further researches only provided the code for QgsVectorFileWriter and how to output a CSV in general, but no information about the vectorFileName to use in QGIS models.

Unfortunately there is no predefined algorithm similar to "package layers" for CSV files.

So I was trying to modify the script itself using QgsVectorFileWriter like this:

QgsVectorFileWriter.writeAsVectorFormat(results, r"C:\tmp\xy.csv", "utf-8", None, "CSV", layerOptions ='GEOMETRY=AS_WKT')

Unfortunately I dont know much about Python yet, so I am struggling to implement this correctly. I think the main issue is which vectorFileName (e.g. results or finaloutput) I should use and where to place this output code correctly.

I now managed to create a partially working script (simplified model which adds fid and id field to points as example):

from qgis.core import QgsProcessing from qgis.core import QgsProcessingAlgorithm from qgis.core import QgsProcessingMultiStepFeedback from qgis.core import QgsProcessingParameterVectorLayer from qgis.core import QgsProcessingParameterFeatureSink from qgis.core import QgsVectorFileWriter from qgis.core import QgsVectorLayer from qgis.core import QgsCoordinateReferenceSystem import processing class Model(QgsProcessingAlgorithm): def initAlgorithm(self, config=None): self.addParameter(QgsProcessingParameterVectorLayer('input', 'input', types=[QgsProcessing.TypeFile], defaultValue=None)) #self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue=None)) #Using this, no CSV will be created self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue='C:/tmp/xy.csv')) #Using this, two CSV will be created #self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeVectorAnyGeometry, createByDefault=True, defaultValue=None)) #Using this no CSV will be created def processAlgorithm(self, parameters, context, model_feedback): # Use a multi-step feedback, so that individual child algorithm progress reports are adjusted for the # overall progress through the model feedback = QgsProcessingMultiStepFeedback(1, model_feedback) results = {} outputs = {} # result alg_params = { 'FIELDS_MAPPING': [{'expression': '$id', 'length': 10, 'name': 'fid', 'precision': 0, 'type': 4},{'expression': '$id-1', 'length': 10, 'name': 'id', 'precision': 0, 'type': 4}], 'INPUT': parameters['input'], 'OUTPUT': parameters['Finaloutput'] } outputs['Result'] = processing.run('qgis:refactorfields', alg_params, context=context, feedback=feedback, is_child_algorithm=True) results['Finaloutput'] = outputs['Result']['OUTPUT'] layer = QgsVectorLayer(results['Finaloutput'], "name", "ogr") crs = QgsCoordinateReferenceSystem("EPSG:4326") QgsVectorFileWriter.writeAsVectorFormat(layer, r'c:\tmp\ab.csv', "utf-8", crs, "CSV", False, [""], layerOptions=["GEOMETRY=AS_YX","SEPARATOR=SEMICOLON"]) return results def name(self): return 'model' def displayName(self): return 'model' def group(self): return '' def groupId(self): return '' def createInstance(self): return Model()

So first it turned out I need to create a QgsVectorLayer such as layer = QgsVectorLayer(results['Finaloutput'], "name", "ogr") first.

Secondly, the API docs as well as the given error messages are really confusing. The main issue was that arguments for QgsVectorFileWriter.writeAsVectorFormat were given the wrong way. First it needs a CRS given as 4th argument instead of None and secondly the 7th argument needs to be a list and not a string. Also the layerOptions needs to be a list and not a string. Using @Fran Raga's hint I figured out this real error (First I thought I need to output the CSV after return because the error then disappeared...).

At last, even more confusing: Using @Joseph's hint it now works partially. But only when setting a defaultvalue like 

self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue='C:/tmp/xy.csv'))

It then outputs two CSV: "xy.csv" (comma as separator) as well as "ab.csv" (semicolon as separator). When leaving 

self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue=None))

no CSV at all is written. Overall, no WKT or YX geometry is written.

How can I properly export the result of a QGIS 3 model to a CSV using pyqgis with predefined settings such as path, delimiter and geometrytype?

Improve this question
4
  • 3
    You has the call return before creating the csv, it never executes the writing of the csv. Put the return after creating the csv file Commented Oct 6, 2019 at 18:10
  • This throws the error TypeError: QgsVectorFileWriter.writeAsVectorFormat(): arguments did not match any overloaded call: overload 1: argument 1 has unexpected type 'dict'. When using results['Finaloutput'] I get TypeError: QgsVectorFileWriter.writeAsVectorFormat(): arguments did not match any overloaded call: overload 1: argument 1 has unexpected type 'str'. Commented Oct 6, 2019 at 18:38
  • 2
    @MrXsquared - You could hardcode the path when setting the defaultValue parameter of the final output: self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue='C:/tmp/xy.csv')) Commented Oct 7, 2019 at 10:14
  • @Joseph thanks to your suggestion setting a different defaultvalue QgsVectorFileWriter.writeAsVectorFormat(layer, r'c:\tmp\ab.csv', "utf-8", crs, "CSV", False, [""], layerOptions=["GEOMETRY=AS_YX","SEPARATOR=SEMICOLON"]) now works, it creates two csv: one with the correct separator (semicolon) "ab.csv" and one with comma "xy.csv". Weird thing is, it does not export any csv when leaving self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue=None). Any idea why? Also it still does not export geometry. Commented Oct 7, 2019 at 20:32

1 Answer 1

Reset to default
4
+50

I think you can do that just with the great internal tool from QGIS3 and without any knowledge of python. All within the graphical interface. You can access to the importing format (extension) by clicking in the '...' in the output element. Here you have the complete video with the process (I can't even upload here the complete gif...).

Here are some screenshot with those tool: enter image description here enter image description here

EDIT: Script changes

I've changed your code a little bit to keep your vectorlayer in memory after refactor tool and just then save it using your own CSV options. As you see, the output parameter is now a string instead of a feature sink. I hope it does the job.

from qgis.core import QgsProcessing from qgis.core import QgsProcessingAlgorithm from qgis.core import QgsProcessingMultiStepFeedback from qgis.core import QgsProcessingParameterVectorLayer from qgis.core import QgsProcessingParameterFeatureSink from qgis.core import QgsVectorFileWriter from qgis.core import QgsVectorLayer from qgis.core import QgsCoordinateReferenceSystem from qgis.core import QgsProject from qgis.core import QgsProcessingParameterString import processing class Model(QgsProcessingAlgorithm): def initAlgorithm(self, config=None): self.addParameter(QgsProcessingParameterVectorLayer('Input', 'input', types=[QgsProcessing.TypeFile], defaultValue=None)) self.addParameter(QgsProcessingParameterString('Finaloutput','finaloutput', 'C:/tmp/xy.csv')) def processAlgorithm(self, parameters, context, model_feedback): # Use a multi-step feedback, so that individual child algorithm progress reports are adjusted for the # overall progress through the model feedback = QgsProcessingMultiStepFeedback(1, model_feedback) results = {} outputs = {} # result alg_params = { 'FIELDS_MAPPING': [{'expression': '$id', 'length': 10, 'name': 'fid', 'precision': 0, 'type': 4},{'expression': '$id-1', 'length': 10, 'name': 'id', 'precision': 0, 'type': 4}], 'INPUT': parameters['Input'], 'OUTPUT': 'memory' } results['Result'] = processing.run('qgis:refactorfields', alg_params, context=context, feedback=feedback, is_child_algorithm=True) outputs['Output'] = results['Result']['OUTPUT'] layer = QgsVectorLayer(outputs['Output'], 'name', 'ogr') crs = QgsCoordinateReferenceSystem("EPSG:4326") QgsVectorFileWriter.writeAsVectorFormat(layer, parameters['Finaloutput'], "utf-8", crs, "CSV", False, [""], layerOptions=["GEOMETRY=AS_YX","SEPARATOR=SEMICOLON"]) return outputs def name(self): return 'model' def displayName(self): return 'model' def group(self): return '' def groupId(self): return '' def createInstance(self): return Model()
Improve this answer
2
  • I see the workaround to export geometry informations. But how can I specify separators (like semicolon, tab, comma, etc)? Commented Oct 11, 2019 at 17:29
  • works like a charm. can you add an explanation to your changes so we (I) can understand how and why this works? Commented Oct 12, 2019 at 21:24

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.