I have a issue about memory management in PyQGIS during loop.
I'm writing a code where there is a while loop.
During the operatione inside the loop that are almost the same,I noticed that the execution time increases and also the RAM occupation too (from 200MB to 128GB). So I tried to write a simplified test script that always executes the same instructions 10.000 times but I notice once again that the execution time increases and the RAM too.
The test script:
- input1: dots shape (23.000 points)
- input2: lines shape (115.000 lines)
- execution: extracts a point from the points shape (always the
same)-->buffer of 10.000m around the point-->extraction of the lines in the buffer. Cycle 10.000 times.
Below you can find the code and a graph of the times laps for each cycle.
Can you explain me why time increases even if the instructions for each cycle are always the same?
I assume that the memory is not freed every cycle.
Is there a way to fix and keep the time the same every loop?
#1A - spatial index on lines alg_params = { 'INPUT': parameters['inputline']#input lines } outputs['ASpatialIndexOnLines'] = processing.run('native:createspatialindex', alg_params, context=context, is_child_algorithm=True) feedback.setCurrentStep(1) n_loop=0 while n_loop<10000: #loop 10.000 times start_time=time.time() # 1 - extract random point alg_params = { 'FIELD': 'k', 'INPUT': parameters['inputpoints'],#input points 'OPERATOR': 0, # = 'VALUE': 100, #extract always point with k=100 'OUTPUT': QgsProcessing.TEMPORARY_OUTPUT } outputs['ExtractRandomPoint'] = processing.run('native:extractbyattribute', alg_params, context=context, is_child_algorithm=True) feedback.setCurrentStep(2) # 2 - Buffer 10000m on random point alg_params = { 'DISSOLVE': False, 'DISTANCE': 10000, 'END_CAP_STYLE': 0, # Arrotondato 'INPUT': outputs['ExtractRandomPoint']['OUTPUT'], 'JOIN_STYLE': 0, # Arrotondato 'MITER_LIMIT': 2, 'SEGMENTS': 5, 'OUTPUT': QgsProcessing.TEMPORARY_OUTPUT } outputs['Buffer10000mOnRandomPoint'] = processing.run('native:buffer', alg_params, context=context, is_child_algorithm=True) feedback.setCurrentStep(3) # 3 - spatial index on buffer alg_params = { 'INPUT': outputs['Buffer10000mOnRandomPoint']['OUTPUT'] } outputs['SpatialIndexOnBuffer'] = processing.run('native:createspatialindex', alg_params, context=context, is_child_algorithm=True) feedback.setCurrentStep(4) # 4 - extract lines in buffer alg_params = { 'INPUT': outputs['ASpatialIndexOnLines']['OUTPUT'], 'INTERSECT': outputs['SpatialIndexOnBuffer']['OUTPUT'], 'PREDICATE': [0], # interseca 'OUTPUT': QgsProcessing.TEMPORARY_OUTPUT } outputs['ExtractLinesInBuffer'] = processing.run('native:extractbylocation', alg_params, context=context, is_child_algorithm=True) end_time=time.time() delta_time=round(end_time-start_time,3) feedback.pushInfo('{},{}'.format(delta_time,n_loop)) #write time and loop n_loop+=1 if feedback.isCanceled(): return {} # when loop finish-->out_test, last extraction alg_params = { 'INPUT': outputs['ExtractLinesInBuffer']['OUTPUT'], 'OUTPUT': parameters['out_test'] } outputs['fixGeometries'] = processing.run('native:fixgeometries', alg_params, context=context, is_child_algorithm=True) return {} 
