1

I have a (1) Flow that calls an (2) Invocable Apex which calls a (3) Queueable class, and that class passes data to an (4) Integration Procedure. The IP does stuff with the data, and I can't not use it.

Due to the number of expected records, I have to split the Flow input into chunks so that I can avoid hitting the CPU limit.

I'm new to queueable apex and processing large amounts of data, so I'm confused if I'm doing this in the right order. I've tried 2 things:

  1. Create the chunks in the Invocable Apex, then pass each chunk to the Queueable Apex where they're passed to the IP
  2. Pass Flow inputs to invocable apex, then pass that to the Queueable Apex. Create the chunk, then pass each chunk to the IP

The inputs from the Flow are below:

listInput = ([0VlWK0000002DM90AM,0VlWK0000002DMA0A2,0VlWK0000002DMB0A2...]) where I currently have 200 records to test

procedureAPIName = AccreditationIP

Here's the code when I do #2:

Invocable Apex:

public class Test_InvokeIPQueueable { @InvocableMethod(label = 'Method that calls the queueable method') public static void callQueuableMethod(List<FlowInputs> flowInputs){ String procedureName = flowInputs.get(0).procedureAPIName; List<String> ids = flowInputs.get(0).listInput; System.debug('procedureName: ' + procedureName); System.debug('ids size ' + ids.size()); Id jobId = System.enqueueJob(new Test_QueueableIP (procedureName, ids)); System.debug(jobId); } public class FlowInputs { @InvocableVariable(label = 'Procedure Name') public String procedureAPIName; @InvocableVariable(label = 'List Input') public List<String> listInput; } 

}

Queueable Class

public class Test_QueueableIP implements Queueable { private List<String> idList; private String ipName; private Integer chunkSize; private Integer numOfChunks; Map<String, Object> ipInput = new Map<String, Object>(); Map<String, Object> ipOptions = new Map<String, Object>(); Map<String, Object> ipOutput = new Map<String, Object>(); public Test_QueueableIP (String procedureAPIName, List<String> listInput){ System.debug('IN QUEUEABLE CLASS'); this.idList = listInput; this.ipName = procedureAPIName; //chunkSize = 35 this.chunkSize = Integer.valueOf(System.label.BatchApexChunkSize); } public void execute(QueueableContext context){ List<List<String>> chunkList; if(idList.size() > chunkSize){ System.debug('ids.size is greater than chunkSize, so we will split into chunks'); //Chunking List<Account> chunk = new List<Account>(); for(Integer i = 0; idList.size() > i; i++){ if(chunk.size() <= chunkSize - 1 ){ chunk.add(idList[i]); } else{ System.debug('chunk :' + chunk); System.debug('chunkSize: ' + chunk.size()); System.debug('chunk 1st: ' + chunk[0] + ' chunk last: ' + chunk[chunk.size()-1]); //Chunk size met, pass data to IP ipOutput = (Map<String, Object>) omnistudio.IntegrationProcedureService.runIntegrationService(ipName, ipInput, ipOptions); // Log the output from the Integration Procedure service System.debug(LoggingLevel.Error, 'Queueable Integration Procedure Output: ' + ipOutput); //Clear the ipInput map to be reusable ipInput.clear(); //Clear the chunk chunk.clear(); chunk.add(idList[i]); } } } System.debug('End!!!!!!'); } } 

Am I correct in assuming that #2 isn't the right way because the debug log says there's only 1 queueable job. Given the amount of records and my chunk size, I expected there to be around 6 batches:enter image description here

and the ApexJobLog for it has 0 enter image description here

Could you please guide me with where I went wrong? Thanks in advance

3
  • 1. Your invocable doesn't handle flow bulkification, only considering the first inputs. 2. If the flow can work out which record IDs to pass, can you not have the queueable work this out for itself, and perform all the query chunking in there, using an Id variable startFromId, the query using ORDER BY Id and LIMIT, a WHERE clause (when startFromId is non-null, i.e. from 2nd chunk) of AND Id > :startFromId, and when chaining the queueable passing the last queried ID as startFromId? 3. Don't use labels to hold settings; custom settings or CMT instead. Commented Nov 8, 2024 at 16:40
  • On point 2, are you trying to run the chunk processing concurrently? That would be a reasonable thing to consider, but then you would probably want the invocable method to work out the sets of records to be passed to the various queueable instances... just remember you are then limited to at most 50 queueables being enqueued (as long as your flow runs in a synchronous transaction). If your flow runs in an async transaction you can only enqueue a single queueable. Commented Nov 8, 2024 at 16:43
  • 1
    Consider also using chained queueables where each queueable does exactly one input, removes from the list, and then enqueues itself with the remaining inputs. This can make your logic so much simpler. If the IP is likely to throw errors, consider having the invocable stuffing each request into a custom object that an async request manager processes (aka Command Objects). This will give you replay abilities Commented Nov 8, 2024 at 18:43

0

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.