2

I have a scheduled batch job that in the start method, I make a REST API call to an external service. The external API has around ~2000 total results to return, but the API only returns 10 records per page. I am getting an apex error stating "Too many callouts 101".

public class CustomerBatchAPI implements Database.Batchable<SObject>, Database.Stateful, Database.AllowsCallouts{ public List<JobApplication__c> start(Database.BatchableContext bc) { public static APISettings__c APICustomSettings = APISettings__c.getInstance('API Integration'); List<JobApplication__c> ja = new List<JobApplication__c>(); List<JobApplication__c> jobAppList = ExternalCustomerAPI.synchJobApps(APICustomSettings.API_Key__c); ja.add(jobAppList); return ja; } }

Is there a way to somehow avoid this error given the number of records I need to retrieve and pages I need to paginate through? For every page in the dataset, I am making a new API call so when there are 2000 records or more I always hit this apex limit.

Improve this question

1 Answer 1

Reset to default
3

You can't break the limit, so your options are basically to either do the callout in the execute method, or use Queueable so you can call it repeatedly until all pages are retrieved:

public class MyQueueable implements Queueable, Database.AllowsCallouts { ExternalCustomerAPI syncPoint = new ExternalCustomerAPI(APISettings__c.getInstance('API Integration').API_Key__c); public void execute(QueueableContext context) { JobApplication__c[] jobApps = new JobApplication__c[0]; while(Limits.getLimitCallouts() > Limit.getCallouts() && !syncPoint.isDone()) { jobApps.addAll(syncPoint.getNextPage()); } insert jobApps; if(!syncPoint.isDone()) { System.enqueueJob(this); } } }

You'll need to do some reworking of your API code so that it can be interrupted (e.g. returns just one page at a time).

Improve this answer
2
  • I am confused about what "syncPoint" represents. The custom settings line I have is just to get the API key used for authentication. Can you elaborate on the methods you are using: isDone() and getNextPage()? Are these documented or more for illustration purposes @sfdcfox Commented Mar 17, 2021 at 0:00
  • @user1669296 They're mostly illustrative, as I don't know what your code is actually doing in the code you're calling now. What you need is a pagination variable and something to track if you hit the last page, which I presume you're already able to determine. In other words, you need to break apart the black box so it can be used one page at a time. You might even move the authentication directly into the class doing the callout, keep all the logic in one place. Commented Mar 17, 2021 at 1:49

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.