0

I have written a routine for queueing a set of http requests so that they happen in sequence instead of concurrently. I am using this async library: https://caolan.github.io/async/v3/. This is running in a browser in a Meteor/React stack.

Normally this code executes correctly, but maybe 1-2% of the time the first task in the queue returns, and the queue fails to call its callback and fails to process subsequent tasks in the queue. The error callback never fires, and the queue never empties.

Are there known errors with this library? Is there some error in my code? Are there any approaches for troubleshooting what is happening to freeze the queue?

import async from 'async'; const taskWorker = async requestPayload => { console.debug( 'taskWorker started' ); const result = await makeAnHttpRequest( requestPayload ); console.debug( 'taskWorker complete' ); return result; }; console.debug( 'creating async queue.' ); const taskQueue = async.queue( taskWorker, 1 ); taskQueue.drain( () => { console.debug( 'all items have been processed' ); } ); taskQueue.error( ( err, task ) => { console.error( 'task experienced an error' ); } ); export const requestItem = requestPayload => { console.debug( 'requestItem started' ); return new Promise( ( resolve, reject ) => { console.debug( 'requestItem promise started' ); taskQueue.push( requestPayload, ( error, result ) => { console.debug( 'taskQueue callback' ); if ( error ) { reject( error ); } else { resolve( result ); } } ); } ); }; 

Typical output:

creating async queue. requestItem started requestItem started requestItem started requestItem promise started requestItem promise started requestItem promise started taskWorker started taskWorker complete taskQueue callback taskWorker started taskWorker complete taskQueue callback taskWorker started taskWorker complete taskQueue callback all items have been processed 

But in some conditions I see this output instead, and it stays like this indefinitely.

 creating async queue. requestItem started requestItem started requestItem started requestItem promise started requestItem promise started requestItem promise started taskWorker started taskWorker complete 

I would expect that the queue would either proceed to the callback for the first task, or it would print 'task experienced an error', but it does neither!

5
  • Since your taskQueue.push is in the promise, what happens if there's a problem when pushing the new requestPayload to the queue ? Could there be an error which would make the promise resolve without ever passing into the reject callback ? Commented Jan 24, 2024 at 16:50
  • Hi @Nicolas - I don't think that is possible. The console.debug( 'requestItem promise started' ); line fires and the next step pushes the task to the queue. (a) I don't see how there could be an error, and (b) I would expect to see something like "Uncaught Exception" in the terminal if there was. Commented Jan 24, 2024 at 16:56
  • 1
    From experience, " I don't see how there could be an error" is never a good reason not to check. You could add a try { } catch {} around the taskQueue.push and reject the promise in the catch block. Without a minimal reproducible example, it's very hard for us to understand the real issue. Commented Jan 24, 2024 at 17:00
  • Thanks @Nicolas. Typically it takes days to produce one failure, so this is the closest I've got to a reproducible example at this point. I'll try the try/catch Commented Jan 24, 2024 at 17:03
  • 1
    We decided to roll our own async queue, based on stackoverflow.com/a/63208885/2496827. It is working like a charm. Commented Feb 1, 2024 at 17:31

0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.