-1

I have an array of Promises that should be run in parallel and be finished as soon as possible. We have a Promise.all(arrayOfPromises) for that. But I'd like to have a kind of a modified Promise.all that can work with a dynamic array. Let me expalain what I mean.

I start with an array of 215 URLs (urlsArray) that I should send a request to. But I can process only 100 requests at a time. So I take the first 100 URLs and create a fetchArray array of those 100 fetch'es and then pass it to Promise.all(fetchArray). Now let's say one of the Promises resolved. I'd like to remove it from the fetchArray and then put the 101st fetch from urlsArray into fetchArray. I could use Promise.race() for this. But it would interrupt all other requests so I would lose all the progess. And it's not what I want since I have to get responses form all 215 URLs as soon as possible.

Also while processing the current requests, new URLs could be added to the urlsArray array. Potentially the urlsArray could be endless and new URLs could be added to the array again and again. So fetchArray should replace the just resolved Promise with the next one from urlsArray (if any) over and over again without losing the progress of other requests.

Is it even possible to implement this? Could you please give me any ideas, hints or code? Maybe some articles or tutorials about how to implement this?

Improve this question
3
  • 1
    "Promises that should be run in parallel": Promises don't run. They are objects, but not functions. Commented Jan 12, 2023 at 19:07
  • Is an Async task manager with maximum number of concurrent "running" tasks what you are looking for? Commented Jan 12, 2023 at 19:09
  • 1
    I know. I say that just to make it sound simplier. "Promises should be pending in parallel" sounds a bit ugly although technically correct Commented Jan 12, 2023 at 19:10

1 Answer 1

Reset to default
1

A code like that should work:

(async() => { const promises = [] // add 10 promises that will resolve between 1 and 4 seconds for (let i = 0; i < 10; i += 1) { addPromise(wait(Math.random() * 3000 + 1000)) } while (promises.length) { // wait for promise to resolve const result = await Promise.race(promises.map(p => p.promise)) //remove resolved promise promises.splice(promises.findIndex(p => p.id === result.id), 1) console.log(result.val) // add new promise addPromise(wait(Math.random() * 3000 + 1000)) console.log('promises left', promises.length) } function addPromise(promise) { const id = crypto.randomUUID() const newPromise = promise.then(val => ({ id, val })) promises.push({ id, promise: newPromise }) } })() // create a promise that resolves after some time function wait(time) { return new Promise(r => setTimeout(() => r(time), time)) }

Although, I would rather use rxjs for this

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Why crypto.randomUUID()? Just use a counter.
@Bergi what's wrong with crypto.randomUUID()? Why is counter better?
Simpler, smaller, and much faster

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.