Most efficient way of handling high volumes of promises?

Question

What is the most efficient way of handling high volumes of promises? I've come up with 2 solutions and determined that Solution 2 (which uses bluebird's promise.map) is faster.

Solution 1 (~38ms per file)

readFile(file) { return new Promise((resolve, reject) => { jsmediatags.read(file, { onSuccess: resolve, onError: reject }) }) } async readFilesHandler() { console.time('readFilesHandler timer') const fileArray = Array.from(this._fileSelectInput.files) const tracksArray = [] for (let file = 0; file < fileArray.length; file++) { await this._readFile(fileArray[file]).then(tags => { tracksArray.push({ id: file + 1, title: tags.tags.title || undefined, artist: tags.tags.artist || undefined, album: tags.tags.album || undefined, year: tags.tags.year || undefined }) }) } this.dispatchEvent(new CustomEvent('tracks-selected', { detail: tracksArray })) console.time('readFilesHandler timer') // ~38ms/file }

Solution 2 (~32ms per file)

_readFiles() { console.time('_readFiles timer') const fileArray = Array.from(this._fileSelectInput.files) window.Promise.map(fileArray, file => { return new Promise((resolve, reject) => { jsmediatags.read(file, { onSuccess: resolve, onError: reject }) }) }, { concurrency: 5 }).then(tags => { const results = tags.map((tag, index) => ({ id: index + 1, title: tag.tags.title || undefined, artist: tag.tags.artist || undefined, album: tag.tags.album || undefined, year: tag.tags.year || undefined })) this.dispatchEvent(new CustomEvent('tracks-selected', { detail: results })) }) console.timeEnd('_readFiles timer') // ~32ms/file }

Are there ways of achieving the same result that are even more performant?

You're not bound by a "high volume of promises". The huge difference between your solutions is that the first reads all file tags sequentially while the second reads 5 at once, being obviously faster. How many files did you select? — Bergi
– Bergi, Commented Nov 23, 2019 at 20:43
Then you could try raising the concurrency setting even higher and should see some further improvements (although not linearly). — Bergi
– Bergi, Commented Nov 23, 2019 at 20:52
@Bergi how can I determine the maximum concurrency value I can use before I start running into memory issues? — jordan
– jordan, Commented Nov 23, 2019 at 20:55

hsjeevan · Accepted Answer · 2021-06-20 19:29:48Z

You could use Promise.allSettled()

Entire Promise array is resolved as fast as the slowest individual promise is resolved.

const bucket = await Promise.allSettled(fileArray.map(file => this._readFile(file))) bucket.forEach(file => { if (file.value) { const tags = file.value; tracksArray.push({ id: file + 1, title: tags.tags.title || undefined, artist: tags.tags.artist || undefined, album: tags.tags.album || undefined, year: tags.tags.year || undefined }) } })

Collectives™ on Stack Overflow

Most efficient way of handling high volumes of promises?

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related