Recursively fetching data with axios and chaining the result

Question

I have an url of the pattern http://www.data.com/1, where the 1 at the end can run up until a not predefined number. It returns an array. I need to concatenate all the arrays I get into one.

My strategy is to recursively perform a get-request, until I get a 404 error, upon which I return the result.

var returnArray = []; function getData(count){ let p = axios.get(`http://www.data.com/${count}`).then(function(response){ returnArray = returnArray.concat(response.data); return getData(count +1); } ).catch(function() { Promise.resolve(returnArray); } )};

When I implement this function, however, my result is undefined.

getData(1).then((response) => console.log(response))

What is the best way to do this?

Not sure about the best way, but I would go with async/await and a while loop which runs until it gets 404 response. — Hashem Qolami
– Hashem Qolami, Commented Apr 15, 2018 at 8:18
I guess that you return promise p from your getData function, but because you expect to get the result from catch did you try to return this also?: return Promise.resolve(returnArray); — Doron Brikman
– Doron Brikman, Commented Apr 15, 2018 at 8:33
Sorry, forgot to rename the function, getLeadData is equal to getData. — wasmachien
– wasmachien, Commented Apr 15, 2018 at 8:43

Wizard · Accepted Answer · 2018-04-15 14:37:25Z

Inside getData, you are not returning the promise, so the function immediately exits with undefined value. So, instead of:

let p = axios.get(`http://www.data.com/${count}`).then(function(response){

use:

return axios.get(`http://www.data.com/${count}`).then(function(response){

EDIT: As Bergi pointed out, you also need to return returnArray within catch handler, so instead of:

Promise.resolve(returnArray);

use:

return returnArray;

Here's the full code:

var returnArray = []; function getData(count){ return axios.get(`http://www.data.com/${count}`).then(function(response){ returnArray = returnArray.concat(response.data); return getData(count +1); } ).catch(function() { return returnArray; } )};

Redu · Accepted Answer · 2018-04-15 10:03:25Z

I believe the order of concatanation depends on the count so you have to sequence promises one after the other.

You may do recursive concatanation by increasing the cnt argument and accumulating the result in the rs argument of the sp function up until finally a promise rejects. Once rejected, since all previous resolutions are already concatenated in the rs argument, just simply return it in the catch stage.

In below code ps array resembles an API and ps[cnt] resembles a call to the API like;

axios.get(`http://www.data.com/${cnt}`);

var ps = [new Promise((v,x) => setTimeout(v,Math.random()*1000,[0,1,2])), new Promise((v,x) => setTimeout(v,Math.random()*1000,[3,4,5])), new Promise((v,x) => setTimeout(v,Math.random()*1000,[6,7,8])), new Promise((v,x) => setTimeout(x,1000,null))], sp = (cnt = 0, rs = []) => ps[cnt].then(vs => sp(++cnt, rs.concat(vs))) .catch(e => rs); sp().then(console.log);

This method of sequencing of course takes time as every other promise registeres only after the previous one gets resolved. So another way to go could be to batch a group of promise jobs in sub arrays and use these chunks with Promise.all(). Which can also be implemented in a similar recursive fashion.

Collectives™ on Stack Overflow

Recursively fetching data with axios and chaining the result

2 Answers 2

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Linked

Related