CasperJS looping through each URL

Question

This question is similar to others, but the problem I had was more basic.

This is my code:

var links = []; var casper = require('casper').create(); function getLinks() { var links = document.querySelectorAll('div#mw-content-text table.wikitable tbody tr td i b a'); return Array.prototype.map.call(links, function(e) { return 'https://en.wikipedia.org' + e.getAttribute('href'); }); } casper.start('https://en.wikipedia.org/wiki/David_Bowie_discography'); casper.then(function() { // aggregate results for the 'casperjs' search links = this.evaluate(getLinks); }); casper.each(links, function (self, link) { self.thenOpen(fullURL, function () { this.echo(this.getTitle() + " - " + link); }); }); casper.run();

I know that links get created as it is copied from the Quickstart, but I then modified it to open all the links that were found.

What I'm getting is that nothing is echo'd instead of outputting the each title which is what I expect. This is how I'm calling the file:

~ $ casperjs casper-google-disco.js

Community · Accepted Answer · 2017-05-23 12:09:53Z

The fix was very easy in the end, but took me ages to find it as there were no errors and no-one else seemed to have hit this.

The problem is that the links variable doesn't get set before the each is called. Putting the each inside the then function solves my problem.

The each.js example in the CasperJS samples was helpful for confirming that you can loop through an array without any need for IIFE.

var links = []; var casper = require('casper').create(); function getLinks() { var links = document.querySelectorAll('div#mw-content-text table.wikitable tbody tr td i b a'); return Array.prototype.map.call(links, function(e) { return 'https://en.wikipedia.org' + e.getAttribute('href'); }); } casper.start('https://en.wikipedia.org/wiki/David_Bowie_discography'); casper.then(function() { // aggregate results for the 'casperjs' search links = this.evaluate(getLinks); casper.each(links, function (self, link) { self.thenOpen(link, function () { this.echo(this.getTitle() + " - " + link); }); }); }); casper.run();

Collectives™ on Stack Overflow

CasperJS looping through each URL

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related