0

I'm building a web scraper using spookyjs: https://github.com/WaterfallEngineering/SpookyJS

I created a new spooky object and I'm trying to evaluate a function that returns the contents of a certain html tag. For the sake of readability I'm trying to use a function (returnDataFromSelector()) to do this each time. However when I pass the function through evaluate it returns null.

When I use an anonymous function it works correctly. How can I get the returnDataFromSelector function work correctly?

 spooky.then(function() { function returnDataFromSelector(selector) { return function () { return document.querySelectorAll(selector)[0].innerHTML; } } var pageData = {}; pageData.projectName = this.evaluate(returnDataFromSelector('a.green-dark')); // returns null pageData.projectName = this.evaluate(function () { return document.querySelectorAll('a.green-dark')[0].innerHTML;}); // returns correct string this.emit('pageData', pageData); }); 

1 Answer 1

1

The selector variable is not in the correct context. You should pass the selector separately into spooky.evaluate:

spooky.then(function() { function returnDataFromSelector(selector) { return document.querySelectorAll(selector)[0].innerHTML; } var pageData = {}; pageData.projectName = this.evaluate(returnDataFromSelector, 'a.green-dark'); pageData.projectName = this.evaluate(function () { return document.querySelectorAll('a.green-dark')[0].innerHTML; }); this.emit('pageData', pageData); }); 
Sign up to request clarification or add additional context in comments.

1 Comment

Thanks, this should work. However it gives back an empty string now.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.