NodeJs + PhantomJs return JQuery result from page.evaluate method

Question

NodeJS newbie here. I am trying to parse html with using NodeJS and PhamtomJS (phantomjs-node). When I run the the JQuery $("[class*='question-summary']") from the browser console it returns an array. However, I couldn't manage to do the same thing on nodejs. I guess stackoverflow has JQuery so I don't need to use includeJs to load jquery. Actually, when I run

Here is the nodejs example I am running;

var phantom = require('phantom'); async function getHtml() { const instance = await phantom.create([ "--load-images=false" ]); const page = await instance.createPage(); await page.on("onResourceRequested", function(requestData) { console.info('Requesting', requestData.url) }); const status = await page.open('http://stackoverflow.com'); console.log("STATUS: " + status); const content = await page.property('content'); console.log(content); var result = await page.evaluate(function(content) { return $("[class*='question-summary']"); }); console.log("Result : " + result); await instance.exit(); }; getHtml();

I run with the command >node --harmony-async-await phantomTest.js. And the process gets stuck after printing content to console.

I suppose node.js is a hard requirement? it's just that doing this in PhantomJS would be soooo much simpler. — Vaviloff
– Vaviloff, Commented Mar 23, 2017 at 5:55
it is not a hard a requirement but I will try to implement a cron job for parsing and persisting some data. and I want to learn nodejs. — quartaela
– quartaela, Commented Mar 23, 2017 at 6:13
I just wanted to note that you can write scripts spicifically for PhantomJS, run them from cron and they will be MUCH simpler, without async/await/promises stuff. — Vaviloff
– Vaviloff, Commented Mar 23, 2017 at 9:42
thanks @Vaviloff I found a simple way to return something from evaluate function — quartaela
– quartaela, Commented Mar 26, 2017 at 3:59

quartaela · Accepted Answer · 2017-03-26 03:58:52Z

Answering my own question here. Creating an array inside evaluate function and pushing elements inside worked. I guess the only limitation is phantom-node just supports returning objects with primitives.

var result = await page.evaluate(function() { var questionSummaries = []; $("[class*='question-summary']").each(function() { questionSummaries.push(this.innerHTML); }); return questionSummaries; });

Collectives™ on Stack Overflow

NodeJs + PhantomJs return JQuery result from page.evaluate method

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related