Remote web content crawler done right.
Sometimes I want to grab some nice images from a url like http://bbs.005.tv/thread-492392-1-1.html, so I made this little program to combine node-fetch
and cheerio
to make my attempt fulfilled.
$ npm install --save recrawler
For Single Page Apps please head to recrawler-spa
const recrawler = require('recrawler')
recrawler('http://some-url.com/a/b/c')
.then($ => {
$('img.nice-images').each(function () {
const url = $(this).attr('src')
console.log(url)
})
})
cheerio options. Except decodeEntities
is false
by default here.
MIT © EGOIST