baha-crawler is a web crawler module designed to scarp data from Bahamut Forum.
MIT License
baha-crawler is a web crawler module designed to scarp data from Bahamut Forum.
Bahamut Forum is the most famous and biggest game forum in Taiwan and game plays are well-know forum. Just search a while, Bahamut Forum crawler modules are not easy to be found especially written by javascript. In order to scrap data from Bahamut Forum by Node.js, I just create a simple Bahamut Forum crawler module by javascript and share it to everyone to use.
Support to scrape pages in one time
Support to skip fixed upper posts
npm install @waynechang65/baha-crawler
const baha_crawler = require('@waynechang65/baha-crawler');
// *** Initialize ***
await baha_crawler.initialize();
// *** GetResult ***
let baha = await baha_crawler.getResults({
board: '23805',
pages: 3,
skipTPs: true
}); // ToS Board(23805), 3 pages, skip fixed upper posts
// *** Close ***
await baha_crawler.close();
{ titles[], urls[] }
git clone https://github.com/WayneChang65/baha-crawler.git
cd baha-crawler
Install dependencies in the cloned baha-crawler folder
npm install
npm run start
options.board: , board name of baha options.pages: , pages options.skipTPs: , skip fixed upper posts or not
baha-crawler (bug)Issue Pull Request:)
Even though baha-crawler is a small project, I hope it can be improving. If there is any issue, please comment and welcome to fork and send Pull Request. Thanks. :)