x-crawl

x-crawl is a flexible Node.js AI-assisted crawler library. Making crawler work more efficient, intelligent and convenient. ------ x-crawl 是一个灵活的 Node.js AI 辅助爬虫库。使爬虫工作变得更加高效、智能和便捷。(v10 版本已发布)

MIT License

Downloads
762
Stars
829
Committers
4

Bot releases are visible (Hide)

x-crawl - v2.4.2

Published by coder-hxl over 1 year ago

🚀 Features

  • Update documents
x-crawl - v2.4.1

Published by coder-hxl over 1 year ago

🐞 Bug Fixes

  • Interval time doesn't work
x-crawl - v2.4.0

Published by coder-hxl over 1 year ago

🚀 Features

  • Update documents
x-crawl - v2.3.0

Published by coder-hxl over 1 year ago

🚨 Breaking Changes

  • fetchHTML API rename fetchPage
x-crawl - v2.2.1

Published by coder-hxl over 1 year ago

🚀 Features

  • Update documents
x-crawl - v2.2.0

Published by coder-hxl over 1 year ago

🚀 Features

  • Parameter config name correction
  • Update documents
x-crawl - v2.1.0

Published by coder-hxl over 1 year ago

🚀 Features

  • The fetchHTML API results remove content options
  • The fetchHTML API cancels the incoming header attribute
  • The running mode of crawling is highlighted
x-crawl - v2.0.0

Published by coder-hxl over 1 year ago

🚨 Breaking Changes

  • Create a crawler method, created by calling
  • The fetchHTML API uses puppeteer to crawl HTML
  • The fetchPolling API renamed to startPolling, removed year and month
x-crawl - v1.1.1

Published by coder-hxl over 1 year ago

🚀 Features

  • Add jump to a detailed type
x-crawl - v1.1.0

Published by coder-hxl over 1 year ago

🚀 Features

  • The fetchFile API uses async for save file operations and sorts errors
  • The fetchFile API fileConfig can pass in nonxisting path
x-crawl - v1.0.1

Published by coder-hxl over 1 year ago

🚀 Features

  • Sorting of error messages and fetchData/fetchFile API results
x-crawl - v1.0.0

Published by coder-hxl over 1 year ago

🚨 Breaking Changes

  • Added the method of getting the request result through the callback function, The result of each request can be obtained through the callback function, and the total result can be obtained through PromIse
  • API internal refactoring
x-crawl - v0.4.0

Published by coder-hxl over 1 year ago

🚀 Features

  • The fetchFile API config can provide the extension of the downloaded file
x-crawl - v0.3.1

Published by coder-hxl over 1 year ago

🚀 Features

  • Update documents
x-crawl - v0.3.0

Published by coder-hxl over 1 year ago

🚨 Breaking Changes

  • Rename the data object raw of fetchHTML return value to html

🚀 Features

  • Add proxy option
x-crawl - v0.2.0

Published by coder-hxl over 1 year ago

🚀 Features

  • Add polling function
x-crawl - v0.1.5

Published by coder-hxl over 1 year ago

🐞 Bug Fixes

  • Fix fetchFile API file save is lost
x-crawl - v0.1.4

Published by coder-hxl over 1 year ago

🚀 Features

  • Add chalk library
x-crawl - v0.1.3

Published by coder-hxl over 1 year ago

🚀 Features

  • fetchHTML API exposes more content
x-crawl - v0.1.2

Published by coder-hxl over 1 year ago

🚀 Features

  • Add requst mode option: async/sync
  • Document adjustment