scrapy-examples

Multifarious Scrapy examples. Spiders for alexa / amazon / douban / douyu / github / linkedin etc.

Stars
3.2K
View Code on GitHub View on X
Ecosystems: Python

Issue Statistics

Past Year

All Time

Total Pull Requests
2
14
Merged Pull Requests
0
7
Total Issues
1
18
Time to Close Issues
N/A
about 1 year
Related Projects
OpenScraper

An open source webapp for scraping: towards a public service for webscraping

20 Feb 2018 92
scraping_tutorial

Basics of scraping with python, requests, beautifulsoup4, selenium, etc.

17 Oct 2019 1
webscraping-from-0-to-hero

The web scraping open project repository aims to share knowledge and experiences about web scrapi...

26 May 2022 1,533
transistor

Transistor, a Python web scraping framework for intelligent use cases.

12 Nov 2018 212
quotesbot

This is a sample Scrapy project for educational purposes

27 Sep 2016 1,296
ParamSpider

Mining URLs from dark corners of Web Archives for bug hunting/fuzzing/further probing

12 Apr 2020 2,471
scrapy_rss

Tools to easy generate RSS feed that contains each scraped item using Scrapy framework.

02 Feb 2017 30
anime_spiders

A collection of self-using anime-related crawlers.

16 Jan 2017 0
boris-spider

boris-spider是一款使用Python语言编写的爬虫框架，于多年的爬虫业务中不断磨合而诞生，相比于scrapy，该框架更易上手，且又满足复杂的需求，支持分布式及批次采集。

22 Apr 2020 82
PythonSpiderNotes

Python入门网络爬虫之精华版

19 Aug 2015 6,877
ECommerceCrawlers

实战🐍多种网站、电商数据爬虫🕷。包含🕸：淘宝商品、微信公众号、大众点评、企查查、招聘网站、闲鱼、阿里任务、博客园、微博、百度贴吧、豆瓣电影、包图网、全景网、豆瓣音乐、某省药监局、搜狐新闻、机器学...

29 Mar 2019 4,682
scrappy

scrapy best practice

02 Mar 2016 37
GoogleScraper

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). In...

06 Dec 2013 2,630
SpiderKeeper

admin ui for scrapy/open source scrapinghub

18 Jan 2016 2,738
Spider

web crawler

27 Feb 2018 36