scrapy-wayback-machine

A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.

ISC License

Downloads
649
Stars
109
Committers
3
View Code on GitHub
Ecosystems: Python

Commit Statistics

Past Year

All Time

Total Commits
48
Total Committers
3
Avg. Commits Per Committer
16.0
Bot Commits
0

Issue Statistics

Past Year

All Time

Total Pull Requests
1
8
Merged Pull Requests
0
4
Total Issues
1
4
Time to Close Issues
N/A
3 months
Package Rankings
Top 8.42% on Pypi.org
Related Projects
waybackpack

Download the entire Wayback Machine archive for a given URL.

11 Apr 2016 2,862
djangoscraper

Django Scrapy App

26 Aug 2009 17
OpenScraper

An open source webapp for scraping: towards a public service for webscraping

20 Feb 2018 92
WebScrapper_Mastodon

The Mastodon Social Platform Scraper is a Python-based web scraping tool designed to explore and ...

04 Feb 2024 0
scraping_tutorial

Basics of scraping with python, requests, beautifulsoup4, selenium, etc.

17 Oct 2019 1
waymore

Find way more from the Wayback Machine, Common Crawl, Alien Vault OTX, URLScan & VirusTotal!

24 Jun 2022 1,675
rsstimemachine

Find archived RSS feeds on archive.org

07 Feb 2014 6
scrapy-scraper

Web crawler and scraper based on Scrapy and Playwright's headless browser.

13 Apr 2023 9
autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

31 Aug 2020 6,197
webscraping-from-0-to-hero

The web scraping open project repository aims to share knowledge and experiences about web scrapi...

26 May 2022 1,533
scrapy-wayback-middleware

Scrapy middleware for submitting URLs to the Internet Archive Wayback Machine

25 Feb 2019 10
scrappy

scrapy best practice

02 Mar 2016 37
GoogleScraper

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). In...

06 Dec 2013 2,630
wayback-machine-scraper

A command-line utility and Scrapy middleware for scraping time series data from Archive.org's Way...

04 Apr 2017 416
logparser

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API o...

20 Jan 2019 88