dude
dude uncomplicated data extraction: A simple framework for writing web scrapers using Python decorators
AGPL-3.0 License
Published by roniemartinez over 2 years ago
What's Changed
- 🐛 Fix incompatible types when sorting by @roniemartinez in https://github.com/roniemartinez/dude/pull/48
Full Changelog: https://github.com/roniemartinez/dude/compare/0.4.0...0.4.1
Published by roniemartinez over 2 years ago
What's Changed
- ✨ Add group() decorator by @roniemartinez in https://github.com/roniemartinez/dude/pull/45
- ✨ Add new parameters to select() by @roniemartinez in https://github.com/roniemartinez/dude/pull/47
- 📚 Add link to project documentation by @roniemartinez in https://github.com/roniemartinez/dude/pull/43
- 🔒 Add CodeQL by @roniemartinez in https://github.com/roniemartinez/dude/pull/44
- ⬆️ Bump mkdocs-material from 8.2.3 to 8.2.4 by @dependabot in https://github.com/roniemartinez/dude/pull/46
Full Changelog: https://github.com/roniemartinez/dude/compare/0.3.0...0.4.0
Published by roniemartinez over 2 years ago
What's Changed
- ⬆️ Bump playwright from 1.19.0 to 1.19.1 by @dependabot in https://github.com/roniemartinez/dude/pull/40
- 📚 MkDocs project documentation by @roniemartinez in https://github.com/roniemartinez/dude/pull/41
Documentation: https://roniemartinez.github.io/dude/
Full Changelog: https://github.com/roniemartinez/dude/compare/0.2.0...0.3.0
Published by roniemartinez over 2 years ago
What's Changed
- ✨ Option to use BeautifulSoup4 (with HTTPX) by @roniemartinez in https://github.com/roniemartinez/dude/pull/32
- 📚 Update examples by @roniemartinez in https://github.com/roniemartinez/dude/pull/28
- 💚 Run CI on specific paths by @roniemartinez in https://github.com/roniemartinez/dude/pull/30
- 📚 Update README by @roniemartinez in https://github.com/roniemartinez/dude/pull/31
- 💚 Coverage and shields by @roniemartinez in https://github.com/roniemartinez/dude/pull/35
- 🐛 Fix PyPI workflow name by @roniemartinez in https://github.com/roniemartinez/dude/pull/36
- 📚 Update README for BeautifulSoup4 by @roniemartinez in https://github.com/roniemartinez/dude/pull/39
Full Changelog: https://github.com/roniemartinez/dude/compare/0.1.0...0.2.0
Published by roniemartinez over 2 years ago
What's New
- Simple Flask-inspired design - build a scraper with decorators.
- Uses Playwright's API - run your scraper in Chrome, Firefox and Webkit and leverage Playwright's powerful selector engine.
- Data grouping - group related scraping data.
- URL pattern matching - run functions on specific URLs.
- Priority - reorder functions based on priority.
- Setup function - enable setup steps (clicking dialogs or login).
- Navigate function - enable navigation steps to move to other pages.
- Custom storage - option to save data to other formats or database.
- Async support - write async handlers.
Full Changelog: https://github.com/roniemartinez/dude/commits/0.1.0