flyscrape
Flyscrape is a command-line web scraping tool designed for those without advanced programming skills.
MPL-2.0 License
Bot releases are visible (Hide)
Changelog
- 92baa16 Improve cookie support for browser mode
Published by github-actions[bot] 8 months ago
Changelog
- 6cc19d5 Browser rendering
- 3dc39e9 Fix typo in example config
- e449cbe Improve file watcher
- ba009a9 Remove the setup function while we can
- e2e0ed8 Use the cookie store from local browser
Published by github-actions[bot] 8 months ago
Changelog
- 76b3fa8 Create curl installer
- c460e48 Fix race condition when writing JSON output
- f4a69b7 Increase url queue size
- c796f41 Make urls more fault tolerant
- 0b4c723 Mark non 2XX status code as errors
- 80e24f0 Respect multiple urls in domain- and urlfilter
Published by github-actions[bot] 8 months ago
Changelog
- faefb42 Add 403 Forbidden as retryable HTTP status code
Published by github-actions[bot] 8 months ago
BREAKING CHANGE
- The
rateconfiguration option changed from requests per second to request per minute.
Changelog
- 0d6494d Add retry module and change rate to requests per minute (#37)
- 8ad9f9f Add single proxy support (#30)
- 60139e7 Fix addition of invalid proxy url
Published by github-actions[bot] 9 months ago
Changelog
- 8b7425e Add version command (#29)
Published by github-actions[bot] 11 months ago
Changelog
- cbdbbd2 Add custom request header (#18)
- 807b9a1 Add missing test in TestJSLibHTTPDownload (#22)
- 13322ed Gracefully handle NaN and Inf values (#21)
- 8c68e0e Make output file and format configurable (#24)
- 0f9e334 Support file imports for .txt and .json (#25)
Published by github-actions[bot] 11 months ago
Changelog
- 94da929 Add file download functionality
- beadfd1 Add multiple starting URLs
- 47d58e6 Add support for CLI float parsing
- 190056e Move docs to flyscrape.com (#11)
- 6aa52bd Support passing config options as CLI arguments (#15)
- 3e01902 Update documentation
Published by github-actions[bot] 11 months ago
Changelog
- eae1042 Enable cgo compilation in goreleaser (#7)
Published by github-actions[bot] 11 months ago
Changelog
- 2bfae5b Add examples
- aadd80b Add login functionality
- 135eff4 Avoid caching of rate limited or errored paged
- d209006 Refactor stdlib to more generic functions
- 2d3cd65 Refactor to prepare for builtin JS functions
- 36374b8 Remove unused import from example
- 72f8346 Update readme
Published by github-actions[bot] almost 1 year ago
Changelog
- 03b3be0 Add HTTP(S) Proxy support
- 11d73f5 Allow configuration of links to follow
Published by github-actions[bot] about 1 year ago
Changelog
- 3684620 Create release pipeline
- 21c3539 Reword flyscrape description
- a8316d6 Update README.md
Published by philippta about 1 year ago
Full Changelog: https://github.com/philippta/flyscrape/commits/v0.2.0
Published by philippta about 1 year ago
Full Changelog: https://github.com/philippta/flyscrape/commits/v0.1.0