scrapy-redis

What's Changed

• docs: fix readthedocs by @rmax in https://github.com/rmax/scrapy-redis/pull/299

Full Changelog: https://github.com/rmax/scrapy-redis/compare/v0.9.0...v0.9.1

What's Changed

• Removed unused import. by @sourcepirate in https://github.com/rmax/scrapy-redis/pull/292
• build: use python3.12 as default version by @rmax in https://github.com/rmax/scrapy-redis/pull/297
• maint: precommit format, fixes and update copyright by @rmax in https://github.com/rmax/scrapy-redis/pull/298
• fix: Scheduler not compatible with BaseDupeFilter by @HairlessVillager in https://github.com/rmax/scrapy-redis/pull/294

New Contributors

• @sourcepirate made their first contribution in https://github.com/rmax/scrapy-redis/pull/292
• @HairlessVillager made their first contribution in https://github.com/rmax/scrapy-redis/pull/294

Full Changelog: https://github.com/rmax/scrapy-redis/compare/v0.8.0...v0.9.0

What's Changed

• [settings] Deprecate REDIS_START_URLS_BATCH_SIZE by @LuckyPigeon in https://github.com/rmax/scrapy-redis/pull/259
• [spiders] Remove duplicate check setting types by @LuckyPigeon in https://github.com/rmax/scrapy-redis/pull/261
• fix(sec): upgrade scrapy to 2.6.0 by @lifefloating in https://github.com/rmax/scrapy-redis/pull/270
• [dev] Deprecated function scrapy.utils.request.request_fingerprint() warning by @LuckyPigeon in https://github.com/rmax/scrapy-redis/pull/275
• [dev] Add customize fingerprint by @LuckyPigeon in https://github.com/rmax/scrapy-redis/pull/280
• [spiders] Remove 'spider' argument to ExecutionEngine.crawl by @mic1on in https://github.com/rmax/scrapy-redis/pull/286
• Fix test requirements and simplified tox config by @rmax in https://github.com/rmax/scrapy-redis/pull/295

New Contributors

• @lifefloating made their first contribution in https://github.com/rmax/scrapy-redis/pull/270
• @mic1on made their first contribution in https://github.com/rmax/scrapy-redis/pull/286

Full Changelog: https://github.com/rmax/scrapy-redis/compare/0.7.3...v0.8.0

Release patch version 0.7.23:

• Move docs to GitHub Wiki (#224)
• Update tox and support dynamic tests (#233, #234, #235, #237, #244, #246, #247)
• Update support for json data (#223, #240)
• Refactor max idle time (#220)
• Add support for python3.7~python3.10 (#227, #244)
• Deprecate python2.x support (#227, #231)

Release patch version 0.7.2:

• Add ')' to spiders.py: https://github.com/rmax/scrapy-redis/pull/202
• Fix RedisStatsCollector._get_key(): https://github.com/rmax/scrapy-redis/pull/200
• Fix redis-py dependency version: https://github.com/rmax/scrapy-redis/pull/199
• Added maximum idle waiting time MAX_IDLE_TIME_BEFORE_CLOSE: https://github.com/rmax/scrapy-redis/pull/193
• Correction of wrongly written characters: https://github.com/rmax/scrapy-redis/pull/191

• Support priority start_urls. https://github.com/rmax/scrapy-redis/pull/172
• Support init batch tasks. https://github.com/rmax/scrapy-redis/pull/176
• Fixes datetime parsing error for redis-py 3.x. https://github.com/rmax/scrapy-redis/pull/190
• Add support for stats extensions. https://github.com/rmax/scrapy-redis/pull/186
• Support using pipeline when read redis lit queue. https://github.com/rmax/scrapy-redis/pull/169
• Support to configure different redis-server db by adding REDIS_DB https://github.com/rmax/scrapy-redis/pull/166
• Add option to decode responses in python3 #146
• Add support for making dupefilter support create from spider #83
• Fix example: update dmoz spider domain #164