An asyncio based parallel file downloader for Python 3.8+
MIT License
Bot releases are hidden (Show)
Published by Cadair over 2 years ago
This new major version of parfive tackles the usability of the API.
The API of parfive grew organically as support for more use cases were added.
To enable both a highly flexible downloader that can be used for any HTTP/FTP downloads while maintaining an accessible and simple API with the 2.0 release there is now a parfive.SessionConfig
object where advanced settings are located.
The other major change in this release is that any files where the download failed before it is completed will be removed from disk. This means that you shouldn't end up with incomplete and corrupt files on disk if the download is interrupted for any reason.
file_progress
, notebook
, use_aiofiles
, and header
keyword arguments to parfive.Downloader
have been removed and moved into the parfive.SessionConfig
object.timeouts
keyword argument has been removed from the run_download()
and download()
methods and replaced by an option in SessionConfig
.SessionConfig
object to allow advanced configuration of the Downloader
by @Cadair in https://github.com/Cadair/parfive/pull/92
PARFIVE_HIDE_PROGRESS
environment variable will disable all progress bars.Downloader
class is instantiated (by the new config classes).aiohttp.ClientSession
by passing a function to the aiohttp_session_generator=
keyword argument to SessionConfig
.SessionConifg
where previously it could only be configured with an environment variable.SessionConfig
and the default when not using aiofiles has been increased from 100 bytes to 1024 bytes.SessionConfig
as well as the HTTP[S]_PROXY
environment variables.aiohttp.ClientTimeout
object to SessionConfig
.asyncio
on Windows.Full Changelog: https://github.com/Cadair/parfive/compare/v1.5.1...v2.0.0
Published by Cadair over 2 years ago
Published by Cadair over 3 years ago
parfive.Downloader.run_download
public API and move all of the parfive.Downloader.download
logic into run_download
. This enabled using parfive from inside an async context. (#54)loop
to Downloader
now does nothing, and parfive spins up its own loop. This is to prepare for python 3.10 where support for passing custom loops to various asyncio
methods will be removed. (#53)max_conn
is changed after construction of the parfive.Downloader
instance. (#56)Published by Cadair over 4 years ago
max_splits
to parfive.Downloader.enqueue_file. (#15)loop=
keyword argument to aiohttp.ClientSession.Published by Cadair over 4 years ago
Fixes a bug with detection of Jupyter notebooks and display of tqdm notebook progress bars.
Published by Cadair over 5 years ago
Published by Cadair over 5 years ago
Published by Cadair over 5 years ago
Published by Cadair over 6 years ago