Versions Supported

Language Stats

Repo Stats

Activity

Build Status

S3 Download

Python module to download objects in an S3 bucket.

Installation

pip install s3-downloader

Usage

Download objects in parallel

import s3

if __name__ == '__main__':
    wrapper = s3.Downloader(bucket_name='BUCKET_NAME')
    wrapper.run_in_parallel(threads=10)  # Defaults to 5

Download objects in sequence

import s3

if __name__ == '__main__':
    wrapper = s3.Downloader(bucket_name='BUCKET_NAME')
    wrapper.run()

Mandatory arg

• bucket_name - Name of the s3 bucket.

Optional kwargs

• prefix - Prefix to filter the objects based on their path. Defaults to None
• logger - Bring your own custom pre-configured logger. Defaults to on-screen logging.
• download_dir - Name/path of the directory where the objects have to be stored.
  Defaults to bucket_name at current working directory.
  
• region_name - AWS region name. Defaults to the env var AWS_DEFAULT_REGION
• profile_name - AWS profile name. Defaults to the env var PROFILE_NAME
• aws_access_key_id - AWS access key ID. Defaults to the env var AWS_ACCESS_KEY_ID
• aws_secret_access_key - AWS secret access key. Defaults to the env var AWS_SECRET_ACCESS_KEY

AWS values are loaded from env vars or the default config at ~/.aws/config / ~/.aws/credentials

Coding Standards

Docstring format: Google Styling conventions: PEP 8 Clean code with pre-commit hooks: flake8 and isort

Release Notes

Requirement

python -m pip install gitverse

Usage

gitverse-release reverse -f release_notes.rst -t 'Release Notes'

Linting

pre-commit will ensure linting, run pytest, generate runbook & release notes, and validate hyperlinks in ALL markdown files (including Wiki pages)

Requirement

pip install sphinx==5.1.1 pre-commit recommonmark

Usage

pre-commit run --all-files

Pypi Package

https://pypi.org/project/s3-downloader/

License & copyright

© Vignesh Rao

Licensed under the MIT License