s3do

A collection of S3 commands for bulk operations.

• Installation
• Usage
• Commands
  • Inventory
  • Tag

Installation

Pull the image from DockerHub and run the container:

$ docker run robvanderleek/s3do

You can also clone this repository, install the dependencies (using virtualenv) and run the Python scripts:

$ git clone https://github.com/robvanderleek/s3do.git
$ cd s3do
$ virtualenv -p python3 venv
$ . venv/bin/activate
$ pip install -r requirements.txt
$ ./s3do.py

Usage

Running the robvanderleek/s3do docker container is the easiest way to use the toolset. You need to supply your AWS credentials to the container, there are two options for that:

1. Mounting a local .aws credentials folder:

$ docker run -v $HOME/.aws:/root/.aws robvanderleek/s3do inventory archpi.dabase.com 

2. Passing credentials through enviornment variables:

$ docker run -e AWS_ACCESS_KEY_ID='...' -e AWS_SECRET_ACCESS_KEY='...' -e AWS_SESSION_TOKEN='...' robvanderleek/s3do inventory archpi.dabase.com

Commands

Inventory

An inventory file contains on each line a comma separated pair of "<bucket name>","<object key>. For example:

"MyBucket","path/to/file.txt"
"MyBycket","path/to/another/file.txt"

To create an inventory file for all objects in a bucket run:

$ docker run -v $HOME/.aws:/root/.aws robvanderleek/s3do inventory archpi.dabase.com 

Create an inventory file for all objects in a bucket that match the images prefix:

$ docker run -v $HOME/.aws:/root/.aws robvanderleek/s3do inventory archpi.dabase.com images

Tag

Tag all objects in a bucket with the tag foo=bar:

$ docker run -v $HOME/.aws:/root/.aws robvanderleek/s3do tag --tag foo=bar archpi.dabase.com 

Tag all objects that match the images prefix with the tags foo=bar and spam=eggs:

$ docker run -v $HOME/.aws:/root/.aws robvanderleek/s3do tag --tag foo=bar --tag spam=eggs inventory archpi.dabase.com images