beeswithmachineguns

h4. Bees with Machine Guns!

A utility for arming (creating) many bees (micro EC2 instances) to attack (load test) targets (web applications).

Also, retribution for "this shameful act":http://kottke.org/10/10/tiny-catapult-for-throwing-pies-at-bees against a proud hive.

h2. Dependencies

• Python 2.6 - 3.6
• boto
• paramiko

h2. Installation for users

h2. Installation for developers (w/ virtualenv + virtualenvwrapper)

h2. Configuring AWS credentials

Bees uses boto to communicate with EC2 and thus supports all the same methods of storing credentials that it does. These include declaring environment variables, machine-global configuration files, and per-user configuration files. You can read more about these options on "boto's configuration page":http://code.google.com/p/boto/wiki/BotoConfig.

At minimum, create a .boto file in your home directory with the following contents:

The credentials used must have sufficient access to EC2.

Make sure the .boto file is only accessible by the current account:

h2. Usage

A typical bees session looks something like this:

A bees session where this is being called from a python file, while specifying content type and a payload file. This is a working example, all of these objects exist in the us-east-1 region.

In this case the data.json is a simple json file, mind the path.

This spins up 4 servers in security group 'public' using the EC2 keypair 'frakkingtoasters', whose private key is expected to reside at ~/.ssh/frakkingtoasters.pem.

Note: the default EC2 security group is called 'default' and by default it locks out SSH access. I recommend creating a 'public' security group for use with the bees and explicitly opening port 22 on that group.

It then uses those 4 servers to send 10,000 requests, 250 at a time, to attack OurNewWebbyHotness.com.

Lastly, it spins down the 4 servers. Please remember to do this--we aren't responsible for your EC2 bills.

If you wanted 3 agents requesting url A and one requesting url B, your attack would look as follows (empty url -> use previous):

For complete options type:

h2. Introduction to additions:

h4. Additions contributed Hurl integration and multi regional testing.

hurl is an http server load tester similar to ab/siege/weighttp/wrk with support for multithreading, parallelism, ssl, url ranges, and an api-server for querying the running performance statistics. hurl is primarily useful for benchmarking http server applications. For more information about hurl please visit https://github.com/VerizonDigital/hlx

Multi regional testing was added so user can call up multiple bees from different regions simultaneously. Users have the ability to “up”, “attack”, and “down” instances from single command. “regions.json” file is supplied which contains public ami images with hurl pre installed for all regions.

What kind of changes were made that's different from the old? Instead of writing bees information into a single ~/.bees file, each zone recognized in arguments creates a new unique bees file. Bees.py was modified to read these files. Up, attack, and down functions are run with threads.

example .bees files in user home directory

h4. Motivation

Having the ability to generate a lot of HTTPS requests from many different regions around the world allows us to better test our platforms and services. This is also real helpful when there are tools that need to be tested for such things as location of requests.

h4. Hurl Usage

h4. bees up

Command line arguments are still the same however to add multiple zones with multiple amis, the values must be comma delimited. The ami and zones must also be in same order for it to work. So for example “-i ami-zone1,ami-zone2,ami-zone3 -z zone1,zone2,zone3”.

h4. bees attack

In order to use the hurl platform, --hurl or -j must be supplied. Attacks will run concurrently and return a summarized output. The output is summarized per region. More information can be seen if user supplies the -o, --long_output options.

h4. bees down

Bringing down bees is the same and will bring down all bees for all regions

regions used: eu-west-1b,ap-southeast-1b,us-west-2b

Some options were added to work with hurl

h4. Examples

A bringing up bees example

A bees attack example

A bees attack example with --long_output

An example bees down

h2. The caveat! (PLEASE READ)

(The following was cribbed from our "original blog post about the bees":http://blog.apps.chicagotribune.com/2010/07/08/bees-with-machine-guns/.)

If you decide to use the Bees, please keep in mind the following important caveat: they are, more-or-less a distributed denial-of-service attack in a fancy package and, therefore, if you point them at any server you don’t own you will behaving unethically, have your Amazon Web Services account locked-out, and be liable in a court of law for any downtime you cause.

You have been warned.

h2. Troubleshooting

h3. EC2 Instances Out Of Sync

If you find yourself in a situation where 'bees report' seems to be out of sync with EC2 instances you know are (or are not) running:

• You can reset the BWMG state by deleting ~/.bees.. Note that this will prevent BWMG from identifying EC2 instances that may now be orphaned by the tool
• You can manually edit ~/.bees. to add or remove instance IDs and force synchronization with the reality from your EC2 dashboard

This is helpful in cases where BWMG crashes, EC2 instances are terminated outside of the control of BWMG, or other situations where BWMG is out of sync with reality.

h2. Bugs

Please log your bugs on the "Github issues tracker":http://github.com/newsapps/beeswithmachineguns/issues.

h2. Credits

The bees are a creation of the News Applications team at the Chicago Tribune--visit "our blog":http://apps.chicagotribune.com/ and read "our original post about the project":http://blog.apps.chicagotribune.com/2010/07/%2008/bees-with-machine-guns/.

Initial refactoring code and inspiration from "Jeff Larson":http://github.com/thejefflarson.

Multiple url support from "timsu":https://github.com/timsu/beeswithmachineguns.

Thanks to everyone who reported bugs against the alpha release.

h2. License

MIT.