thread6

Simple parallel processing interface for python

Why?

Python's built in parallel processing and threading library is pretty simple to implement but sometimes you just want to chuck data at a function and make it run faster

Requirements

Python 3+

Installation

$ pip install thread6

Quickstart

Use the threaded decorator to turn a method into a threaded method. That's it!

@thread6.threaded()
def threaded_print():
    print("")
    return 1

Alternatively, use run_threaded function

thread6.run_threaded(threaded_print)

Both the threaded decorator and run_threaded method will return an instance of ResultThread. This allow you to optionally wait for the function to finish executing and get the return value. To get the return value, use .await_output()

result = threaded_print()
result.await_output()  # this will return 1

If you have a function that needs to execute on a large list of data, use run_chunked

def update_items(items):
    ...

items = [...]
thread6.run_chunked(update_items, items)

.await_output() also work with run_chunked but will return a list of return values instead

Run the tests

Run tests with

python tests.py

Todo

• threaded function decorator
• run something in a separate thread function
• split data into chunk and run in separate threads
• add way for errors to fail loudly
• auto spawn to run fx on a set of data
• explore multi processing?