The fastest way to iterate and deploy AI workloads on your own infra. Unobtrusive, debuggable, PyTorch-like APIs.
APACHE-2.0 License
Bot releases are hidden (Show)
Improved parallelism, clearer exceptions, and saving resources within Den orgs
API_SERVER_URL
(#708)Full Changelog: https://github.com/run-house/runhouse/compare/v0.0.24...v0.0.25
Published by dongreenberg 6 months ago
Fast-follow bugfixes for CPU parallelism and log streaming
Full Changelog: https://github.com/run-house/runhouse/compare/v0.0.23...v0.0.24
Published by dongreenberg 6 months ago
Richer async support, performance improvements, and bugfixes
run_async
as an argument into the method call. If your method is already defined as async, this will be applied automatically without specifying run_async
so your code can await
the remote method just as it did the original. You can still explicitly set run_async=False
in that case to make the local call sync.requests.Session
created in HTTPClient (#694)runhouse login
bugs (#717)Full Changelog: https://github.com/run-house/runhouse/compare/v0.0.22...v0.0.23
Published by carolineechen 7 months ago
Performance improvements + bug fixes
AuthCache
logic to request per keypair (#684)Published by carolineechen 7 months ago
Some performance and feature improvements, bug fixes, and new examples.
load_config
when dependency is missing (#595)runhouse stop
(#596)module.to(rh.here)
throws error if local server is not initialized (#597)data
field (#602)None
in failed mapper replicas (#605)sshtunnel
library dependency (#625, #634, #640)check_server
instead of is_up
with refresh for ondemand cluster endpoint (#614)register_activity
calls within env servlet (#629)runhouse[aws]
(#613)put_resource
(#626)
reqs
and setup_cmds
in rh.function.to()
removed. Pass it into the env
instead.access_type
removed in Resource
and share
. Use access_level
instead.rh.here.put/get/delete/keys/clear
instead.system
to rh.function/module
does not send code to the system and can be misleading. Use .to
or get_or_to
to sync code to the cluster.See rendered examples on https://www.run.house/examples
Published by carolineechen 7 months ago
We’ve made it easier to share clusters across different environments and with other users. You can now share and load a cluster just as you would any other resource.
my_cluster = rh.cluster("rh-cluster", ips=[...], ...)
my_cluster.share(["[email protected]", "username2"])
# load the box with
shared_cluster = rh.cluster("owner_username/rh-cluster")
Shared users will be able to seamlessly run shared apps on that cluster, or SSH directly onto the remote box. To enable this, we persist the SSH credentials for the cluster as a Runhouse Secret object, which can easily be reloaded when another user tries to connect.
rh.Mapper
was first introduced in runhouse v0.0.15, an extension of functions/modules to handle mapping, replicating, and load balancing. Further improvements and some bug fixes were included in this release, plus a BC-breaking variable name (see section below).
def local_sum(arg1, arg2, arg3):
return arg1 + arg2 + arg3
remote_fn = rh.function(local_sum).to(my_cluster)
mapper = rh.mapper(remote_fn, replicas=2)
mapper.map([1, 2], [1, 4], [2, 3])
# output: [4, 9]
rh.mapper
factory function args renaming
num_replicas
-> replicas
replicas
-> concurrency
See updated tutorials on Runhouse docs
See new Runhouse examples on GitHub or webpage
Published by carolineechen 8 months ago
Previously, the Runhouse server was strictly designed to allow you to deploy apps to it remotely with my_module.to(my_cluster)
. Now, you can now start the Runhouse server daemon directly to be able to deploy it locally like a traditional web server. Access the local daemon's Cluster object in Python with rh.here
. rh.here
always refers to the locally running daemon, so you can use within an existing Runhouse cluster as well.
Start your local Runhouse server:
$ runhouse restart
$ runhouse status
To send a module:
def concat(a, b):
return a+b
import runhouse as rh
rh.function(concat).to(rh.here)
To try out your service:
curl -X "GET" 'http://localhost:32300/concat/call?a=run&b=house'
>>> {"data":"\"runhouse\"","error":null,"traceback":null,"output_type":"result_serialized","serialization":"json"}
This is also particularly useful for debugging. You can ssh onto your cluster, start a Python shell, and run methods like rh.here.call("my_module", "my_method")
to test or analyze your deployed module's behavior or contents quickly.
Use Caddy as a reverse proxy for the Runhouse server launched on clusters, as well as automatically generating and auto-renewing self-signed certificates, making it easy to secure your cluster with HTTPS right out of the box.
reqs
and setup_cmds
removed from function .to (#373)Published by carolineechen 8 months ago
This release largely consists of updates to Runhouse infrastructure and build, with the addition of two new runhouse CLI commands (stop and status), basic ASGI support, and some bug fixes.
Note: We’ve removed some dependencies to better support local-only use cases of Runhouse. To use Runhouse with a cluster, please install with pip install “runhouse[sky]”
, and to use Runhouse data structures like tables, please install with pip install “runhouse[data]”
rh.here
working (#338)Published by carolineechen 9 months ago
The Mapper expands Runhouse functions and Module methods to handle mapping, replicating, and load balancing. A Mapper object is constructed simply by passing in a function or module and module method, along with the number of replicas to use, and optionally your own user-specified replicas. It takes the function and creates replicas of it and its envs, and round-robin calls the replicas to run function calls in parallel.
def local_sum(arg1, arg2, arg3):
return arg1 + arg2 + arg3
remote_fn = rh.function(local_sum).to(my_cluster)
mapper = rh.mapper(remote_fn, num_replicas=2)
mapper.map([1, 2], [1, 4], [2, 3])
# output: [4,9]
instance_count
with num_instances
for cluster class (#269)Published by carolineechen 10 months ago
The rh.Secrets
class is being deprecated in favor of converting secrets to a Runhouse resource type. As with other resources, the new Secret class supports saving, reloading, and sending secrets across clusters.
There are various builtin secret provider types, for keeping track of compute providers (aws, azure, gcp..), api key based providers (openai, anthropic, …), and ssh key pairs.
# non-provider secret, in-memory
my_secret = rh.secret(name=”my_secret”, values={“key1”: “val1”, “key2”: “val2”})
my_secret.save()
reloaded_secret = rh.secret(“my_secret”)
# provider secret, in-memory or loaded from default location
aws_secret = rh.provider_secret(“aws”) # loads from ~/.aws/credentials or from env vars
openai_secret = rh.provider_secret(“openai”, values={“api_key”: “my_openai_key”}) # explicitly provided values
There are also various APIs for syncing secrets across your clusters and environments:
aws_secret.to(cluster, env)
cluster.sync_secrets([“aws”, “gcp”], env)
env = rh.env(secrets=[“aws”, “openai”]
fn.to(cluster, env)
Please refer to the API tutorial for a more in-depth walkthrough of using Secrets, or the documentation for specific APIs and a full list of builtin providers.
Runhouse is extending functions to Amazon Web Services (AWS) Lambda Compute. These functions are deployed directly on AWS serverless compute, with Lambda’s infra and servers handled under the hood, making the Lambda onboarding process more smooth and removing the need to translate code through Lambda-specific APIs.
Note: Lambda Functions are in Alpha and the APIs are subject to change. A more stable release along with examples will be published soon. In the meantime, you can find documentation here.
access_type
deprecated and renamed to access_level
for resource and sharing (#223, #224, #231)rh.Secrets
class deprecated in favor of convert Secrets to a resource type ((#135). Some old APIs are removed, and others are deprecated. Please refer to docs and tutorial for the new secrets flow.Published by DenisYay 11 months ago
Runhouse is integrating with Amazon Web Services (AWS) SageMaker to allow rapid onboarding onto SageMaker, usually within minutes, and to remove the need to translate code into SageMaker-specific APIs so it can still be used dynamically with other compute infra.
The SageMaker cluster follows the Runhouse cluster definition and usage, but uses Sagemaker compute under the hood.
If you already use SageMaker with your AWS account, you should already be set to use Runhouse SageMaker support. For full SageMaker setup and dependencies, please refer to the docs.
Example 1: Launch a new SageMaker instance and keep it up indefinitely.
# Note: this will use Role ARN associated with the "sagemaker" profile defined in the local AWS config (e.g. `~/.aws/config`).
import runhouse as rh
c = rh.sagemaker_cluster(name='sm-cluster', profile="sagemaker").save()
Example 2: Running a training job with a provided Estimator
c = rh.sagemaker_cluster(name='sagemaker-cluster',
estimator=PyTorch(entry_point='train.py',
role='arn:aws:iam::123456789012:role/MySageMakerRole',
source_dir='/Users/myuser/dev/sagemaker',
framework_version='1.8.1',
py_version='py36',
instance_type='ml.p3.2xlarge'),
).save()
Adds an option for starting up the Runhouse API server on the cluster with HTTPS, including optionally creating self-signed certs and proxying through Nginx. This makes it incredibly fast and easy to stand up a microservice with standard bearer token authentication (using a Runhouse token), allowing users to share Runhouse resources with collaborators, teams, customers, etc.
Supports several new server connection types, including tls
, ssh
. For more information on these types, please refer to docs.
32300
(#124)paramiko
dependency for password clusters (#131)env
(#132)rh.env(
name="my_env",
reqs=["torch", "diffusers"],
setup_cmds=["source ~/.bashrc"]
)
host
parameter for the runhouse start
and runhouse restart
commands, which now defaults to 0.0.0.0
(#110)runhouse restart --host 0.0.0.0
Published by carolineechen about 1 year ago
rh.Module
resource, and a resulting performance and feature improvementsAs mentioned in the 0.0.11 Release Notes, we've redesigned how we handle remote resources, resulting in performance and feature improvements, as well as support for a new type of resource. Basic notes can be found below, or a more comprehensive technical overview can be found in our 0.0.12 blog post (coming soon!)
rh.Module represents a class that can be accessed and used remotely, including all its class methods and variables, and with out-of-the-box support for capabilities like streaming logs/results, async, queuing, etc
rh.module()
factory function for wrapping existing Python classesrh.Module
class that can be subclasses to write natively Runhouse-compatible classesStoring large objects, such as models, in Python memory can reduce time spent loading objects from disk or sending them over.
rh.here.get()
and rh.here.put()
APIs, where rh.here
returns the cluster it is called fromRunhouse is integrating with SageMaker to make the SageMaker onboarding process more smooth, and removing the need to translate code through SageMaker specific estimators or APIs. This will be described in more detail in the 0.0.13 release, or check out the documentation in the meantime.
.remote()
now returns a remote object, rather than a string associated with the object/run. To get the contents of the result, use result.fetch()
Published by carolineechen about 1 year ago
We revamped our underlying code implementation for handling remote code execution, resulting in improvements and added support for:
rh.Module
resource)These new features and updates will be explained in more detail in the following (0.0.12) release
Documentation is now supported and hosted directly in our website, at run.house/docs. Easily access documentation for any of or current and past releases.
runhouse start
commandrunhouse restart_server
command to runhouse restart
Published by carolineechen about 1 year ago
Support for BYO clusters requiring a password
rh.cluster("cluster-name", host=["hostname or ips"], ssh_creds={'ssh_user': '...', 'password':'*****'},
Funhouse/Tutorials Updates
Sentry integration for Runhouse error reports
Published by carolineechen over 1 year ago
Patch release to upgrade skypilot version to v0.3.3, which resolves a critical dependency fix for PyYAML following the Cython 3 release. On Runhouse side, fix a bug for handling git function environment requirements.
Published by dongreenberg over 1 year ago
OnDemandCluster
s broke following the release of SkyPilot 0.3.0, as SkyPilot began to use their own Ray server on a separate port. When we started the Runhouse server, we were inadvertently killing the SkyPilot server, which caused the cluster status to show as in the INIT state indefinitely and suspended autostop.PyOpenSSL<21.1.0
, which was causing pesky breakage in some multiprocessing scenarios. We've pinned pyOpenSSL>=21.1.0
.Published by carolineechen over 1 year ago
env_vars
and custom working_dir
to Env resource (#75)load
parameter, instead will automatically try to load from name
argument if that is the only argument provideddryrun=False
ondemand_cluster
instead of cluster
for On-Demand cloud clustersPublished by carolineechen over 1 year ago
--yes
/-y
option for interactive CLI login (#53)reqs
and setup_cmds
in support for env
(#54)