For the past six weeks, we've been diligently overhauling dstack with the aim of significantly simplifying the process of configuring clouds and enhancing the functionality of the API. Please take note of the breaking changes, as they necessitate careful migration.

Cloud setup

Previously, the only way to configure clouds for a project was through the UI. Additionally, you had to specify not only the credentials but also set up a storage bucket for each cloud to store metadata.

Now, you can configure clouds for a project via ~/.dstack/server/config.yml. Example:

projects:
- name: main
  backends:
  - type: aws
    creds:
      type: access_key
      access_key: AIZKISCVKUKO5AAKLAEH
      secret_key: QSbmpqJIUBn1V5U3pyM9S6lwwiu8/fOJ2dgfwFdW

Regions and other settings are optional. Learn more on what credential types are supported
via Clouds.

Enhanced API

The earlier introduced Python API is now greatly refined.

Creating a dstack client is as easy as this:

from dstack.api import Client, ClientError

try:
    client = Client.from_config()
except ClientError:
    print("Can't connect to the server")

Now, you can submit a task or a service:

from dstack.api import Task, Resources, GPU

task = Task(
    image="ghcr.io/huggingface/text-generation-inference:latest",
    env={"MODEL_ID": "TheBloke/Llama-2-13B-chat-GPTQ"},
    commands=[
        "text-generation-launcher --trust-remote-code --quantize gptq",
    ],
    ports=["80"],
)

run = client.runs.submit(
    run_name="my-awesome-run",
    configuration=task,
    resources=Resources(gpu=GPU(memory="24GB")),
)

The dstack.api.Run instance provides methods for various operations including attaching to the run,
forwarding ports to localhost, retrieving status, stopping, and accessing logs. For more details, refer to
the example and reference.

Other changes

• Because we've prioritized CLI and API UX over the UI, the UI is no longer bundled.
  Please inform us if you experience any significant inconvenience related to this.
• Gateways should now be configured using the dstack gateway command, and their usage requires you to specify a domain.
  Learn more about how to set up a gateway.
• The dstack start command is now dstack server.
• The Python API classes were moved from the dstack package to dstack.api.

Migration

Unfortunately, when upgrading to 0.12.0, there is no automatic migration for data.
This means you'll need to delete ~/.dstack and configure dstack from scratch.

1. pip install "dstack[all]==0.12.0"
2. Delete ~/.dstack
3. Configure clouds via ~/.dstack/server/config.yml (see the new guide)
4. Run dstack server

The documentation and examples are updated.

Give it a try

Getting started with dstack takes less than a minute. Go ahead and give it a try.

$ pip install "dstack[all]" -U
$ dstack server

Feedback and support"

Questions and requests for help are very much welcome in our Discord server.

For the past six weeks, we've been diligently overhauling dstack. Today, we're excited to introduce the first release candidate of this major update. 🥳🎊

Major changes

While this update enhances dstack, it also brings significant ⚠️ breaking changes, necessitating careful migration.

1. Runs and metadata are now stored with the dstack server. No need to specify a storage bucket for configuring a cloud backend.
2. Instead of supporting the UI, we decided to improve the UX of the CLI and API. Since 0.12.0, the UI is no longer bundled.
3. You can now configure clouds via YAML (~/.dstack/server/config.yml) instead of using the UI.
4. There is no migration for the data. To upgrade to 0.12.0, you'll need to delete ~/.dstack/server/data and configure dstack from scratch.

Minor changes

1. The dstack start command is now dstack server.
2. The Python API classes were moved from the dstack package to dstack.api.

Work in progress

Secrets and private Docker registry support coming in 0.12.0 or 0.12.1.

Documentation

The updated documentation can be found in the repo. It will be published once 0.12.0 is released (within a week or so).

Migration

1. pip install "dstack[all]==0.12.0rc1"
2. Delete ~/.dstack
3. Configure clouds via ~/.dstack/server/config.yml (see the new guide)
4. Run dstack server

Feedback

1. Report any issues to GitHub
2. Ask questions in Discord

What's coming next

1. Expect 0.12.0 to be released within the next week or so.
2. Any discovered bugs will be fixed within the next couple of weeks and included in both 0.12.0 and 0.12.1.
3. Thanks to the significant internal improvements made in this update, we'll be able to add new features much faster. Stay tuned for some exciting updates in the next month.

The 0.11.3 update fixes several bugs related to the Python API.

Give it a try

Here's how to install the update:

pip install "dstack[all]==0.11.3"

Please report any issues to the tracker or on Discord.

Full Changelog: https://github.com/dstackai/dstack/compare/0.11.2...0.11.3

Previously, dstack offered the CLI as the primary method for submitting and managing tasks, services, and dev environments. However, the latest release of dstack now allows for programmatic submission and management of runs via Python API.

Here's what the API enables you to do:

• Submit tasks and services (e.g., for batch jobs and deploying LLMs or other apps).
• Retrieve information for a run using its unique name.
• Stop a run using its unique name.
• Access real-time logs for a run using its unique name.

You can learn more about the API at https://dstack.ai/docs/reference/api/python.

To showcase how this API functions, we've created a simple example: a Streamlit app that enables deploying an LLM of your choice as a task.

Learn more about the example (and view its source code) at https://dstack.ai/examples/python-api/.

Give it a try

Here's how to install the update:

pip install "dstack[all]==0.11.2"

We encourage you to try the update and report any issues either through the tracker or by writing directly on Slack.

The upcoming 0.11.2 release brings the long-requested Python API that enables you to submit and manage tasks and services programmatically from your Python code.

You can test this new API using this 0.11.2rc1 pre-release build.

For more information on how to install it and how to use the Python API, refer to the documentation.

Services is a preview feature that allows serving LLMs via public endpoints. The latest update of dstack makes the use of services much easier.

Default gateways

Previously, to run a service, you had to create a gateway using the dstack gateway create command and pass its address via the gateway property in the service configuration file.

Now, you don't need to use the gateway property anymore, as you can create a gateway via the UI and mark it as default.

Wildcard domain

Once the gateway is created (and assigned an external IP), you can set up an A record with your DNS provider to map *.<your domain name> to the gateway's IP and specify this wildcard domain in the gateway's settings.

If a wildcard domain is configured, dstack automatically enables HTTPS and runs services at https://<run name>.<your domain name>.

Task scheduling

In other news, the update fixes a few bugs with the --retry-limit argument in dstack run. Now, it works again, allowing you to schedule tasks even if there is no required capacity at the moment.

Last but not least, we've updated the entire documentation and examples.

Give it a try

Here's how to install the update:

pip install "dstack[all]==0.11.1"

We encourage you to try the update and report any issues either through the tracker or by writing directly on Slack.

The 0.11 version of dstack introduces automatic discovery of the best GPU price and availability across multiple configured cloud providers and regions.

Multiple backends per project

Now, dstack run leverages price data from multiple configured cloud providers and regions to automatically suggest the most cost-effective options.

To implement this change, we have modified the way projects are configured. You can now configure multiple clouds and regions within a single project.

Custom domains and HTTPS

Also, it is now possible to deploy services using HTTPS. All you need to do is configure a wildcard domain (e.g., *.mydomain.com), point it to the gateway IP address, and then pass the subdomain you want to use (e.g., myservice.mydomain.com) to the gateway property in YAML (instead of the gateway IP address).

Other changes

.dstack/profiles.yml

• The project property is no longer supported.
• You can now use max_price to set the maximum price per hour in dollars.

dstack run

• Using the dstack run command, you are now able to utilize options such as --gpu, --memory, --env, --max-price, and several other arguments to override the profile settings.

Lastly, the local backend is no longer supported. Now, you can run everything using only a cloud backend.

The documentation is updated to reflect the changes in the release.

Migration to 0.11

The dstack version 0.11 update brings significant changes that break backward compatibility. If you used prior dstack versions, after updating to dstack==0.11, you'll need to log in to the UI and reconfigure clouds. We apologize for any inconvenience and aim to ensure future updates maintain backward compatibility.

Give it a try

Here's how to install the update:

pip install "dstack[all]==0.11"

We encourage you to try the update and report issues to the tracker, or write directly on Slack.

The upcoming major release, dstack 0.11, introduces multi-cloud projects. The first release candidate, dstack 0.11rc, allows you to test this new functionality.

How this works?

Now, dstack run leverages price data from multiple configured cloud providers and regions to automatically suggest the most cost-effective options.

Notable changes in 0.11rc1

1. Multiple clouds per project: Configure multiple cloud backends in a project. dstack suggests cost-effective choices using data from configured clouds.
2. Multiple regions per project: Configure multiple regions in a project to combat GPU shortages.
3. Max price: Set the max price per hour in dollars for dstack to find capacity within your budget.
4. Custom domains and HTTPS: You can now deploy services with HTTPS using wildcard domains and subdomains.
5. .dstack/profiles.yml: The project property is no longer supported.
6. dstack run: The dstack run command now supports --gpu, --memory, --env, --max-price, and other arguments.

Migration from 0.10.x

Version 0.11 brings significant changes that break backward compatibility. You'll need to reconfigure project and cloud backends. We apologize for any inconvenience and aim to maintain backward compatibility in future updates.

Note
The documentation will be updated within a week after the final dstack 0.11 release. In case of questions, write us in the Slack chat.

In the meantime, we encourage you to try dstack 0.11rc1 and report issues to the tracker or write on Slack.

This minor update addresses a crucial issue that was causing services to malfunction. If you're utilizing services, be sure to update.

Changelog: https://github.com/dstackai/dstack/compare/0.10.7...0.10.8

Until now, dstack has supported dev-environment and task as configuration types. With the 0.10.7 update, we introduce
service, a dedicated configuration type for serving.

Services

Usage example:

type: service

gateway: ${{ secrets.GATEWAY_ADDRESS }}

image: ghcr.io/huggingface/text-generation-inference:0.9.3

port: 8000

commands: 
  - text-generation-launcher --hostname 0.0.0.0 --port 8000 --trust-remote-code

The gateway property represents the address of a special cloud instance that wraps the running service with a public endpoint.

Configuring a gateway

Before you can run a service, you have to configure a gateway.

First, you have to create a gateway in a project of your choice using the dstack gateway create command:

dstack gateway create

Once the gateway is up, the command will print its address. Go ahead and create a secret with this address.

dstack secrets add GATEWAY_ADDRESS <gateway address>

That's it! Now you can run your service using the dstack run command, which deploys the service and forwards the traffic to the gateway, thereby providing you with a public endpoint.

This initial support for services is the first step towards providing multi-cloud and cost-effective inference. In the near future, we plan to make it more functional and easier to use.

Give it a try

A detailed guide on how to try services can be found in our documentation.

As always, your feedback and questions are very much welcome in our Slack chat.

What's Changed

• Support restarting local and GCP runs by @r4victor in https://github.com/dstackai/dstack/pull/590
• Add API for listing all runs by @r4victor in https://github.com/dstackai/dstack/pull/591
• The dstack init should handle invalid Git credentials by @peterschmidt85 in https://github.com/dstackai/dstack/pull/594
• Support custom run names by @r4victor in https://github.com/dstackai/dstack/pull/595
• dstack doesn't work if the repo contains a submodule with SSH URL by @peterschmidt85 in https://github.com/dstackai/dstack/pull/598
• Always use cuda images for instances with GPU by @r4victor in https://github.com/dstackai/dstack/pull/602
• Introduce gateways for services publication by @Egor-S in https://github.com/dstackai/dstack/pull/596
• Small ports refactoring by @Egor-S in https://github.com/dstackai/dstack/pull/603
• #588 Created All run list page. by @olgenn in https://github.com/dstackai/dstack/pull/607
• Improve gateway security, show verbose errors by @Egor-S in https://github.com/dstackai/dstack/pull/608
• Do not require sshd in task configuration for custom docker images by @Egor-S in https://github.com/dstackai/dstack/pull/609
• #588 all runs list by @olgenn in https://github.com/dstackai/dstack/pull/611
• [Bug]: Doesn't run a dev environment if code is not configured in PATH by @peterschmidt85 in https://github.com/dstackai/dstack/pull/613
• Allow to create gateways in AWS and Azure by @Egor-S in https://github.com/dstackai/dstack/pull/614
• [Bug]: dstack init doesn't work if .git/config or ~/.gitconfig doesn't have the user section by @peterschmidt85 in https://github.com/dstackai/dstack/pull/617
• [Bug]: dstack config doesn't work if ~/.dstack doesn't exist by @peterschmidt85 in https://github.com/dstackai/dstack/pull/618
• Migrate to Gen2 images for Azure, add A100 support by @Egor-S in https://github.com/dstackai/dstack/pull/619
• Add time.Sleep() in /logsws handler by @r4victor in https://github.com/dstackai/dstack/pull/623
• Services & gateway docs by @Egor-S in https://github.com/dstackai/dstack/pull/620
• Replace localhost with gateway hostname in service logs by @Egor-S in https://github.com/dstackai/dstack/pull/624

Changelog: https://github.com/dstackai/dstack/compare/0.10.6...0.10.7

The latest release of dstack brings numerous improvements in many areas.

Port mapping

Any task that is running on dstack can expose ports. Here's an example:

type: task

ports:
  - 7860

commands:
  - pip install -r requirements.txt
  - gradio app.py

When you run it with dstack run, by default, dstack forwards the traffic from the specified port to the same port on your local machine.

With this update, you now have the option to override the local machine's port for traffic forwarding.

$ dstack run . -f serve.dstack.yml --port 3000:7860

This command forwards the traffic to port 3000 on your local machine.

If you specify a port on your local machine already taken by another process, dstack will notify you before provisioning cloud resources.

Max duration

Previously, when running a dev environment or task with dstack and forgetting about it, it would continue indefinitely. Now, you can use the max_duration property in .dstack/profiles.yml to set a maximum time for workloads.

Example:

profiles:
  - name: gcp-t4
    project: gcp
    resources:
      memory: 24GB
      gpu:
        name: T4
    max_duration: 2h

With this profile, dstack will automatically stop the workload after 2 hours.

If you don't specify max_duration, dstack defaults to 6h for dev environments and 72h for tasks.

To disable max duration, you can set it to off.

Imagine the amount of money your team can save with this minor configuration.

More supported GPUs

With the CUDA version updated to 11.8, dstack now supports additional GPU types, including NVIDIA T4 and NVIDIA L4. These GPUs are highly efficient for LLM development, offering excellent performance at low costs!

If you are using a custom Docker image, you can now utilize a CUDA version up to 12.2.

Last but not least, the K80 GPU is no longer supported.

Examples

Make sure to check the new page with examples.

The documentation is updated to reflect the changes in the release.

Give it a try

Getting started with dstack takes less than a minute. Go ahead and give it a try.

pip install "dstack[aws,gcp,azure,lambda]" -U
dstack start

We warmly welcome your feedback in our Slack chat.

What's Changed

• Elaborate error message on unmatched requirements for local backend by @r4victor in https://github.com/dstackai/dstack/pull/563
• Handle no permissions for listing aws buckets by @r4victor in https://github.com/dstackai/dstack/pull/566
• Support max_duration by @r4victor in https://github.com/dstackai/dstack/pull/571
• Fix aws not using default region without default creds by @r4victor in https://github.com/dstackai/dstack/pull/572
• Add --ports to dstack run by @Egor-S in https://github.com/dstackai/dstack/pull/573
• Add T4 support for GCP by @Egor-S in https://github.com/dstackai/dstack/pull/575
• Close #537 Added field extra_regions for aws backend by @olgenn in https://github.com/dstackai/dstack/pull/577
• Reserve ports before creating instance by @Egor-S in https://github.com/dstackai/dstack/pull/578
• Upgrade instance images by @Egor-S in https://github.com/dstackai/dstack/pull/581
• Update CUDA version to 11.8 by @peterschmidt85 in https://github.com/dstackai/dstack/pull/584
• Use identity ports mapping by default by @Egor-S in https://github.com/dstackai/dstack/pull/587

Changelog: https://github.com/dstackai/dstack/compare/0.10.5...0.10.6

The Lambda Cloud integration has significantly improved with this release. Additionally, we've added support for custom Docker images.

Lambda Cloud

We've added the possibility to create Lambda Cloud projects via the user interface.

All you need to do is provide your Lambda Cloud API key and specify an S3 bucket and AWS credentials for storing state and artifacts.
Check the docs for more details.

Custom Docker images

By default, dstack uses its own base Docker images to run dev environments and tasks. These base images come pre-configured with Python, Conda, and essential CUDA drivers. However, there may be times when you need additional dependencies that you don't want to install every time you run your dev environment or task.

To address this, dstack now allows specifying custom Docker images. Here's an example:

type: task

image: ghcr.io/huggingface/text-generation-inference:0.9

env:
  - MODEL_ID=tiiuae/falcon-7b

ports:
 - 3000

commands: 
  - text-generation-launcher --hostname 0.0.0.0 --port 3000 --trust-remote-code

Note
Dev environments require the Docker image to have openssh-server pre-installed.

Give it a try

Getting started with dstack takes less than a minute. Go ahead and give it a try.

pip install "dstack[aws,gcp,azure,lambda]" -U
dstack start

What's Changed

• Refactor providers, add custom docker images support by @Egor-S in https://github.com/dstackai/dstack/pull/544
• Use nvidia docker runtime for build if available by @Egor-S in https://github.com/dstackai/dstack/pull/552
• Allow env variables interpolation by @Egor-S in https://github.com/dstackai/dstack/pull/553
• Display docker pull progress by @Egor-S in https://github.com/dstackai/dstack/pull/554
• Handle backend creds becoming invalid by @r4victor in https://github.com/dstackai/dstack/pull/556
• Add PATH to .profile by @Egor-S in https://github.com/dstackai/dstack/pull/557
• Close #529 Supported lambda backend type in UI by @olgenn in https://github.com/dstackai/dstack/pull/558
• #529 Fixed labels by @olgenn in https://github.com/dstackai/dstack/pull/560
• 529 fixes after review by @olgenn in https://github.com/dstackai/dstack/pull/562

Changelog: https://github.com/dstackai/dstack/compare/0.10.4...0.10.5

dstack 0.10.4 is a bugfix update that addresses two major issues related to AWS and Lambda Cloud.

Changelog

• Fixed the issue where creating an AWS project was not possible #545
• Resolved the problem with provisioning on Lambda Cloud #542

Update and provide feedback

Feel free to install the update, give it a try, and share your feedback in our Slack community.

What's Changed

• [Bug]: Provisioning on Lambda Cloud doesn't work #542 by @peterschmidt85 in https://github.com/dstackai/dstack/pull/543
• [Bug]: Cannot create an AWS project #545 by @peterschmidt85 in https://github.com/dstackai/dstack/pull/546

Changelog: https://github.com/dstackai/dstack/compare/0.10.3...0.10.4

With the 0.10.3 update, dstack now allows provisioning infrastructure in Lambda Cloud while storing state and artifacts in an S3 bucket.

See the Reference for detailed instructions on how to configure a project that uses Lambda Cloud.

Note, there are a few limitations in the preview:

1. Since Lambda Cloud does not have its own object storage, dstack requires you to specify an S3 bucket, along with AWS credentials for storing state and artifacts.
2. At the moment, there is no possibility to create a Lambda project via the UI. Currently, you can only create a Lambda project through an
   API request.

In other news, we have pre-configured the base Docker image with the required Conda channel, enabling you to install additional CUDA tools like nvcc using conda install cuda. Note that you only need it for building a custom CUDA kernel; otherwise, the essential CUDA drivers are already pre-installed and not necessary.

The documentation and examples are updated to reflect the changes.

Give it a try and share feedback

Go ahead, and install the update, give it a spin, and share your feedback in our Slack community.

What's Changed

• Updated fields in run details page by @olgenn in https://github.com/dstackai/dstack/pull/515
• Update gcp.md by @axitkhurana in https://github.com/dstackai/dstack/pull/517
• Warn user about long git diff by @Egor-S in https://github.com/dstackai/dstack/pull/518
• Close #523 Implemented building run status by @olgenn in https://github.com/dstackai/dstack/pull/525
• Implement Lambda Labs backend by @r4victor in https://github.com/dstackai/dstack/pull/528
• Restore cache functionality by @Egor-S in https://github.com/dstackai/dstack/pull/527
• Runner storages refactoring by @Egor-S in https://github.com/dstackai/dstack/pull/532
• Refactor backends by @r4victor in https://github.com/dstackai/dstack/pull/533
• Add multi-region compute support for AWS by @r4victor in https://github.com/dstackai/dstack/pull/536
• Check if the build exists before starting an instance by @Egor-S in https://github.com/dstackai/dstack/pull/534
• Pre-configure the nvidia/label/cuda-11.4.3 channel for conda in the CUDA image by @peterschmidt85 in https://github.com/dstackai/dstack/pull/541

New Contributors

• @axitkhurana made their first contribution in https://github.com/dstackai/dstack/pull/517

Changelog: https://github.com/dstackai/dstack/compare/0.10.2...0.10.3

With the dstack 0.10.2 release, we've added two major new features that make running dev environments and ML tasks in the cloud 10-x easier.

Build command

Consider the following configuration:

type: dev-environment
build:
  - apt-get update
  - apt-get install -y ffmpeg
  - pip install -r requirements.txt
ide: vscode

If you run it using dstack run and there is no pre-built image available for this configuration, it will fail and prompt you to either use the dstack build command or add the --build flag to the dstack run command. If the pre-built image is available, it will be used automatically.

Now, if your setup is complex or time-consuming, you don't necessarily have to build and publish your own Docker images. You can simply define the build property and use the dstack build command or add the --build flag to the dstack run command.

If you have specific commands that need to be executed before the dev environment starts, you can now use the init property in YAML.

You can find more details on the updated syntax of .dstack.yml in the Reference section.

Spot and retry policies

Another new major feature is support for spot_policy and retry_policy in .dstack/profiles.yml.

The spot_policy determines if dstack should use spot or on-demand cloud instances for running dev environments and
tasks. It can be set to the following values:

• spot – Always uses spot cloud instances only
• on-demand – Always uses on-demand cloud instances only
• auto – Try to use spot cloud instances if available and on-demand if not

If you don't specify spot_policy, it is set to on-demand by default for dev environments, while for tasks,
the default is auto.

Retry policy

The retry_policy determines if dstack should retry when there is no cloud capacity for running the dev environment or
task. It can have the following nested properties:

• retry – true if dstack should retry, and false if not
• limit – The duration within which dstack should retry. The default is 1d (one day).

If you don't specify the retry_policy, it is enabled for tasks by default, while for dev environments, it is disabled.

You can find more details on the updated syntax of .dstack/profiles.yml in the Reference section.

With these introduced policies, using spot cloud instances becomes easier. Additionally, you can now schedule tasks that
utilize on-demand instances. This is particularly useful when there is limited capacity in a specific region and you
want the task to automatically run once the capacity becomes available.

Other changes

Other improvements in this release:

• For GCP, it now checks if your cloud account has the necessary permissions to edit project backend settings.
• Deletion of repositories is now possible through the UI.
• When running a dev environment from a Git repo, you can now pull and push changes directly from the dev environment,
  with dstack correctly configuring your Git credentials.
• The newly added Python API for working with artifacts is now documented here.

The documentation and examples are updated to reflect the changes.

Give it a try and share feedback"

Go ahead, and install the update, give it a spin, and share your feedback in our Slack community.

What's Changed

• Hub by @olgenn in https://github.com/dstackai/dstack/pull/473
• Implement spot policy and retry policy by @r4victor in https://github.com/dstackai/dstack/pull/474
• Fix hub freezes on long backend init by @r4victor in https://github.com/dstackai/dstack/pull/476
• Check cloud creds permissions by @r4victor in https://github.com/dstackai/dstack/pull/479
• Make artifacts Python API public by @r4victor in https://github.com/dstackai/dstack/pull/482
• Hub by @olgenn in https://github.com/dstackai/dstack/pull/483
• Handle 503 when provisioning gcp instances by @r4victor in https://github.com/dstackai/dstack/pull/489
• Export/import prebuild image via overlay2 internals by @Egor-S in https://github.com/dstackai/dstack/pull/477
• Add API endpoint to delete repos by @r4victor in https://github.com/dstackai/dstack/pull/491
• Improve dstack run and dstack server output by @r4victor in https://github.com/dstackai/dstack/pull/493
• Inform users about image pull for local runs by @r4victor in https://github.com/dstackai/dstack/pull/495
• Restore cursor on abort by @r4victor in https://github.com/dstackai/dstack/pull/496
• Allocate a vacant port for logs streaming by @Egor-S in https://github.com/dstackai/dstack/pull/497
• Implement --diagnose for dstack logs by @r4victor in https://github.com/dstackai/dstack/pull/498
• Fixed defect with default credentials by @olgenn in https://github.com/dstackai/dstack/pull/501
• Implemented showing all backend types by @olgenn in https://github.com/dstackai/dstack/pull/500
• Close #502 Implemented deleting repositories from list by @olgenn in https://github.com/dstackai/dstack/pull/503
• Change build semantics by @Egor-S in https://github.com/dstackai/dstack/pull/506
• Mark jobs as failed when instance is terminated by @r4victor in https://github.com/dstackai/dstack/pull/507
• Configure git repository in dev environment by @Egor-S in https://github.com/dstackai/dstack/pull/510
• Rephrase build related messages by @Egor-S in https://github.com/dstackai/dstack/pull/511
• Update docs on the build feature by @Egor-S in https://github.com/dstackai/dstack/pull/513
• Run default dev environment without any build by @Egor-S in https://github.com/dstackai/dstack/pull/514

Changelog: https://github.com/dstackai/dstack/compare/0.10.1...0.10.2

The new update brings several significant improvements.

Azure support

First and foremost, dstack now enables running dev environments, workflows, and apps with Azure.
All you need to do is create the corresponding project via the UI and provide your Azure credentials.

For detailed instructions on setting up dstack for Azure, refer to the documentation.

UI for logs and artifacts

Secondly, you can now browse the logs and artifacts of any run through the user interface.

Better documentation

Last but not least, with the update, we have reworked the documentation to provide a greater emphasis on specific use cases.

Try it out

Please note that when installing dstack via pip, you now need to specify the exact list of cloud providers you intend to use:

pip install "dstack[aws,gcp,azure]" -U

This requirement applies only when using the CLI to start the server locally. If you wish to connect to a server hosted elsewhere, you can still use the previous syntax:pip install dstack.

If you have any feedback, including issues or questions, please share them in our Slack chat or file it as a GitHub issue.

Local Hub

A couple of weeks ago, we introduced Hub, an extension of dstack that makes it easier for teams to use dstack. Because Hub not only helps teams, but also has other valuable features, such as a user interface, more secure management of cloud credentials, and better workflow scheduling (e.g., job queues), we decided to make Hub the default and only way to use dstack.

Here's what specifically changed:

1. To use dstack, you now always need Hub, even if you run workflows locally. You can start Hub by running the dstack start command. On startup, this command creates the default project with the backend type Local, which means that workflows run locally.

2. To run workflows in the cloud, you need to log in to Hub, create the corresponding project, and configure the CLI to use it via dstack config.

3. The dstack config command now allows you to configure Hub projects only. However, you can configure multiple projects and always have one as the default.

4. The --remote flag in dstack run is now dropped. Instead, you can use the --project NAME argument to specify the project. The --project argument is now supported by all commands, including dstack init, dstack ps, etc.

Why this matters:

1. Hub is now the only way to use dstack, making it simpler to set up and configure.
2. All Hub features are now enabled for local development as well.

Note that this change is not backward compatible. In case of any issues, please write in Slack or raise GitHub issues.

To try the new experience, follow our updated Quick start guide.

Docker image

While the dstack start command is the most recommended way to run the Hub application locally, Docker is the recommended way to deploy Hub for remote access.

The updated Docker image is now available here, and you will also find the updated documentation there.

Note:
By default, the Hub application stores the state locally (via SQLite) in the ${HOME}/.dstack/hub folder.

To help persist the state, the Hub application bundles Litestream, a tool that can automatically persist the state in the object store in your cloud account.

You'll find instructions on how to do it in the README section of the Docker image.

Hugging Face integration

If you want to run Hub on Hugging Face, it's now very easy to do so.

All you have to do is to duplicate this template and configure your projects.
The admin token can be found in the logs.

Note:
Because Hugging Face doesn't persist data, feel free to configure Litestream settings via Hugging Face secrets. For example, you can set your space to automatically persist data in your AWS, GCP, or Azure object storage.

Git-less repos

Previously, dstack always required the folder where you run it to be a Git repo with a remote branch configured. Now, this requirement is dropped. Now you can run dstack on any folder.

Stop on Ctrl+C

Last but not least, we've changed the behavior of dstack when you press Ctrl+C. If earlier it aborted the run immediately, now it stops it gracefully, allowing your workflow to stop, and save output artifacts.

Try it now

To install the update, simply run the following command:

pip install dstack==0.8

If you have any feedback, including issues or questions, please share them in our Slack chat or file it as a GitHub issue.

dstack Hub

dstack 0.7 is a massive update that introduces a major new feature called Hub. Hub is a new extension of dstack specifically designed for teams, with the purpose of providing a unified interface that enables teams to effectively manage their model development workflows on any cloud. To learn more about Hub, please read the Hub announcement in our blog.

Cache

Apart from introducing Hub, the update also adds the "Cache" feature which enables caching of files across runs of the same workflow. You can learn more about this feature in our documentation.

Try it now

To install the latest version of dstack, simply run the following command

pip install dstack --upgrade

If you have any feedback, including issues or questions, please share them in our Slack chat or file it as a GitHub issue.

The release of dstack 0.2.6 addresses various bugs, extends the providers with the ssh property, and introduces Playground. Please read on for additional information.

SSH

The bash, code, lab, and notebook providers now support the ssh property. By setting it to true, you can connect to the running workflow via SSH.

See the example below:

workflows:
  - name: hello-ssh
    provider: bash
    commands:
      - tail -f /dev/null
    ssh: true 

Run this workflow:

$ dstack run hello-ssh
 RUN          WORKFLOW    SUBMITTED  STATUS     TAG  BACKENDS
 moody-emu-1  hello-ssh   now        Submitted       local

Provisioning... It may take up to a minute. ✓

To interrupt, press Ctrl+C.

To connect via SSH, use:
  ssh -i /Users/cheptsov/.ssh/id_rsa root@localhost -p 56947

This enables connection to the workflow through SSH using the command provided in the output. Similar to other provider properties, the ssh feature can be used for both local and remote workflows.

Bugfixes

Moreover, this update resolves a few issues related to dstack init. We recommend updating, regardless of whether you encountered any problems.

Playground

Lastly, we have significantly revamped our examples and introduced a Playground that can be accessed in Gitpod. This will automatically install dstack and prepare the environment for you.

The playground is the easiest way to run examples and learn how dstack operates.

Try it now

To install the latest version of dstack, simply run the following command

pip install dstack --upgrade

If you have any feedback, including issues or questions, please share them in our Slack chat or file it as a GitHub issue.

The latest update to dstack 0.2.2 improves the developer experience of remote workflows.

Previously, when running a remote workflow that used ports (for example, to host dev environments or applications), dstack made those ports on the remote machine public, allowing anyone within the configured VPC to access them via HTTP.

This solution was insecure and posed a security risk. Additionally, dev environments or applications that used iframes, like VS Code rendering Jupyter notebooks inside the IDE, did not work due to security reasons.

SSH port forwarding

However, the dstack 0.2.2 update resolves this issue by establishing an SSH tunnel between the local and remote machine and forwarding remote ports to your local machine. This solves both issues, as workflow ports are not exposed publicly, allowing only authorized users to access them, and dev environments and applications work seamlessly even if they use iframes.

How does it work?

To initialize your project using dstack, run dstack init. The dstack backend will use your public SSH key (which you can override by using dstack init --ssh-identity-file PATH). Then, when you run a remote workflow via dstack run, it establishes an SSH tunnel to the remote machine and forwards ports to your local machine.

Even though the workflow is running remotely (e.g. in GCP), it prints the URL on which the application is available on your local machine. For example, if you run the code provider, you can use the built-in Jupyter notebooks conveniently.

This new feature makes it easy to set up remote dev environments and run remote applications securely.

In the meantime, we are working on another SSH-related feature that will allow you to attach any IDE or terminal inside the running workflow conveniently via SSH.

Try it now

To install the latest version of dstack, simply run the following command

pip install dstack --upgrade

If you have any feedback, including issues or questions, please share them in our Slack chat or file it as a GitHub issue.