Dstack

TensorDock

With dstack `0.12.2`, you can now access TensorDock's cloud GPUs, leveraging their highly competitive pricing.

Configuring your TensorDock account with `dstack` is very easy. Simply generate an authorization key in your TensorDock
API settings and set it up in `~/.dstack/server/config.yml`:

yaml
projects:
- name: main
backends:
- type: tensordock
creds:
type: api_key
api_key: 248e621d-9317-7494-dc1557fa5825b-98b
api_token: FyBI3YbnFEYXdth2xqYRnQI7hiusssBC


Now you can restart the server and proceed to using the CLI or API for running development environments, tasks, and services.

shell
dstack run . -f .dstack.yml --gpu 40GB

Min resources 1xGPU (40GB)
Max price -
Max duration 6h
Retry policy no

REGION INSTANCE RESOURCES SPOT PRICE
1 unitedstates ef483076 10xCPU, 80GB, 1xA6000 (48GB) no $0.6235
2 canada 0ca177e7 10xCPU, 80GB, 1xA6000 (48GB) no $0.6435
3 canada 45d0cabd 10xCPU, 80GB, 1xA6000 (48GB) no $0.6435
...

Continue? [y/n]:

Server configuration

Previously, the only way to configure clouds for a project was through the UI. Additionally, you had to specify not only the credentials but also set up a storage bucket for each cloud to store metadata.

Now, you can configure clouds for a project via `~/.dstack/server/config.yml`. Example:


yaml
projects:
- name: main
backends:
- type: aws
creds:
type: access_key
access_key: AIZKISCVKUKO5AAKLAEH
secret_key: QSbmpqJIUBn1V5U3pyM9S6lwwiu8/fOJ2dgfwFdW


Enhanced Python API

The earlier introduced Python API is now greatly refined.

Creating a `dstack` client is as easy as this:

python
from dstack.api import Client, ClientError

try:
client = Client.from_config()
except ClientError:
print("Can't connect to the server")


Now, you can submit a task or a service:

python
from dstack.api import Task, Resources, GPU

task = Task(
image="ghcr.io/huggingface/text-generation-inference:latest",
env={"MODEL_ID": "TheBloke/Llama-2-13B-chat-GPTQ"},
commands=[
"text-generation-launcher --trust-remote-code --quantize gptq",
],
ports=["80"],
)

run = client.runs.submit(
run_name="my-awesome-run",
configuration=task,
resources=Resources(gpu=GPU(memory="24GB")),
)


The `dstack.api.Run` instance provides methods for various operations including attaching to the run,
forwarding ports to `localhost`, retrieving status, stopping, and accessing logs. For more details, refer to
the [example](https://dstack.ai/examples/deploy-python.md) and [reference](https://dstack.ai/docs/reference/api/python/index.md).

Other changes

- Because we've prioritized CLI and API UX over the UI, the UI is no longer bundled.
Please inform us if you experience any significant inconvenience related to this.
- Gateways should now be configured using the `dstack gateway` command, and their usage requires you to specify a domain.
Learn more about how to [set up a gateway](../../docs/guides/services.mdset-up-a-gateway).
- The `dstack start` command is now `dstack server`.
- The Python API classes were moved from the `dstack` package to `dstack.api`.

Migration

Unfortunately, when upgrading to 0.12.0, there is no automatic migration for data.
This means you'll need to delete `~/.dstack` and configure `dstack` from scratch.

1. `pip install "dstack[all]==0.12.0"`
2. Delete `~/.dstack`
3. Configure clouds via `~/.dstack/server/config.yml` (see the [new guide](https://dstack.ai/docs/guides/clouds.md))
4. Run `dstack server`

**Full Changelog**: https://github.com/dstackai/dstack/compare/0.11.2...0.11.3

Default gateway

Previously, to run a service, you had to create a gateway using the `dstack gateway create` command and pass its address via the `gateway` property in the service configuration file.

Now, you don't need to use the `gateway` property anymore, as you can create a gateway via the UI and mark it as default.

<img width="750" alt="dstack-hub-view-project-empty" src="https://github.com/dstackai/dstack/assets/54148038/eeda34f0-4189-4a0d-9976-5543e088b357">

Gateway domain

Once the gateway is created (and assigned an external IP), you can set up an A record with your DNS provider to map `*.<your domain name>` to the gateway's IP and specify this wildcard domain in the gateway's settings.

<img width="750" alt="dstack-hub-edit-gateway" src="https://github.com/dstackai/dstack/assets/54148038/7ad1778b-201b-4b0e-a91e-caff7e924258">

If a wildcard domain is configured, dstack automatically enables HTTPS and runs services at `https://<run name>.<your domain name>`.

Retry policy

In other news, the update fixes a few bugs with the `--retry-limit` argument in `dstack run`. Now, it works again, allowing you to schedule tasks even if there is no required capacity at the moment.

<img width="750" alt="Screenshot 2023-08-31 at 15 02 00" src="https://github.com/dstackai/dstack/assets/54148038/eedc7390-3f8e-4ea7-8e1e-f95c675c1f8e">

Last but not least, we've updated the entire [documentation](https://dstack.ai/docs/) and [examples](https://dstack.ai/examples/).

Multiple backends per project

Now, `dstack run` leverages price data from multiple configured cloud providers and regions to automatically suggest the most cost-effective options.

<img src="https://github.com/dstackai/dstack/assets/54148038/1b7eacff-fa99-4ac7-9145-d6ab0602eaf6" width="700"/>

To implement this change, we have modified the way projects are configured. You can now configure multiple clouds and regions within a single project.

<img src="https://github.com/dstackai/dstack/assets/54148038/a7eca63e-a360-41a7-b8f2-0195e1204981" width="700"/>

Custom domains and HTTPS

Also, it is now possible to deploy services using [HTTPS](https://dstack.ai/docs/guides/services/#configure-a-domain-and-enable-https-optional). All you need to do is configure a wildcard domain (e.g., `*.mydomain.com`), point it to the gateway IP address, and then pass the subdomain you want to use (e.g., `myservice.mydomain.com`) to the gateway property in YAML (instead of the gateway IP address).

Other changes

`.dstack/profiles.yml`

- The `project` property is no longer supported.
- You can now use `max_price` to set the maximum price per hour in dollars.

`dstack run`

- Using the `dstack run` command, you are now able to utilize options such as `--gpu`, `--memory`, `--env`, `--max-price`, and several [other arguments](https://dstack.ai/docs/reference/cli/run/) to override the profile settings.

Lastly, the local backend is no longer supported. Now, you can run everything using only a cloud backend.

The [documentation](https://dstack.ai/docs/) is updated to reflect the changes in the release.

Migration

The `dstack` version 0.11 update brings significant changes that break backward compatibility. If you used prior `dstack` versions, after updating to `dstack==0.11`, you'll need to log in to the UI and reconfigure clouds. We apologize for any inconvenience and aim to ensure future updates maintain backward compatibility.

This minor update addresses a crucial issue that was causing services to malfunction. If you're utilizing services, be sure to update.

**Changelog**: https://github.com/dstackai/dstack/compare/0.10.7...0.10.8