Dstack

CUDO Compute

The `0.16.4` update introduces the `cudo` backend, which allows running workloads with CUDO Compute, a cloud GPU marketplace.

To configure the `cudo` backend, you simply need to specify your CUDO Compute project ID and API key:

yaml
projects:
- name: main
backends:
- type: cudo
project_id: my-cudo-project
creds:
type: api_key
api_key: 7487240a466624b48de22865589


Once it's done, you can restart the `dstack server` and use the `dstack` CLI or API to run workloads.

> [!NOTE]
> **Limitations**
> - The `dstack gateway` feature is not yet compatible with `cudo`, but it is expected to be supported in version `0.17.0`,
> planned for release within a week.
> - The `cudo` backend cannot yet be used with [dstack Sky](https://sky.dstack.ai), but it will also be enabled within a week.


**Full changelog**: https://github.com/dstackai/dstack/compare/0.16.3...0.16.4

Bug-fixes

- [Bug] The `shm_size` property in `resources` doesn't take effect 1006
- [Bug]: It's not possible to configure projects other than main via `~/.dstack/server/config.yml` 991
- [Bug] Spot instances don't work on GCP if the username has upper case letters 975

**Full changelog**: https://github.com/dstackai/dstack/compare/0.16.2...0.16.3

Improvements to `dstack pool`

- Change default idle duration for `dstack pool add` to `72h` 964
- Set the default spot policy in `dstack pool add` to `on-demand` 962
- Add pool support for `lambda`, `azure`, and `tensordock` 923
- Allow to pass idle duration and spot policy in `dstack pool add` 918
- `dstack run` does not respect pool-related `profiles.yml` parameters 949

Bug-fixes

- Runs submitted via Python API have no termination policy 955
- The `vastai` backend doesn't show any offers since `0.16.0` 958
- Handle permission error when adding Include to `~/.ssh/config` 937
- The SSH tunnel fails because of a messy `~/.ssh/config` 933
- The `PATH` is overridden when logging via SSH 930
- The SSH tunnel fails with `Too many authentication failures` 927

We've also updated our guide on how to add new backends. It's now available [here](https://github.com/dstackai/dstack/blob/master/contributing/BACKENDS.md).

New contributors

* iRohith made their first contribution in https://github.com/dstackai/dstack/pull/959
* spott made their first contribution in https://github.com/dstackai/dstack/pull/934
* KevKibe made their first contribution in https://github.com/dstackai/dstack/pull/917

**Full Changelog**: https://github.com/dstackai/dstack/compare/0.16.0...0.16.1

Pools

The `0.16.0` release is the next major update, which, in addition to many bug fixes, introduces pools, a major new feature that enables a more efficient way to manage instance lifecycles and reuse instances across runs.

`dstack run`

Previously, when running a dev environment, task, or service, `dstack` provisioned an instance in a configured
backend, and upon completion of the run, deleted the instance.

Now, when using the `dstack run` command, it tries to reuse an instance from a pool. If no ready instance meets the
requirements, `dstack` automatically provisions a new one and adds it to the pool.

Once the workload finishes, the instance is marked as `idle`.
If the instance remains idle for the configured duration, `dstack` tears it down.

`dstack pool`

The `dstack pool` command allows for managing instances within pools.

To manually add an instance to a pool, use [`dstack pool add`](../../docs/reference/cli/index.mddstack-pool-add):

shell
dstack pool add --gpu 80GB --idle-duration 1d


The `dstack pool add` command allows specifying resource requirements, along with the spot policy, idle duration, max
price, retry policy, and other policies.

If no idle duration is configured, by default, `dstack` sets it to `72h`.
To override it, use the `--idle-duration DURATION` argument.

To learn more about pools, refer to the [official documentation](https://dstack.ai/docs/concepts/pools/). To learn more about `0.16.0`, refer to the [changelog](https://dstack.ai/changelog/0.16.0).

What's changed

* Add dstack pool by TheBits in https://github.com/dstackai/dstack/pull/880
* Pools: fix failed instance status by Egor-S in https://github.com/dstackai/dstack/pull/889
* Add columns to `dstack pool show` by TheBits in https://github.com/dstackai/dstack/pull/898
* Add submit stop by TheBits in https://github.com/dstackai/dstack/pull/895
* Add kubernetes logo by plutov in https://github.com/dstackai/dstack/pull/900
* Handle exceptions from backend.compute().get_offers by r4victor in https://github.com/dstackai/dstack/pull/904
* Fix process_finished_jobs parsing None job_model.job_provisioning_data by r4victor in https://github.com/dstackai/dstack/pull/905
* Validate run_name by r4victor in https://github.com/dstackai/dstack/pull/906
* Filter out private subnets when provisioning in custom aws vpc by r4victor in https://github.com/dstackai/dstack/pull/909
* Issue 894 rework failed instance status by TheBits in https://github.com/dstackai/dstack/pull/899
* Handle unexpected exceptions from run_job by r4victor in https://github.com/dstackai/dstack/pull/911
* Request GPU in docker with --gpus=all by Egor-S in https://github.com/dstackai/dstack/pull/913
* Issue 918 fix cli argimenuts for dstack pool add by TheBits in https://github.com/dstackai/dstack/pull/919
* Added router tests for pools by TheBits in https://github.com/dstackai/dstack/pull/916
* Fix 921 by TheBits in https://github.com/dstackai/dstack/pull/922

New contributors

* plutov made their first contribution in https://github.com/dstackai/dstack/pull/900

**Full changelog**: https://github.com/dstackai/dstack/compare/0.15.1...0.16.0

Bug-fixes

* Exclude private subnets when provisioning in AWS 908
* Ollama doesn't detect the GPU (requires `--gpus==all` instead of `--runtime=nvidia`) 910

**Full changelog**: https://github.com/dstackai/dstack/compare/0.15.1...0.15.2rc2

Kubernetes

With the latest update, it's now possible to configure a Kubernetes backend. In this case, if you run a workload, dstack will provision infrastructure within your Kubernetes cluster. This may work with both self-managed and managed clusters.

Specifying a custom VPC for AWS

If you're using dstack with AWS, it's now possible to configure a `vpc_name` via `~/.dstack/server/config.yml`.

** Learn more about the new features in detail on the [changelog](https://dstack.ai/changelog/0.15.1/) page.**

What's changed

* Print total offers count in run plan by Egor-S in https://github.com/dstackai/dstack/pull/862
* Add OpenAPI reference to the docs by Egor-S in https://github.com/dstackai/dstack/pull/863
* Fixes 864 by pinning the APScheduler dep to < 4 by tleyden in https://github.com/dstackai/dstack/pull/867
* Support gateway creation for Kubernetes by r4victor in https://github.com/dstackai/dstack/pull/870
* Improve `get_latest_runner_build` by Egor-S in https://github.com/dstackai/dstack/pull/871
* Added ruff by TheBits in https://github.com/dstackai/dstack/pull/850
* Handle ResourceNotExistsError instead of 404 by r4victor in https://github.com/dstackai/dstack/pull/875
* Simplify Kubernetes backend config by r4victor in https://github.com/dstackai/dstack/pull/879
* Add SSH keys to GCP metadata by Egor-S in https://github.com/dstackai/dstack/pull/881
* Allow to configure VPC for an AWS backend by r4victor in https://github.com/dstackai/dstack/pull/883

New contributors

* tleyden made their first contribution in https://github.com/dstackai/dstack/pull/867

**Full Changelog**: https://github.com/dstackai/dstack/compare/0.15.0...0.15.1