Dstack

Intel Gaudi

`dstack` now supports [Intel Gaudi](https://habana.ai/products/) accelerators with SSH fleets.

To use Intel Gaudi with `dstack`, [create an SSH fleet](https://dstack.ai/docs/concepts/fleets/#ssh), and once it's up, feel free to specify `gaudi`, `gaudi2`, or `gaudi3` as a GPU name (or `intel` as a vendor name) in your run configuration:

[^1]: My reference.

yaml
type: dev-environment

This release fixes a bug introduced in 0.18.36.

What's Changed
* [Docs]: Update Distributed Tasks docs by jvstme in https://github.com/dstackai/dstack/pull/2212
* Fix Internal Server Error in services w/o gateways by jvstme in https://github.com/dstackai/dstack/pull/2225


**Full Changelog**: https://github.com/dstackai/dstack/compare/0.18.36...0.18.37

Vultr

Cluster placement

The [`vultr`](https://dstack.ai/docs/concepts/backends/#vultr) backend can now provision fleets with [cluster placement](https://dstack.ai/docs/concepts/fleets/#cloud-placement).
yaml
type: fleet

nodes: 4
placement: cluster

resources:
gpu: MI300X:8

backends: [vultr]


Nodes in such a cluster will be interconnected and can be used to run [distributed tasks](https://dstack.ai/docs/concepts/tasks/#distributed-tasks).

Performance

The update optimizes the performance of `dstack server`, allowing a single server replica to handle up to 150 active runs, jobs, and instances. Capacity can be further increased by using [PostgreSQL](https://dstack.ai/docs/guides/server-deployment/#postgresql) and running multiple server replicas.

Last, getting instance offers from backends when you run `dstack apply` has also been optimized and now takes less time.

What's changed

* Increase max active resources supported by server by r4victor in https://github.com/dstackai/dstack/pull/2189
* Implement bridge network mode for jobs by un-def in https://github.com/dstackai/dstack/pull/2191
* [Internal] Fix `python-json-logger` deprecation warning by jvstme in https://github.com/dstackai/dstack/pull/2201
* Fix local backend by r4victor in https://github.com/dstackai/dstack/pull/2203
* Implement offers cache by r4victor in https://github.com/dstackai/dstack/pull/2197
* Add `/api/instances/list` by jvstme in https://github.com/dstackai/dstack/pull/2199
* Allow getting by ID in `/api/project/_/fleets/get` by jvstme in https://github.com/dstackai/dstack/pull/2200
* Add termination reason and message to the runner API by r4victor in https://github.com/dstackai/dstack/pull/2204
* Add vpc cluster support in Vultr by Bihan in https://github.com/dstackai/dstack/pull/2196
* Fix instance_types not respected for pool instances by r4victor in https://github.com/dstackai/dstack/pull/2205
* Delete manually created empty fleets by r4victor in https://github.com/dstackai/dstack/pull/2206
* Return repo errors from runner by r4victor in https://github.com/dstackai/dstack/pull/2207
* Fix caching offers with GPU requirements by jvstme in https://github.com/dstackai/dstack/pull/2210
* Fix filtering idle instances by instance type by jvstme in https://github.com/dstackai/dstack/pull/2214
* Add more project URLs on PyPI by jvstme in https://github.com/dstackai/dstack/pull/2215


**Full changelog**: https://github.com/dstackai/dstack/compare/0.18.35...0.18.36

Vultr

This update introduces initial integration with [Vultr](https://vultr.com/). This cloud provider offers a diverse range of NVIDIA and AMD accelerators, from cost-effective fractional GPUs to multi-GPU bare-metal hosts.

shell
$ dstack apply -f examples/.dstack.yml

BACKEND REGION RESOURCES PRICE
1 vultr ewr 2xCPU, 8GB, 1xA16 (2GB), 50.0GB (disk) $0.059
2 vultr ewr 1xCPU, 5GB, 1xA40 (2GB), 90.0GB (disk) $0.075
3 vultr ewr 1xCPU, 6GB, 1xA100 (4GB), 70.0GB (disk) $0.123
...
18 vultr ewr 32xCPU, 375GB, 2xL40S (48GB), 2200.0GB (disk) $3.342
19 vultr ewr 24xCPU, 240GB, 2xA100 (80GB), 1400.0GB (disk) $4.795
20 vultr ewr 96xCPU, 960GB, 16xA16 (16GB), 1700.0GB (disk) $7.534
21 vultr ewr 96xCPU, 1024GB, 4xA100 (80GB), 450.0GB (disk) $9.589


See the [docs](https://dstack.ai/docs/concepts/backends/#vultr) for detailed instructions on configuring the `vultr` backend.

> [!NOTE]
> This release includes all dstack features except support for volumes and clusters. These features will be added in an upcoming update.

Vast.ai

Previously, the `vastai` backend only allowed using Docker images where `root` is the default user. This limitation has been removed, so you can now run [NVIDIA NIM](https://dstack.ai/examples/deployment/nim/) or any other image regardless of the user.

Backward compatibility

If you are going to configure the `vultr` backend, make sure you update all your dstack CLI and API clients to the latest version. Clients prior to 0.18.35 will not work when Vultr is configured.

What's changed

* [`dstack-shim`] Revamp logging and CLI by un-def in https://github.com/dstackai/dstack/pull/2176
* Download `dstack-runner` to a well-known location by un-def in https://github.com/dstackai/dstack/pull/2179
* Add Vultr Support by Bihan in https://github.com/dstackai/dstack/pull/2132
* Support non-root Docker images in Vast.ai by jvstme in https://github.com/dstackai/dstack/pull/2185
* Refactor idle instance termination by jvstme in https://github.com/dstackai/dstack/pull/2188
* Retry instance termination in case of errors by jvstme in https://github.com/dstackai/dstack/pull/2190
* Update PyPI Development Status classifier by jvstme in https://github.com/dstackai/dstack/pull/2192
* Add Vultr to `Concepts` and `Reference` pages by Bihan in https://github.com/dstackai/dstack/pull/2186


**Full changelog**: https://github.com/dstackai/dstack/compare/0.18.34...0.18.35

Idle duration

If provisioned fleet instances aren’t used, they are marked as `idle` for reuse within the configured idle duration. After this period, instances are automatically deleted. This behavior was previously configured using the `termination_policy` and `termination_idle_time` properties in run or fleet configurations.

With this update, we replace these two properties with `idle_duration`, a simpler way to configure this behavior. This property can be set to a specific duration or to `off` for unlimited time.

yaml
type: dev-environment
name: vscode

This update fixes TPU v6e support and a potential gateway upgrade issue.

What's Changed
* Fix runtime version for TPU v6e by r4victor in https://github.com/dstackai/dstack/pull/2149
* Update `state.json` migration on gateways by jvstme in https://github.com/dstackai/dstack/pull/2152
* Optimize gateway startup and service update time by jvstme in https://github.com/dstackai/dstack/pull/2153


**Full Changelog**: https://github.com/dstackai/dstack/compare/0.18.32...0.18.33