Dstack

Git

Previously, when you called `dstack init`, Git credentials were reused between users of the same project and repository.

Starting with this release, to improve security, `dstack` no longer shares Git credentials across users.

> [!WARNING]
> If you submitted credentials earlier with `dstack init`, they will continue to work. However, it is recommended that each user call `dstack init` again to ensure they do not reuse credentials from other users.

<details>
<summary>Deleting legacy credentials</summary>

To ensure no credentials submitted earlier are shared across users, you can run the following SQL statements:

sql
UPDATE repos SET creds = NULL;


</details>

UI

This update brings a few UI improvements:

* Added `Delete` button to the `Volumes` page
* Added `Refresh` button to all pages with lists: `Runs`, `Models`, `Fleets`, `Volumes`, `Projects`
* Improved `Code` button on the model page

What's changed

* Implement per-user repo creds storage by un-def in 2004
* [UI] Add Refresh button to all pages with lists by olgenn in 2007
* [UI] Include base URL and authentication token in the code snippets by olgenn in 2006
* [UI] The Code button improvements on the Model page by olgenn in 2001
* [UI] It's not possible to select and delete volumes by olgenn in 2000
* [UI] [Bug]: Services without model mapping are displayed in Models UI by olgenn in 1993
* Ensure sshd privsep dir in container is properly set up by un-def in 2008
* [Docs] Many minor improvements to docs and examples by peterschmidt85 in 2013
* [Docs] Services without a gateway by jvstme in 2011
* [Docs] Add deployment section with vLLM, TGI and NIM. Remove alignment handbook by Bihan in 1990
* [Docs] Updated Installation and Server deployment guides to include CloudFormation by peterschmidt85
* [Docs] Update services docs to reflect that gateway is now optional by peterschmidt85 in 2005
* [Examples] Add a CloudFormation template showing how to deploy dstack server to AWS by peterschmidt85 in 1944
* [Examples] Add Airflow example by r4victor in 1991

**Full changelog**: https://github.com/dstackai/dstack/compare/0.18.25...0.18.26

Multiple volumes per mount point

It's now possible to specify a list of volumes for a mount point in run configurations:

yaml
...
volumes:
- name: [my-aws-eu-west-1-volume, my-aws-us-east-1-volume]
path: /volume_data


`dstack` will choose and mount one volume from the list. This can be used to increase GPU availability by specifying different volumes for different regions, which is desirable for use cases like caching. Previously, it was possible to specify only one volume per mount point, so if there was no compute capacity in the volume's region, provisioning would fail.

`DSTACK_NODES_IPS` environment variable

A new `DSTACK_NODES_IPS` environment variable is now available for multi-node tasks. It contains a list of internal IP addresses of all nodes in the cluster, e.g. `DSTACK_NODES_IPS="10.128.0.47\n10.128.0.48\n10.128.0.49"`. This feature enables cluster workloads that require configuring IP addresses of all the nodes.

What's Changed
* Adding an example of NIM by deep-diver in https://github.com/dstackai/dstack/pull/1853
* Support specifying multiple volumes per mount point by r4victor in https://github.com/dstackai/dstack/pull/1983
* Expose DSTACK_NODES_IPS env var by r4victor in https://github.com/dstackai/dstack/pull/1985
* Set minimum paramiko version to 3.2.0 by un-def in https://github.com/dstackai/dstack/pull/1984
* Limit azure-mgmt-network>=23.0.0,<28.0.0 by r4victor in https://github.com/dstackai/dstack/pull/1988


**Full Changelog**: https://github.com/dstackai/dstack/compare/0.18.24...0.18.25

Backward compatibility

This update includes a hotfix for a backward compatibility issue that prevented CLI v0.18.23 from working with older versions of the dstack server.

What's changed

* Fix backward compatibility broken in 0.18.23 by jvstme in https://github.com/dstackai/dstack/pull/1974

**Full changelog**: https://github.com/dstackai/dstack/compare/0.18.23...0.18.24

Gateway is optional

Previously, running any service required setting up a gateway. With this update, a gateway is no longer needed to run a service for development purposes.

Service endpoint

* If no gateway is created, the service’s endpoint will be accessible at `<dstack server URL>/proxy/services/<project name>/<run name>/`.
* If a service has a model mapping, the model will be accessible via the OpenAI-compatible endpoint at `<dstack server URL>/proxy/models/<project name>/`.

> [!NOTE]
> While this change makes it much easier to use services for development, you will still need a gateway if you want to use a custom domain, enable HTTPS, or use auto-scaling.

Gateway property

If a gateway is created but isn’t needed for a service, set the `gateway` property to `false`. If you have multiple gateways, you can choose one by setting `gateway` to the name of the gateway.

Model mapping

If the model is in OpenAI format, you can now use a shorter syntax for model mapping—simply set the `model` property to the model's name.

yaml
type: service

image: ollama/ollama
commands:
- ollama serve &
- sleep 3
- ollama pull llama3.1
- fg
port: 11434

model: llama3.1


The [longer syntax](https://dstack.ai/docs/reference/dstack.yml/service/#modelformatopenai) with more settings remains available.

Updating running services

Previously, updating a service’s configuration required restarting it. Now, you can update the `replicas` and `scaling` properties in place. Just run `dstack apply`, and the changes will take effect. New replicas will be created while the old ones continue running.

What's changed

* [dind] Update `start-dockerd` script by un-def in https://github.com/dstackai/dstack/pull/1928
* Add `/proxy` prefix to dstack-proxy endpoints by jvstme in https://github.com/dstackai/dstack/pull/1939
* [shim] Unmount volumes when run exits by un-def in https://github.com/dstackai/dstack/pull/1937
* Return error when instance added to multiple fleets(1699) by swsvc in https://github.com/dstackai/dstack/pull/1938
* unify project administration by olgenn in https://github.com/dstackai/dstack/pull/1946
* [shim] Change NVIDIA GPU detection method by un-def in https://github.com/dstackai/dstack/pull/1945
* Support service scaling via in-place updates by r4victor in https://github.com/dstackai/dstack/pull/1958
* [Docs] Document `resources.gpu.vendor` property by un-def in https://github.com/dstackai/dstack/pull/1957
* Fix SSH fleet hosts validation by un-def in https://github.com/dstackai/dstack/pull/1955
* Support chat models in `dstack-proxy` by jvstme in https://github.com/dstackai/dstack/pull/1953
* Add user tag to AWS and GCP volumes by james-boydell in https://github.com/dstackai/dstack/pull/1948
* Fix dstack-proxy dependencies by jvstme in https://github.com/dstackai/dstack/pull/1959
* Support DSTACK_SERVER_ADMIN_TOKEN env by r4victor in https://github.com/dstackai/dstack/pull/1960
* Fix migration `82b32a135ea2` by un-def in https://github.com/dstackai/dstack/pull/1962
* Fix dstack apply runs with new names by r4victor in https://github.com/dstackai/dstack/pull/1964
* [Blog] Introducing instance volumes to persist data on instances by peterschmidt85 in https://github.com/dstackai/dstack/pull/1965
* [UI]: Support in-server model proxy by olgenn in https://github.com/dstackai/dstack/pull/1966
* Short model mapping syntax by jvstme in https://github.com/dstackai/dstack/pull/1967
* Fix VolumeModel.user not loaded for volume detach by r4victor in https://github.com/dstackai/dstack/pull/1970
* Drop the `PROXY` feature flag by jvstme in https://github.com/dstackai/dstack/pull/1971
* Allow specifying gateway in service configurations by jvstme in https://github.com/dstackai/dstack/pull/1972
* Improve error handling in model proxy by jvstme in https://github.com/dstackai/dstack/pull/1973

New contributors

* james-boydell made their first contribution in https://github.com/dstackai/dstack/pull/1948

**Full changelog**: https://github.com/dstackai/dstack/compare/0.18.22...0.18.23

Custom OS images on AWS

You can now configure your own AMIs for the AWS backend.

yaml
projects:
- name: main
backends:
- type: aws
creds:
type: default
os_images:
cpu:
name: my-cpu-ami
user: admin
nvidia:
name: my-nvidia-ami
user: ubuntu


This can be used as an alternative way to bring your software or data to the AWS instance and mount it into your runs using [Instance volumes](https://dstack.ai/docs/concepts/volumes/#instance-volumes).

See the [AWS backend reference](https://dstack.ai/docs/reference/server/config.yml/#aws) for details on configuring OS images. Support for custom OS images in other backends is coming in future releases.

What's Changed
* [Blog] Docker and Docker Compose inside container by peterschmidt85 in https://github.com/dstackai/dstack/pull/1916
* [Examples] Update Chat UI compose.yaml by un-def in https://github.com/dstackai/dstack/pull/1919
* [Bug]: [UI] Dark YAML editor theme won't work bug ui by olgenn in https://github.com/dstackai/dstack/pull/1923
* Remove Cloud NAT check when provisioning by r4victor in https://github.com/dstackai/dstack/pull/1925
* Allow to customize AMIs used by AWS backend by un-def in https://github.com/dstackai/dstack/pull/1920
* Fix Azure hostname assignment by r4victor in https://github.com/dstackai/dstack/pull/1930
* Support GCP Shared VPC for some subnets by r4victor in https://github.com/dstackai/dstack/pull/1933
* Increase request body size limit for services by jvstme in https://github.com/dstackai/dstack/pull/1934


**Full Changelog**: https://github.com/dstackai/dstack/compare/0.18.21...0.18.22

Instance volumes

In addition to [network volumes](https://dstack.ai/docs/concepts/volumes/#network-volumes), `dstack` now allows to mount instance (host) filesystems inside the run container. As contents of the instance volume are specific to the instance where the run is executed, such volumes can be used in cases where data persistence is not critical, for example, as a cache:

yaml
type: task

commands:
- pip install -r requirements.txt

volumes:
reuse pip cache between runs
- /dstack-cache/pip:/root/.cache/pip


See the [instance volumes](https://dstack.ai/docs/concepts/volumes/#instance-volumes) documentation for more information.

Azure custom and private networks

`dstack` now supports configuring custom Azure networks, which was only possible on AWS and GCP before. In addition, you can now configure `dstack` to provision instances without public IPs on Azure to take advantage of private networks:

yaml
type: azure
tenant_id: my_tenant_id
subscription_id: my_subscription_id
regions: [westeurope]
public_ips: false
vpc_ids:
westeurope: test-networks-rg/test-network
creds:
type: default


Read more about Azure networking configuration in the [docs](https://dstack.ai/docs/reference/server/config.yml/#azure).

Python 3.13 support for `dstack` package

The [previous 0.18.20 release](https://github.com/dstackai/dstack/releases/tag/0.18.20) added support for Python 3.13 in run configurations. This release updates the `dstack` package itself so that it works under Python 3.13. The `dstack` package also drops Python 3.8 support that [reached end of life](https://devguide.python.org/versions/). Note that `python: 3.8` in run configurations is deprecated but still supported.

Multi-job UI

The control plain UI now displays detailed info on each job in the run, improving support for multi-node tasks and replicated services:

<img width="1208" alt="Screenshot 2024-10-30 at 15 09 03" src="https://github.com/user-attachments/assets/5d3145f8-ec7a-45fd-9d8e-21f53b6d9406">

What's Changed
* Show all jobs in runs UI by olgenn in https://github.com/dstackai/dstack/pull/1887
* Revert the list of projects and users in Administration by olgenn in https://github.com/dstackai/dstack/pull/1888
* Fix instance price discrepancies in RunPod by jvstme in https://github.com/dstackai/dstack/pull/1891
* [Website] Change `Backends` to `Partners` by peterschmidt85 in https://github.com/dstackai/dstack/pull/1893
* Support `auth: true` services in dstack-proxy by jvstme in https://github.com/dstackai/dstack/pull/1885
* [Docs]: Fix typos by dheerajsir in https://github.com/dstackai/dstack/pull/1897
* Support custom and private networks for Azure by r4victor in https://github.com/dstackai/dstack/pull/1896
* Fix Dstack Server Deployment Link by SagarSharma101 in https://github.com/dstackai/dstack/pull/1898
* Add instance volumes by un-def in https://github.com/dstackai/dstack/pull/1895
* Add nat_check option to GCP config by r4victor in https://github.com/dstackai/dstack/pull/1904
* Handle deleted volumes in attach_volume() by r4victor in https://github.com/dstackai/dstack/pull/1907
* Fix log message on getting run volumes by r4victor in https://github.com/dstackai/dstack/pull/1909
* [dind] Improve start-dockerd script by un-def in https://github.com/dstackai/dstack/pull/1908
* Show warning on missing backend deps by r4victor in https://github.com/dstackai/dstack/pull/1911
* Support Python 3.13, drop Python 3.8 by jvstme in https://github.com/dstackai/dstack/pull/1910
* Update pydantic-duality to fix the infinite recursion bug by zmievsa in https://github.com/dstackai/dstack/pull/1902
* Set ping_interval on WebSocket connection by r4victor in https://github.com/dstackai/dstack/pull/1918

New Contributors
* dheerajsir made their first contribution in https://github.com/dstackai/dstack/pull/1897
* SagarSharma101 made their first contribution in https://github.com/dstackai/dstack/pull/1898
* zmievsa made their first contribution in https://github.com/dstackai/dstack/pull/1902

**Full Changelog**: https://github.com/dstackai/dstack/compare/0.18.20...0.18.21