Determined

Changelog

* 710d5575b chore: bump version: 0.18.4-rc4 -> 0.18.4
* 990e62b04 docs: add release notes for 0.18.4 (4547)
* a12fd4520 chore: bump version: 0.18.4-rc3 -> 0.18.4-rc4
* 57e4e7e8a chore: bump version: 0.18.4-rc2 -> 0.18.4-rc3
* 06dcaacc0 chore: bump version: 0.18.4-rc1 -> 0.18.4-rc2
* 09bf496ab fix: correct allocation end times when cluster heartbeat is before allocation start time (4556)
* 75d52378c chore: bump version: 0.18.4-rc0 -> 0.18.4-rc1
* 47f920731 fix: tensorboard redirecting due to missing rp (4542)
* 56bf894de chore: bump version: 0.18.4-dev0 -> 0.18.4-rc0
* 900b134cd chore: lock api state for backward compatibility check
* a14986bbf test: `set_priority` should respect the master url. (4527)
* 8a3c07a39 test: bring back tests for generic commands on master restarts. (4490)
* 73268ac6b docs: add ClusterInfo docs (4496)
* 9a34b3d48 chore: bumpenvs (4520)
* b1a292731 fix: ensure correct master url is used for priority scheduler and other managed devcluster tests. (4512)
* 667e313c8 fix: deepspeed examples config (4511)
* 2f6373ce5 style: add uplot cursor styles for dark mode (4502)
* 7657c15c1 feat: add props to style avatar differently (4514)
* 5272c0916 test: set up shared web to test with local strategy (4504)
* 2f0950317 chore: bump version: 0.18.3-dev0 -> 0.18.4-dev0
* 5f30b6714 docs: add release notes for 0.18.3 (4491)
* 758c2e929 test: fix local time tests (4503)
* 2a968bf7a fix: give a reasonable name to model def download (4493)
* 7e20b5d11 docs: minor improvements and Homebrew build instructions (4489)
* 38fe30220 chore: add helpers for working with shared-code (4452)
* 28c7b895d fix: make actor system `GetOrElseTimeout` actually timeout. (4499)
* 5a4198677 feat: environment_variables in task_container_defaults [DET-7638] (4485)
* 1184bb3dd test: fix editable metadata test flake (4495)
* 7898dc1a6 style: theme tuning [DET-7362, DET-7363, DET-7364, DET-7365, DET-7366, DET-7383, DET-7425, DET-7498, DET-7551] (4378)
* cd28b37a1 chore: fix react fmt (4487)
* 97d3e6029 refactor: update modals to better support contexts [DET-6297] (4468)
* 855e8324b fix: allocation errors on finished hp search [DET-7724] (4467)
* ce296c5f4 fix: correct path selector for tensorboard upload (4474)
* c3d650984 fix: add priority scheduler e2e tests [DET-7106] (4429)
* 7e50a0416 fix: close button should not show in non-embedded log viewer [DET-7723] (4447)
* 1d61f3a32 fix: adjust spinner for the InlineEditor (4367)
* a69813bfa feat: persist user web setting [DET-7501] (4394)
* be48f7654 feat: support non-root init container k8s [DET-7109] (4460)
* 2015f145e fix: disable pointer events on log viewer spinner block (4470)
* db65bf45a test: tweak `test_agent_reconnect_keep_experiment` test timeouts. (4465)
* 5f1340767 chore: bump the resources for react tests (4459)
* 60ec81f14 fix: sync agent actor init on master restart [DET-7746] (4463)
* 473f7b620 fix: loadingState cleanup for WorkspaceList (4461)
* e1fd06275 feat: add spinner to tiral/experiment tabs to show data fetching (4436)
* d123d90c3 ci: fix autolabeling for shared (4456)
* 7347bb66b test: update ci job for testing shared-code (4444)
* 6450aa140 docs: workspaces docs (4448)
* a1a06c4ac fix: trim workspace and project name whitespace [DET-7390] [DET-7747] (4451)
* 908cdb19e docs: PyTorchTrialCallback.on_validation_end (4457)
* e9144916c ci: add codeowners for shared directory (4455)
* d28d1c245 chore: share AvatarCard and Avatar comps [DET-7714 DET-7713] (4430)
* efc5ffb06 fix: only call move API for permissioned experiments (4445)
* 62b9a2df3 fix: map name to name instead of description in getProjectExperiments (4450)
* 6475a19c7 fix: mac path for --agent-config-path (4449)
* df830a118 fix: long tag wrap (4446)
* 10022e8ed fix: various workspaces fixes (4443)
* 6790aee73 feat: master config option for additional fluent outputs [DET-7549] (4415)
* c0ddb8dbd chore: webui changes for slurm (4427)
* 4df29fcd1 chore: resolve warnings from yaml and numpy, fix supposed fstrings, black formatting (4372)


Docker images

- `docker pull determinedai/determined-master:0.18.4`
- `docker pull determinedai/determined-master:710d5575b`
- `docker pull determinedai/determined-master:710d5575b8565fc50b5e65143b0b27dd661b0d17`
- `docker pull determinedai/determined-dev:determined-master-710d5575b`
- `docker pull determinedai/determined-dev:determined-master-710d5575b8565fc50b5e65143b0b27dd661b0d17`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:0.18.4`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:710d5575b`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:710d5575b8565fc50b5e65143b0b27dd661b0d17`

Changelog

* 5cf7e8b5a chore: bump version: 0.18.3-rc8 -> 0.18.3
* 6d19e1795 docs: add release notes for 0.18.3 (4491)
* a7cc461ff chore: bump version: 0.18.3-rc7 -> 0.18.3-rc8
* 6dba678bb fix: make actor system `GetOrElseTimeout` actually timeout. (4499)
* 250aca7b0 chore: bump version: 0.18.3-rc6 -> 0.18.3-rc7
* f02911147 chore: bump version: 0.18.3-rc5 -> 0.18.3-rc6
* 73fd9f09b fix: allocation errors on finished hp search [DET-7724] (4467)
* c85186be2 fix: correct path selector for tensorboard upload (4474)
* 2193001e0 feat: support non-root init container k8s [DET-7109] (4460)
* 40d4cd149 chore: bump version: 0.18.3-rc4 -> 0.18.3-rc5
* d996a96a0 fix: sync agent actor init on master restart [DET-7746] (4463)
* 97b5a0c5e fix: loadingState cleanup for WorkspaceList (4461)
* e2fb0f865 docs: workspaces docs (4448)
* d835b1e8a fix: trim workspace and project name whitespace [DET-7390] [DET-7747] (4451)
* bcb8b3ea9 chore: bump version: 0.18.3-rc3 -> 0.18.3-rc4
* e34bbe1b6 fix: map name to name instead of description in getProjectExperiments (4450)
* 9b86fc343 fix: mac path for --agent-config-path (4449)
* 0918a7123 fix: only call move API for permissioned experiments (4445)
* 5a8a4c462 chore: bump version: 0.18.3-rc2 -> 0.18.3-rc3
* 054d089b3 fix: long tag wrap (4446)
* 29f224e42 fix: various workspaces fixes (4443)
* c7d6b68bf chore: bump version: 0.18.3-rc1 -> 0.18.3-rc2
* 650cce517 chore: webui changes for slurm (4427)
* ca975c8b4 chore: bump version: 0.18.3-rc0 -> 0.18.3-rc1
* 761e6ad52 feat: master config option for additional fluent outputs [DET-7549] (4415)
* 759e5d643 chore: bump version: 0.18.3-dev0 -> 0.18.3-rc0
* ca0a7c116 chore: lock api state for backward compatibility check
* 4010b330b feat: experiment comparison page (4410)
* 030de4a39 fix: default max length properly to undefined (4437)
* 6e6420ec8 feat: task container checks for occupied gpus [DET-5091] (4323)
* e414235c9 test: resolve `useCustomizeColumnsModal` test flakes [DET-7635] (4433)
* bd576678e chore: update deepspeed launcher to work with newer versions (4156)
* 4277e991d Split 'webui/react/src/shared/' into commit '3302ddf5d45c5a73cdcbbc750936bfb32e025c06'
* 9fbf9d5d0 fix: hide continue trial for experiments with 0 or more than 1 trial (4434)
* 79b9d90b1 chore: Merge Slurm-infrastructure support changes (4376)
* 8b799e97d ci: send Docker image scans to the infrastructure engineering channel instead (4425)
* 7436c6c82 fix: properly close experiment create modal [DET-7553] (4421)
* 169c0d835 docs: clarify count requirement for grid search (4428)
* 47c393f5c test: tune web tests (4409)
* 2e8b92b27 feat: install a SIGUSR1 signal handler to print stacktraces (4281)
* bff6d1afc fix: registry auth in master.yaml (4412)
* 69d6b1e82 fix: Sync epoch_idx across all workers for PyTorchTrial [DET-7488] (4303)
* 3302ddf5d chore: make core styles and configs shareable [DET-7316] (4398)
* b8a7afcb0 chore: make core styles and configs shareable [DET-7316] (4398)
* 182af4956 fix: allocation state mismatch with proto [DET-7593] (4408)
* 2c4a77a57 chore: don't launch horovod for hypothetical zero slot trials (4414)
* e1c1eacf1 feat: shm_size can take a string with units like docker run allows (4314)
* c05ebd772 chore: migrate shared-web to subtree (4402)
* 050542264 chore: migrate shared-web to subtree (4402)
* cb1fc53fa fix: always flush tensorboard ready log (4387)
* c82d64c81 fix: End agent stats log to debug level (4411)
* b5e239714 fix: fix cluster resource pool selection [DET-7517 DET-7567 DET-7517] (4391)
* 6672fd8e1 chore: Merge Slurm-infrastructure support changes trial (4396)
* 6d0fe6bb1 fix: recover from permission error when deleting preexisting file or link (4397)
* 1e8bbac6d fix: move scale import (4406)
* 624d17904 chore: Add nolintlint to deadcode for unused oss method (4404)
* 8e50499b2 fix: Do not fail if no checkpoints to gc [FOUNDENG-83] (4405)
* e3ee40f8e fix: test agent config flake (4386)
* b701fcdf8 feat: Trial Metrics - Summary endpoints (4392)
* c9d9d5354 feat: add log scale to experiment viz (4273)
* 2d82f0148 chore: sync node version requirements with package-lock (4395)
* b2acc5daa chore: Update task logging setup [FOUNDENG-81] (290) (4389)
* 0ad82fd5f chore: Merge Slurm-infrastructure support changes task_trial (4393)
* 9149c1a49 chore: Merge Slurm-infrastructure support changes job.go (4388)
* 3d306378e chore: Merge Slurm-infrastructure support changes archive (4390)
* e9f3b1a9b feat: rolling upgrades support for generic command types (notebooks etc.) [DET-7218] (4371)
* b130e3f32 fix: Add compute driver capabilities to determined-agent Dockerfile (4385)
* 6d8ea5304 chore: Restrict display names [DET-7356] (4266)
* 72d4bcb22 fix: update experiment list tag truncate rule [DET-7530] (4373)
* c78e92eb1 feat: stress test for agent enable/disable and reconnect spam [DET-6733] (4269)
* e4a275b26 feat: add --agent-config-path to det deploy local agent-up [DET-6278] (4366)
* 7f40e8642 chore: handle setResourcePool message in commands (3571)
* 9a1ab33a3 style: ban `fmt.Println` and `fmt.Printf` in go code. (4369)
* cbc37cd74 chore: upgrade mockery to latest (4363)
* 76423ba86 fix: don't GC registered checkpoints [DET-7418] (4316)
* 81fed29cd chore: idempotent container running messages [DET-7555] (4364)
* 19f311e9e feat: workspaces and projects [DET-6461] (4203)
* d257b95e9 chore: update resource pool cloud icons [DET-7160] (4317)
* c1da1f29f chore: restrict access to master.yaml and agent.yaml to the user/owner only. (4299)
* 8d91f0fb5 ci: Add dependabot for all the things (4313)
* 058cf5cd5 ci: make test-unit-storage run on any upstream branch (4320)
* 49966efa5 feat: Tensorboard profiles from all hosts (4142)
* 762cee189 ci: merge separate codeowners (4322)
* 5e6ba22a8 chore: bump version: 0.18.2-dev0 -> 0.18.3-dev0
* bb5ccc123 docs: add release notes for 0.18.2 (4318)
* 13b6059c9 ci: create default codeowner file (4321)
* 0527c48dc fix: harness azure dependencies (4319)
* 869dea1f9 chore: upgrade to typescript 4.7 [DET-7499] (4285)
* 9c94d1fce ci: use parallelism for `test-e2e-managed-devcluster`. (4193)
* 98e4b0bd9 chore: delete dead code (4312)
* 3df0fb13c fix: provide unique port offsets to trials (4315)
* c5eb030e5 chore: drop pbt, adaptive and adaptive_simple from docs and web (4311)
* ca57ffcd6 chore: cleanup container terminations resent on agent exit (4309) [DET-7533]
* 11d17fe47 chore: add logging for e2e gpu test (4232)
* b14c28142 docs: more consistently use Determined AI vs. Determined (4310)
* 0a93e76dc fix: skip allocation check on checkpoint save to fix pulling preemption (4308)
* 4610d7071 ci: skip e2e_tests for webui
* 2e9bae652 fix: remove limit of length 4 for task id (4304)
* ee6b118d5 chore: use vanilla markdown in examples (4268)
* db5cd90d9 fix: harness dependency class fore azure (4302)
* 19aa55c61 chore: stamp out unexpected messages [DET-7492] (4284)
* 342236a33 chore: FOUNDENG-55 For deepspeed, set the NCCL_SOCKET_IFNAME env variable based on dtrain_network_interface (4297)
* 69ff97386 ci: fix log message grepped for by test_launch_layer_cifar (4301)
* 1299ef395 ci: remove overzealous assert that is a race with a quick noop task (4296)
* 5c2830431 docs: move description of agent client cert fields to the right place (4293)


Docker images

- `docker pull determinedai/determined-master:0.18.3`
- `docker pull determinedai/determined-master:5cf7e8b5a`
- `docker pull determinedai/determined-master:5cf7e8b5a6a8393b04c1d54a5363cbbe6e8792d2`
- `docker pull determinedai/determined-dev:determined-master-5cf7e8b5a`
- `docker pull determinedai/determined-dev:determined-master-5cf7e8b5a6a8393b04c1d54a5363cbbe6e8792d2`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:0.18.3`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:5cf7e8b5a`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:5cf7e8b5a6a8393b04c1d54a5363cbbe6e8792d2`

Changelog

* d214a34df chore: bump version: 0.18.2-rc2 -> 0.18.2
* 10b172bcc docs: add release notes for 0.18.2 (4318)
* 38e0efb15 chore: bump version: 0.18.2-rc1 -> 0.18.2-rc2
* c5cb23fc9 fix: harness azure dependencies (4319)
* f623c1c1e chore: bump version: 0.18.2-rc0 -> 0.18.2-rc1
* 350de3838 chore: cleanup container terminations resent on agent exit (4309) [DET-7533]
* f02f11bc5 fix: skip allocation check on checkpoint save to fix pulling preemption (4308)
* cfc0f8108 fix: remove limit of length 4 for task id (4304)
* 0645ba968 fix: harness dependency class fore azure (4302)
* 9991528a2 chore: bump version: 0.18.2-dev0 -> 0.18.2-rc0
* 86b6c28a4 chore: lock api state for backward compatibility check
* 271ae41c7 chore: ensure container terminations aren't lost in the event of network failures [DET-7440, DET-7441] (4272)
* 2d5a11865 fix: check for errors from actor asks in createResourcePoolSummary [DET-7494] (4294)
* daee03661 fix: stale agent state crashes resource pool [DET-7493] (4295)
* 4722caa95 fix: limit waiting for logger to 30 seconds (4289)
* b8f9463c6 fix: jupyterlab modal stale values, not sending values to api
* 42615b4b1 feat: add cli and api for delete checkpoints [DET-7119], [DET-7120] (4246)
* ad9bcd809 chore: restore determined_version in Trial checkpoint metadata (4288)
* 0b7dcdbd2 chore: widen node version detection to 16.29 (4290)
* 118a850da fix: pass signals through wrapper processes (4286)
* 0089ede8c fix: enable Full Configuration in appropriate modals [DET-7495] (4280)
* 45c62f20c feat: show only selected trials in learning curve
* 922b11349 chore: reduce use of any in InteractiveTable
* 5b03672a4 chore: update docs to accurate node support (4279)
* 59f686c96 fix: make trials table sort properly (4275)
* 695b30117 feat: incrementally release resources (4278)
* c0f53fa26 perf: avoid Seq Scanning raw steps (4244)
* b36596f13 fix: backslashes in show_ssh_command for windows (4260)
* ec503e06d fix: correct error message when command list fails (4235)
* 15b0525fc feat: add master-side verification for agent mTLS (4220)
* c6e824937 fix: don't hardcode /bin/which in entrypoints (4257)
* 236c24117 refactor: migrating tables to use InteractiveTable component [DET-7382] (4229)
* 156ff504b fix: reshow drop targets for customize columns modal (4199)
* 5e9962686 docs: delete mnist_tf_layers example (4263)
* 9d8223113 Revert "add autosync action"
* 8288461f5 add autosync action
* 70e534f9e fix: det task logs unable to use trial task IDs and checkpoint GC task IDs [DET-7424] (4258)
* 12d21c2e0 fix: gcs storage upstream test failure (4253)
* b09ec407e feat: improve trial logs to have some system events [DET-5885] (4215)
* c20ad4334 ci: Lints migrations to ensure new migrations have higher timestamps than old ones [DET-7146] (4250)
* da83e6509 chore: small cleanups for slurm (4211)
* b71f179aa feat: det deploy now can use --yes to skip prompts [DET-7408] (4255)
* 7b3fee44c chore: better slurm option override support (4254)
* 7af3cfe48 feat: add google cloud storage (gcs) prefix support [DET-6883] (4238)
* b5c2b14f8 docs: add enhanced launcher user guide (4248)
* eb1d9aaac chore: add dev server support for embedded tasks view (4243)
* 9d19a8704 perf: dont repeatedly reprocess profiler data
* b6582c02b ci: add check to prevent ssh git url (4240)
* 50bdbc3a8 chore: bumpenvs (4239)
* 1065314ea chore: add node_modules to eslintignore (4237)
* c74dee646 feat: Add theme toggle to user settings [DET-7321] (4204)
* 3016fc103 chore: remove legacy code/docs for NCCL/Gloo port range config (4187)
* f1e8d3cdc ci: check ulimit before 4x4 distributed test on macs (4234)
* 7d9e999cd fix: adjust page to preserve props.children (4231)
* 35258efd7 feat: Enable sending empty string for displayname with fallback to username [DET-7031] (4140)
* 2e77ec58c fix: det shell start/open, in windows (4227)
* 7217cfd94 feat: k8s detect non-det tasks (4154)
* a88f5c450 chore: share webui base page (4218)
* 15bd75895 fix: use custom image for tensor board [DET-7242] (4123)
* de926b1eb chore: fix rendezvous timeout logic (4226)
* c29e97a45 chore: base Dockerfile TensorFlow 2.6, 2.7, 2.8 security patches [DET-7325] (4223)
* 7882381cd fix: authenticate pprof endpoints [DET-7402]
* 2b0ccb830 chore: bump version: 0.18.1-dev0 -> 0.18.2-dev0
* bcbab4d92 docs: add release notes for 0.18.1 (4216)
* 034b9570c chore: revert rename of `RestoreResourcesFailure -> ResourcesFailure`. (4210)
* a7c4c2afe feat: enable agent-side mTLS for connection to master (4212)
* c9e13b64a feat: save connection in context (4213)
* 15f65ab37 feat: pix2pix example (4125)
* db987de7d chore: delete "conditional" json-schema extension (4177)
* 9b33f825c fix: use bigint for checkpoint size in `proto_get_trials_plus` (4208)
* 2c4c847d7 docs: update release note instructions with important admonition (4207)
* 138caf8f3 fix: pool detail page tab count when loading (4200)
* c5f685f18 feat: move task logs to embedded view [DET-7169] (4179)
* 54982c9be perf: tweak proto_get_trials_plus plan (4206)
* e585ebbb1 refactor: cleanup task logging shell scripts (4113)
* 4e2913f4a chore: update entrypoint in expconf docs (4198)
* fcff1c2aa fix: agent panic on commands with unusal formatted environment variables [DET-6649] (4202)
* 4172a4690 refactor: pull in user service code changes from EE (4183)
* 1c48fa618 docs: improve OpenTelemetry docs slightly (4182)
* 7f508e462 fix: allow `internal: null` for pre-0.15.6 experiments (4197)
* 55957fe06 fix: add restarts back to get_trial_ids for sorting
* dd8d3f377 feat: add det experiment logs <EXP_ID> [DET-7145] (4190)
* 3ef36d56b chore: refactor action dropdown comp to be reused [DET-7171] (4164)
* 623e60d37 ci: bust circleci cache (4189)
* af56e0103 docs: document using AWS Load Balancer on EKS [DET-6669] (4174)
* 36e56670d feat: allow enabling Prometheus monitoring through helm [DET-6993] (4158)
* 3bb7bb1d9 style: minor theme fixes and style adjustments [DET-7349] (4161)
* 2166dfc88 docs: update screen shots for cluster UI (4188)
* b7a327819 style: address new `flake8-comprehensions`, `pyzmq==23.0.0`. (4185)
* 5e1a81cb2 feat: allow setting of checkpointStorage.prefix through helm [DET-7152] (4152)
* 526e1dcc1 feat: display trial restarts [DET-7347] (4160)
* 2fdc6d710 fix: agent can now be control-C while connecting to master [DET-6287] (4178)
* d339f7be2 chore: migrate det a list to new api and bindings (4186)
* ed257d38e refactor: rip out `UseFluentLogging`. (4184)
* 21b85907b docs: update fluent-bit version. (4181)
* 4b325a558 docs: document database SSL options (4169)
* 5d228f8a5 chore: make core-api tutorial Windows-friendly (4176)
* 187505148 fix: sync slot usage for k8s [DET-7350] (4172)
* 92a944f96 chore: add .dccache to .gitignore (4173)
* 5e7a30c43 docs: fix typo in release note (4170)
* a52210f9e feat: chart sync provider [DET-7309] (4139)
* a7fafbb9a fix: enable currently active side nav item (4167)
* 0a5d54d45 chore: fix hardcoded url in schema logic (4171)
* 7df89b293 chore: allow deleting delete failed experiments (4141) [DET-7070]
* 4ef2b6771 perf: fixup query for latest training per trial (4166) [DET-7352]
* 3d3fe1cd0 fix: include both old and new checkpoints in total checkpoint size (4165)
* 4d98cee3d fix: replace carriage returns with newlines in task output [DET-5302] (3945)
* a2f878aaf chore: only warn on invalid calls to daemonize resources for slurm (4108)
* c00ce0a4c chore: check git state in lock-api-state.sh (4163)
* ec743d754 ci: turn off github annotations. (4146)
* 0efd44d08 doc: fix a broken file reference (4131)
* b911d85ed fix: avoid potential race between AllocationReady and Running state (4159)
* cc59985dc revert: partial revert of 96e0e584 (4162)
* f07633b4e fix: port collisions for multiple shared-non distributed jobs (4120) [HAL-2894]
* 80d1bb152 feat: Add embedded experience for JupyterLab and TensorBoard [DET-7162] (4134)
* 83cc1ec8f fix: prevent experiment name in header from flowing entire vertical space of screen during resize (4157)


Docker images

- `docker pull determinedai/determined-master:0.18.2`
- `docker pull determinedai/determined-master:d214a34df`
- `docker pull determinedai/determined-master:d214a34df0c0eb2e5e38ae63d1359862fd2af8f1`
- `docker pull determinedai/determined-dev:determined-master-d214a34df`
- `docker pull determinedai/determined-dev:determined-master-d214a34df0c0eb2e5e38ae63d1359862fd2af8f1`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:0.18.2`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:d214a34df`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:d214a34df0c0eb2e5e38ae63d1359862fd2af8f1`

Changelog

* 9284a3aa6 chore: bump version: 0.18.1-rc7 -> 0.18.1
* eb09c3449 docs: add release notes for 0.18.1 (4216)
* bad9a336a chore: bump version: 0.18.1-rc6 -> 0.18.1-rc7
* 6cff1b0a5 fix: use bigint for checkpoint size in `proto_get_trials_plus` (4208)
* 60291e934 chore: bump version: 0.18.1-rc5 -> 0.18.1-rc6
* 8f4a7977d perf: tweak proto_get_trials_plus plan (4206)
* 414bcd2a4 chore: bump version: 0.18.1-rc4 -> 0.18.1-rc5
* 789b39c9c fix: allow `internal: null` for pre-0.15.6 experiments (4197)
* fd27bac05 fix: add restarts back to get_trial_ids for sorting
* 845f2f06e chore: bump version: 0.18.1-rc3 -> 0.18.1-rc4
* eed09e911 docs: update screen shots for cluster UI (4188)
* 88271dd14 style: minor theme fixes and style adjustments [DET-7349] (4161)
* f2a4e5ee4 feat: display trial restarts [DET-7347] (4160)
* eaf84e681 chore: bump version: 0.18.1-rc2 -> 0.18.1-rc3
* a8ddc8208 fix: sync slot usage for k8s [DET-7350] (4172)
* 1f13710b3 fix: enable currently active side nav item (4167)
* e9333d2f1 perf: fixup query for latest training per trial (4166) [DET-7352]
* 764ef2d06 fix: include both old and new checkpoints in total checkpoint size (4165)
* 4157c82a9 chore: bump version: 0.18.1-rc1 -> 0.18.1-rc2
* e2f949ab2 chore: bump version: 0.18.1-rc0 -> 0.18.1-rc1
* 26ede20fb chore: revert scheduling docs
* 8ea2a5256 fix: prevent experiment name in header from flowing entire vertical space of screen during resize (4157)
* 1e244e048 chore: bump version: 0.18.1-dev0 -> 0.18.1-rc0
* 96e0e5840 chore: lock api state for backward compatibility check
* 82f0366dc feat: allow NaN validation metrics [DET-7177] (4150)
* 0bbeec1f7 feat: upload all tb files DET-7139 (4155)
* 90b918acf fix: adjust upscaling of column widths [DET-7220] (4138)
* 4b66bf07f feat: rolling upgrades v0 [DET-6548] (4031)
* beea2451a ci: disable most checks on ci-only changes (4118)
* 5f7e74a6c fix: upstream test failures due to config being admin protected (4153)
* da1dcd76a fix: return user data when new user is created [DET-7255] (4149)
* 1eba7a20c docs: a vain attempt to pass ci test on already approved pr4110 content changes (4151)
* 7aea015d0 fix: No redirecting url when model name is changed (4127)
* 00171f82b feat: Cluster UI improvement [DET-7072, DET-7073] (4009)
* 27e04e4d8 feat: require admin privileges for cluster managment [DET-7186] (4129)
* 09a8ff698 ci: update gke version. (4147)
* fa3a95960 ci: Increase package-and-push-system-local resource class (4143)
* 0d4fe2385 chore: fix boolean urlparams for grpc (4136)
* b1829b8c8 feat: enable SLURM preemption (4114) [FOUNDENG-21]
* b4ef2735f chore: add a local docs server (4117)
* 5dea21112 refactor: theme architecture [DET-6211] (4004)
* aef66b088 build: make docs build incremental and idempotent (4116)
* ec7007cef ci: persist debs and rpms in circleci for dev, rc, and release builds (4124)
* b89b0a359 fix: user filter on dashboard [DET-7251] (4132)
* e3e50a916 chore: add job ID and experiment labels to prometheus endpoint mappings [DET-6964] (4119)
* b0d8a935e ci: make codecov information for sure now. (4130)
* b313503d0 chore: restructure shareable webui utils and types (4112)
* 78505d6e6 ci: turn off codecov bot PR comments (4122)
* a91866fdb chore: fix rank determination for horovod with mpi (4109)
* 3368a7727 fix: NCCL interface in distributed tests (4111)
* 18aadd0dd chore: bump version: 0.18.0-dev0 -> 0.18.1-dev0
* 7500d6fc6 docs: add release notes for 0.18.0 (4102)
* 1f4a64260 chore: bump version: 0.17.16-dev0 -> 0.18.0-dev0
* 59928ec4f chore: explicit naming of preemption and coscheduler resources [DET-7140] (4101)
* ac2564b87 fix: add missing task log teardown for trials (4107)
* c41e1dcc0 docs: rework quickstart for ml developers (4091)
* 2c7564d13 chore: use reported slots available for on prem deployments (4095)
* 62049aeab chore: change codecov to informational only (4105)
* 5061a8fd3 fix: mark distributed tests as parallel (4093)
* 87b8e5396 chore: enable codecov enforcement (4084)
* 15dd7e3d4 fix: bindings sessions in experiment apis. (4096)
* eb65b09ff chore: cleanup and fixes for "det deploy" (4103)
* d78a7120a perf: improve plan for proto_get_trial_plus.sql (4073)
* a980c560b build: update submodules on webui get-deps (4082)
* 78d4e8b7d chore: HAL-2879 Cleanly shutdown all sshd servers on exit (176) (4087)
* 940f8f703 chore: Refactor JupyterLabModal pattern [DET-6276] (4072)
* ec5553ab4 chore: make container proxy support more flexible, for slurm (3948)
* 3614c83a6 chore: wait for process substition log filters [DET-6712] (3930)
* 474742f9a chore: clean up useCallback dependency (4092)
* 39588ee25 feat: add det.LOG_FORMAT constant (4090)
* bcc50f97c fix: Support rendering rank of 0 (4083)
* c523523e8 fix: consistent total slot calculation for cluster overview [DET-7182] (4080)
* 9c163497c feat: add wrap_rank helper script (4086)
* ea4a94911 fix: dont show archived in column picker [DET-7187] (4085)
* a4e5f8410 feat: authenticate task proxies (4071)
* 5fab38417 fix: wrap torch.distributed launch in pid server/client (4077)
* e73f06309 fix: use displayNames in ClusterHistoricalUsage (4059)
* 3368dd86a chore: filter out NaN, +/- Infinity metric values for charts for now. (4076)
* f8b5bf50e chore: add and consolidate code coverage to codecov (4064)
* 487b04c21 docs: release note for core api (4069)
* 15a668be9 fix: add user column back to experiment list (4070)
* 880b769ec feat: break workload info from trial endpoint into a new endpoint [DET-6729] (3635)
* d703c968c fix: show notification when delete experiment fail [DET-6811] (4051)


Docker images

- `docker pull determinedai/determined-master:0.18.1`
- `docker pull determinedai/determined-master:9284a3aa6`
- `docker pull determinedai/determined-master:9284a3aa6e307c61426c93b5e09730c664725604`
- `docker pull determinedai/determined-dev:determined-master-9284a3aa6`
- `docker pull determinedai/determined-dev:determined-master-9284a3aa6e307c61426c93b5e09730c664725604`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:0.18.1`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:9284a3aa6`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:9284a3aa6e307c61426c93b5e09730c664725604`

Changelog

* 3c00fc281 chore: bump version: 0.18.0-rc3 -> 0.18.0
* 797ceeca4 docs: add release notes for 0.18.0 (4102)
* 505102457 chore: bump version: 0.18.0-rc2 -> 0.18.0-rc3
* bd87faa20 perf: improve plan for proto_get_trial_plus.sql (4073)
* c2a3ba920 fix: Support rendering rank of 0 (4083)
* d46995f27 fix: consistent total slot calculation for cluster overview [DET-7182] (4080)
* 1b946122f feat: add det.LOG_FORMAT constant (4090)
* aacd23be7 feat: add wrap_rank helper script (4086)
* 8fbf0a78a fix: dont show archived in column picker [DET-7187] (4085)
* a190a1f88 chore: bump version: 0.18.0-rc1 -> 0.18.0-rc2
* 88fc38b7f chore: bump version: 0.18.0-rc0 -> 0.18.0-rc1
* 4d73ceaa6 feat: authenticate task proxies (4071)
* 3e594e47b chore: filter out NaN, +/- Infinity metric values for charts for now. (4076)
* 1038840af fix: wrap torch.distributed launch in pid server/client (4077)
* 7df0f0013 docs: release note for core api (4069)
* 7721553cc fix: add user column back to experiment list (4070)
* db5f434de chore: bump version: 0.17.16-dev0 -> 0.18.0-rc0
* 02114cf37 chore: lock api state for backward compatibility check
* 7be795af1 docs: Core API reference and cookbook docs (4054)
* 5e364d1a6 chore: ancient checkpoints for very old pytorch (4068)
* 51e80d8ea perf: minimize create/destroy of uPlots [DET-6972] [DET-6972] [DET-6796] [DET-6853] [DET-6672] (3935)
* 531af28cc chore: update removed reducer methods (4067)
* 6f63f13ec chore: remove det.pytorch.reset_parameters() (4066)
* 4fb1886b8 feat: generic checkpoints and making Core API public (3859)
* db0f8ef5c fix: correct the return type for readStream (4063)
* 33a1a6c66 chore: remove PBT searcher (4058)
* a35d49a77 feat: support for torch native dtrain (3807)
* 29385bc09 chore: update k8s scheduler to run latest image (4061)
* e7f328962 chore: remove remainder of native api (4055)
* 87008acad chore: deprecate data layer (4056)
* 75a6bf59d chore: remove deprecated experimental custom reducer methods (4060)
* 948518cee chore: remove unnecessary use of username in webui (DET-6922) (4049)
* d2cc8d425 refactor: simplify apiConfig to reduce redundancy (4043)
* 707dd2680 chore: Refactor CheckpointModal to be hook based [DET-7136] (4034)
* ea535b035 fix: make "Show full config" modal larger (4053)
* 625070edb chore: move webui codecov upload to use env var instead of hardcoded token (4045)
* 0c1821eed fix: remove dead shell start code [DET-7131] (4016)
* 59e5b9b9a chore: Recommend git clone --recurse-submodules for submodules (4036)
* ad06cd431 fix: move allocation resources migration to the top.
* a6bf58b65 refactor: rewrite ndjson streamer [DET-7121] (4014)
* 6d018f09e fix: Alter Boolean arg default handling (4038)
* b278ef040 chore: store allocation resources and agent RM containers in DB. (3946)
* 7afc75f49 ci: Add build/coverage badge for webui/react [DET-6767] (4028)
* 628f07c96 move up profile-pics migration to assure it happens
* 600d77d52 fix: put profile pic migrations in correct directory
* 3569a6f80 build: set up shared-web submodule [DET-6961] (4006)
* 0686befc4 test: ensure that agent disabling doesn't count for experiment restarts [DET-5916] (4029)
* b61a83dec refactor: remove `NewAllocationID`. (3959)
* b605da88d fix: add another missing key fix (4033)
* 64e6f7d9e feat: allow position modification in k8s [DET-6967] [DET-6968] (3938)
* 24bbec93f feat: Creating a table for user profile pictures
* b831dbd34 fix: Human-readable option for empty filters in logs [DET-6781, DET-6999] (4017)
* 5d04a258c fix: Prevent archived models from appearing in the Register Checkpoint modal [DET-7132] (4015)
* c1523dab1 fix: Ensure table offset does not exceed pagination total [DET-6829] (4011)
* 84401b26c fix: change timeout in e2e_tests/tests/cluster/test_logging.py/test_trial_logs (4019)
* ad6f3d54a fix: add key for cancel operation (4023)
* 9ba0f0459 chore: remove deprecated type provider for moment-timezone (4018)
* be3125b8d docs: update copyright year (4005)
* ea4ab8854 feat: add product feedback link [DET-5811]
* 616241874 chore: fix documentation comment for /ws/data-layer
* 482ecd61c docs: fix grammar in training-run index (3988)
* 5480903ae docs: fix indent in pytorch-porting-tutorial (3989)
* 9a6d75260 chore: warn about ambiguous enum params (3997)
* 4de96b955 chore: give the latest-master deploy job a name (3900)
* 64b46627c fix: pass task time not from logCtx (3993)
* 59796b027 chore: update node version to active LTS (DET-7046) (3932)
* 07de42689 chore: show appropriate severity level on job launch failures (4002)
* c68b7a429 test: add option to disable `compare_stats`. (4008)
* 915184c77 chore: Removing CODEOWNERS entry for release notes (3978)
* 93b98c269 fix: handle zeros correctly in HpTrialTable metricSorter (4003)
* cd0e53149 chore: bump version: 0.17.15-dev0 -> 0.17.16-dev0
* 7d1493d00 docs: add release notes for 0.17.15 (3986)
* 63eb86c92 feat: add modal to explain why users cannot delete items [DET-6998] (3994)
* f6fbc05b6 fix: guard trial and allocation exit logic correctness (3983)
* 08218c128 fix: do not default to `noverify` for `bindings` sessions in CLI. (3991)
* 4c5dad8a1 chore: add expconf environment.slurm (3966)
* d2610c557 fix: handle infinite metrics in searcher snapshots [DET-7122] (3999)
* 57d1b38f9 chore: handle rank id in log entries (3995)
* f915a6952 fix: handle infinite validation metric values in more cases (3992)
* 8527e60c5 fix: Experiment columns filter is still applied after closing [DET-6837] (3982)
* c7f4610f0 fix: avoid permanent filtered state in model registry [DET-6946] (3984)
* 35ac5ff88 docs: update release note process (3990)
* d565f842e ci: bump profiling test timeout back up (3981)
* a43fcffed chore: handle errors from starting allocations [DET-5862] (3975)
* bd5484561 feat: add det support bundle [DET-5886] (3904)
* 899db1998 feat: add drag and drop functionality to experiment list column table [DET-7044] (3956)
* cf5c94fb6 chore: document usage of /ws/data-layer [DET-6685] (3971)
* 6f6e4a26a feat: Add overall allocation bar to new cluster page [DET-7074] (3955)
* ff78aa985 chore: add error type for non retry-able resource manager errors (3947)
* e9d26af42 chore: apply filtering to task logs (3963)
* 2a981a198 fix: add test warmup command e2e [DET-5803] (3965)
* a77b8c189 docs: fix tutorial link swap (3979)
* 5d65f6443 ci: remove old semantic PR app config. (3980)
* 896725bc7 fix: notebook logs filter by level (3967)
* f3660c2cb chore: remove old notebook logs endpoint (3960)
* 8a99e03c8 fix: task log level parsing (3973)
* 005d14402 fix: forward job.DeleteJob through agentRM (3968)
* 8e21bf0f8 docs: add release notes for PR 3914 (3962)
* 28ab2fa42 chore: Update list of false alarms in Docker image scanning (3933)
* 80a1dc7f6 ci: new semantic pull request check. (3958)
* 189f148f7 docs: update package versions, add ROCm, edit to style guide (3950)
* 0d180fe96 fix: update HPE logo sizes (3953)
* 393de717f chore: add message to cleanup external RM resources on delete, for slurm (3902)


Docker images

- `docker pull determinedai/determined-master:0.18.0`
- `docker pull determinedai/determined-master:3c00fc281`
- `docker pull determinedai/determined-master:3c00fc281542c272c1591c7d1c86eb53db8f230c`
- `docker pull determinedai/determined-dev:determined-master-3c00fc281`
- `docker pull determinedai/determined-dev:determined-master-3c00fc281542c272c1591c7d1c86eb53db8f230c`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:0.18.0`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:3c00fc281`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:3c00fc281542c272c1591c7d1c86eb53db8f230c`

Changelog

* 9b74e5444 chore: bump version: 0.17.15-rc4 -> 0.17.15
* 931983ac6 chore: bump version: 0.17.15-rc3 -> 0.17.15-rc4
* 3bd2b4afa fix: handle infinite metrics in searcher snapshots [DET-7122] (3999)
* 153221eda chore: bump version: 0.17.15-rc2 -> 0.17.15-rc3
* 1e9f5c4b6 chore: handle rank id in log entries (3995)
* d12c8921e fix: handle infinite validation metric values in more cases (3992)
* 59b180d26 docs: add release notes for 0.17.15 (3986)
* 9e3e5419f chore: bump version: 0.17.15-rc1 -> 0.17.15-rc2
* 1c97bc76a chore: apply filtering to task logs (3963)
* 8dd01c87f fix: task log level parsing (3973)
* 9c241218e docs: add release notes for PR 3914 (3962)
* 507cfe75d fix: update HPE logo sizes (3953)
* e1a3de646 chore: bump version: 0.17.15-rc0 -> 0.17.15-rc1
* dc3ef476f chore: bump version: 0.17.15-dev0 -> 0.17.15-rc0
* 6c38aad74 chore: lock api state for backward compatibility check
* f206919e8 fix: take out job summary caching [DET-6695] (3849)
* c9941ac85 chore: add missing icons to mobile navbar [DET-7009]
* 927ef9417 fix: parse different time format in compare stats script [DET-7039] (3909)
* 55fcf65f4 fix: checkpoint gc job should close (3943)
* b8f1073e1 feat: track task stats [DET-6872, DET-6926, DET-6927] (3852)
* b1a470d7c chore: add key attribute to avatar ActionCard action (3939)
* 7c7375c26 chore: give det job a default command to run (3934)
* 9028fe001 chore: mask registry auth password in harness [DET-6279] (3867)
* 7d0234b16 chore: add filtering by userIds to API endpoints (DET-7019) (3898)
* 180a79f47 chore: mask registry creds in webui [DET-7013] (3881)
* 2d4456439 replace username with userId for user API (3914)
* b6b2c7172 test: add interaction tests for spinner [DET-6665] (3826)
* d86d09ba5 chore: prevent InteractiveTable scroll from moving pagination or other controls [DET-7037] (3923)
* c9a24582f fix: crash on upgrade to InteractiveTable [DET-7036] (3922)
* 4c0ef956c hide archived experiments unless using --all (3918)
* 586bba4a2 docs: tweak docs for socket activation (3926)
* 64ae7ca2a perf: add infiniband-related libraries to environment (3832)
* bcc954a0d chore: disallow getting metadata from dummy checkpoint context (3920)
* 575cc21f7 chore: update dep requirements for react (3901)
* e359843b3 chore: demote _get_last_validation to internal (3919)
* 5c93cb68a chore: log checkpoint uuids in core, not in wlsq (3924)
* 7086df4b7 chore: add missing docstrings in core api (3911)
* ea0db53a9 feat: add slurm rendezvous (3777)
* d5e793ba1 chore: make store_path auto-create the directory (3916)
* 1f3c4b232 feat: add core.DownloadMode (3910)
* c334a5385 docs: fix install cli typo (3917)
* 069eb6f27 fix: run scheduling on agent connection/enable events, reconnectBacklog replay. (3906)
* b75afe6d9 chore: bump version: 0.17.15-dev0-dev0 -> 0.17.15-dev0
* d06dc9429 refactor: remove dependency of settings in the updateSettings call (3894)
* 1b4c8e951 chore: remove pr preview cluster address [DET-7040] (3907)
* e5723c7ea docs: Release notes for 0.17.14 (3912)
* 43b9a7c06 chore: bump version: 0.17.14 -> 0.17.15-dev0
* 5c4554437 fix: RM crashes when setting cmd priority (3908)
* 0c5236786 chore: fix deepspeed nightly tests (3897)
* 3e6267f0d chore: update StorageManager and extend CheckpointContext (3829)
* f76cc4599 docs: fix description of scheduling_unit behavior (3890)
* 1815ee3cb chore: sweeping rename of Core API components (3896)
* 0b7141b88 feat: deepspeed DCGAN example (3758)
* 015a8e07c feat: use enums instead of chief_only bool in Core API (3888)
* 129d841cb fix: use otel only if enabled (3893)
* 0ab3288d8 hide sizeChanger on RoutePagination (3892)
* ae50d3cec fix: match reported rp name for k8 across endpoints [DET-7006] (3870)
* c49561c6f docs: various fixes for master configuration and k8s docs (3889)
* c716759b1 chore: bump version: 0.17.13-dev0 -> 0.17.14-dev0
* 2fdb2efc7 docs: add release notes for 0.17.13 (3879)
* a34962262 chore: Clean up tests with fewer Optional types and asserts (3872)
* b0e0c96ea feat: make core.Searcher multiworker-safe (3871)
* 50c9f669d feat: show agent version in `/agents` and `det agent list` [DET-6847] (3873)
* 954cf9714 fix: avoid double-timestamps in logs (3876)
* b00124a41 feat: Drag to Reorder and Resize Experiment List columns [DET-6438] [DET-6809] (3765)
* be5848588 chore: use better NCCL SOCKET setting for gpt-neox (3874)
* 65e1a97d3 chore: remove internal flag from det.launch.horovod --help (3875)
* 9b1fa3bd0 feat: Search models by name and description substring [DET-6939] (3869)
* 78ba4a33a feat: add opentel to determined master [DET-6775] (3851)
* 789f16dce chore: cleanup stray changes (3868)


Docker images

- `docker pull determinedai/determined-master:0.17.15`
- `docker pull determinedai/determined-master:9b74e5444`
- `docker pull determinedai/determined-master:9b74e54448d64009ce574e1a68b52149c1d00fe7`
- `docker pull determinedai/determined-dev:determined-master-9b74e5444`
- `docker pull determinedai/determined-dev:determined-master-9b74e54448d64009ce574e1a68b52149c1d00fe7`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:0.17.15`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:9b74e5444`
- `docker pull nvcr.io/isv-ngc-partner/determined/determined-master:9b74e54448d64009ce574e1a68b52149c1d00fe7`