Wandb

Added

- Added `api.runs().histories()` to fetch history metrics for runs that meet specified conditions by thanos-wandb in https://github.com/wandb/wandb/pull/7690
- Display warning when Kubernetes pod fails to schedule by TimH98 in https://github.com/wandb/wandb/pull/7576
- Added `ArtifactCollection.save()` to allow persisting changes by amusipatla-wandb in https://github.com/wandb/wandb/pull/7555
- Added the ability to overwrite history of previous runs at an arbitrary step and continue logging from that step by dannygoldstein in https://github.com/wandb/wandb/pull/7711
- Added new Workspace API for programatically editing W&B Workspaces by andrewtruong in https://github.com/wandb/wandb/pull/7728
- Added `Artifact.unlink()` to allow programmatic unlinking of artifacts by tonyyli-wandb in https://github.com/wandb/wandb/pull/7735
- Added basic TensorBoard support when running with `wandb.require("core")` by timoffex in https://github.com/wandb/wandb/pull/7725
- The TensorBoard tab in W&B will work.
- Charts show up in W&B, possibly better than when running without core.
- Not all types of data are supported yet. Unsupported data is not shown in charts.

Fixed

- Fix `define_metric` behavior for multiple metrics in `shared` mode by dmitryduev in https://github.com/wandb/wandb/pull/7715
- Correctly pass in project name to internal api from run while calling run.use_artifact() by ibindlish in https://github.com/wandb/wandb/pull/7701
- Correctly upload console output log files when resuming runs enabled with `console_multipart` setting by kptkin in https://github.com/wandb/wandb/pull/7694 and dmitryduev in https://github.com/wandb/wandb/pull/7697
- Interpret non-octal strings with leading zeros as strings and not integers in sweep configs by KyleGoyette https://github.com/wandb/wandb/pull/7649
- Support Azure repo URI format in Launch KyleGoyette https://github.com/wandb/wandb/pull/7664
- Fix path parsing for artifacts with aliases containing forward slashes by amusipatla-wandb in https://github.com/wandb/wandb/pull/7676
- Add missing docstrings for any public methods in `Api` class by tonyyli-wandb in https://github.com/wandb/wandb/pull/7713
- Correctly add latest alias to jobs built by the job builder KyleGoyette https://github.com/wandb/wandb/pull/7727

Changed

- Option to change naming scheme for console output logs from `output.log` to `logs/YYYYMMDD_HHmmss.ms_output.log` by kptkin in https://github.com/wandb/wandb/pull/7694
- Require `unsafe=True` in `use_model` calls that could potentially load and deserialize unsafe pickle files by anandwandb https://github.com/wandb/wandb/pull/7663
- Update order in api.runs() to ascending to prevent duplicate responses by thanos-wandb https://github.com/wandb/wandb/pull/7675
- Eliminate signed URL timeout errors during artifact file uploads in core by moredatarequired in https://github.com/wandb/wandb/pull/7586

Deprecated

- Deprecated `ArtifactCollection.change_type()` in favor of `ArtifactCollection.save()` by amusipatla-wandb in https://github.com/wandb/wandb/pull/7555

Added

- The `wandb` package now includes the `wandb-core` binary by timoffex in https://github.com/wandb/wandb/pull/7381
- `wandb-core` is a new and improved backend for the W&B SDK that focuses on performance, versatility, and robustness.
- Currently, it is opt-in. To start using the new backend, add `wandb.require("core")` to your script after importing `wandb`.
- `wandb-core` now supports Artifact file caching by moredatarequired in https://github.com/wandb/wandb/pull/7364 and https://github.com/wandb/wandb/pull/7366
- Added artifact_exists() and artifact_collection_exists() methods to Api to check if an artifact or collection exists by amusipatla-wandb in https://github.com/wandb/wandb/pull/7483
- `wandb launch -u <git-uri | local-path> ` creates and launches a job from the given source code by bcsherma in https://github.com/wandb/wandb/pull/7485

Fixed

- Prevent crash on `run.summary` for finished runs by dmitryduev in https://github.com/wandb/wandb/pull/7440
- Correctly report file upload errors when using wandb-core by moredatarequired in https://github.com/wandb/wandb/pull/7196
- Implemented a stricter check for AMD GPU availability by dmitryduev in https://github.com/wandb/wandb/pull/7322
- Fixed `run.save()` on Windows by timoffex in https://github.com/wandb/wandb/pull/7412
- Show a warning instead of failing when using registries other than ECR and GAR with the Kaniko builder by TimH98 in https://github.com/wandb/wandb/pull/7461
- Fixed `wandb.init()` type signature including `None` by timoffex in https://github.com/wandb/wandb/pull/7563

Changed

- When using `wandb-core` need to specify a required flag (`wandb.require("core")`) to enable it, before it was picked up automatically by kptkin in https://github.com/wandb/wandb/pull/7228
- Use ETags instead of MD5 hashes for GCS reference artifacts by moredatarequired in https://github.com/wandb/wandb/pull/7337

Removed

- Removed the deprecated `wandb.plots.*` functions and top-level third-party integrations `wandb.[catboost,fastai,keras,lightgbm,sacred,xgboost]`. Please use `wandb.plot` instead of `wandb.plots` and `wandb.integration.[catboost,fastai,keras,lightgbm,sacred,xgboost]` instead of `wandb.[catboost,fastai,keras,lightgbm,sacred,xgboost]`. By dmitryduev in https://github.com/wandb/wandb/pull/7552
- Removed the `[async]` extra and the `_async_upload_concurrency_limit` setting by moredatarequired in https://github.com/wandb/wandb/pull/7416
- Removed undocumented settings: `_except_exit` and `problem` by timoffex in https://github.com/wandb/wandb/pull/7563

Added

- Added support for overriding kaniko builder settings in the agent config by TimH98 in https://github.com/wandb/wandb/pull/7191
- Added link to the project workspace of a run in the footer by kptkin in https://github.com/wandb/wandb/pull/7276
- Added support for overriding stopped run grace period in the agent config by TimH98 in https://github.com/wandb/wandb/pull/7281
- Added setting (`_disable_update_check`) to disable version checks during init by kptkin in https://github.com/wandb/wandb/pull/7287
- `WandbLogger.sync` in the OpenAI Fine-Tuning integration gets a new `log_datasets` boolean argument to turn off automatic logging of datasets to Artifacts by morganmcg1 in https://github.com/wandb/wandb/pull/7150
- Reduced default status print frequency of launch agent. Added verbosity controls to allow for increased status print frequency and printing debug information to stdout by TimH98 in https://github.com/wandb/wandb/pull/7126

Changed

- Limit policy option on artifact cli's put() to choices, ["mutable", "immutable"] by ibindish in https://github.com/wandb/wandb/pull/7172
- Updated artifact public api methods to handle nullable Project field on the ArtifactSequence/ArtifactCollection type, based on gorilla server changes by ibindlish in https://github.com/wandb/wandb/pull/7201

Fixed

- Fixed `run.save()` not working with files inside `run.dir`, introduced in previous release
- Fixed rare panic during large artifact uploads by moredatarequire in https://github.com/wandb/wandb/pull/7272
- Fixed wandb.login causing runs not to be associated with launch queue by KyleGoyette in https://github.com/wandb/wandb/pull/7280
- Fixed job artifact download failing silently and causing run crash when using W&B Launch by KyleGoyette https://github.com/wandb/wandb/pull/7285
- Fix handling of saving training files to Artifacts in the OpenAI Fine-Tuning integration by morganmcg1 in https://github.com/wandb/wandb/pull/7150

Added

- Added feature to move staging files to cache (instead of copying) for mutable artifact file uploads when caching is enabled by ibindlish in https://github.com/wandb/wandb/pull/7143
- Added support to skip caching files to the local filesystem while uploading files to artifacts by ibindlish in https://github.com/wandb/wandb/pull/7098
- Added support to skip staging artifact files during upload by selecting a storage policy by ibindlish in https://github.com/wandb/wandb/pull/7142
- Preliminary support for forking a run using `wandb.init(fork_from=...)` by dannygoldstein in https://github.com/wandb/wandb/pull/7078
- `run.save()` accepts `pathlib.Path` values; by timoffex in https://github.com/wandb/wandb/pull/7146

Changed

- When printing the run link point to the workspace explicitly by kptkin in https://github.com/wandb/wandb/pull/7132

Fixed

- In case of transient server issues when creating the wandb API key kubernetes secret, we'll retry up to 5 times by TimH98 in https://github.com/wandb/wandb/pull/7108

Removed

- When printing run's information in the terminal remove links to jobs by kptkin in https://github.com/wandb/wandb/pull/7132

Added

- Added ability to change artifact collection types by biaslucas in https://github.com/wandb/wandb/pull/6971
- Add support for installing deps from pyproject.toml by bcsherma in https://github.com/wandb/wandb/pull/6964
- Support kaniko build with user-provided pvc and docker config by bcsherma in https://github.com/wandb/wandb/pull/7059
- Added ability to import runs between W&B instances by andrewtruong in https://github.com/wandb/wandb/pull/6897

Changed

- wandb-core rate-limits requests to the backend and respects RateLimit-\* headers
by timoffex in https://github.com/wandb/wandb/pull/7065

Fixed

- Fix passing of template variables in the sweeps-on-launch scheduler by dannygoldstein in https://github.com/wandb/wandb/pull/6959
- Link job artifact to a run to be specified as input by kptkin in https://github.com/wandb/wandb/pull/6940
- Fix sagemaker entrypoint to use given entrypoint by KyleGoyette in https://github.com/wandb/wandb/pull/6969
- Parse upload headers correctly by kptkin in https://github.com/wandb/wandb/pull/6983
- Properly propagate server errors by kptkin in https://github.com/wandb/wandb/pull/6944
- Make file upload faster by using parallelism by kptkin in https://github.com/wandb/wandb/pull/6975
- Don't send git data if it's not populated by kptkin in https://github.com/wandb/wandb/pull/6984
- Fix console logging resumption, avoid overwrite by kptkin in https://github.com/wandb/wandb/pull/6963
- Remove hostname validation when using --host on wandb login by Jamil in https://github.com/wandb/wandb/pull/6999
- Don't discard past visualizations when resuming a run by timoffex in https://github.com/wandb/wandb/pull/7005
- Avoid retrying on conflict status code by kptkin in https://github.com/wandb/wandb/pull/7011
- Fix visualization config merging for resumed runs in wandb-core by timoffex in https://github.com/wandb/wandb/pull/7012
- Replace usage of standard library's json with `segmentio`'s by kptkin in https://github.com/wandb/wandb/pull/7027
- Remove stderr as writer for the logs by kptkin in https://github.com/wandb/wandb/pull/7022
- Disable negative steps from initialization by kptkin in https://github.com/wandb/wandb/pull/7030
- Fix report loading in pydantic26 by andrewtruong in https://github.com/wandb/wandb/pull/6988
- Revert "make upload request async to support progress reporting (6497)" by jlzhao27 in https://github.com/wandb/wandb/pull/7049
- Fix entrypoint specification when using a Dockerfile.wandb by KyleGoyette in https://github.com/wandb/wandb/pull/7080
- Fix stream releasing probe handle too early by jlzhao27 in https://github.com/wandb/wandb/pull/7056
- Always attempt to pull latest image for local container by KyleGoyette in https://github.com/wandb/wandb/pull/7079

New Contributors

- Jamil made their first contribution in https://github.com/wandb/wandb/pull/6999

:magic_wand: Enhancements

- feat(core): generate data type info in core by dmitryduev in https://github.com/wandb/wandb/pull/6827
- feat(core): add support for Launch 🚀 by kptkin in https://github.com/wandb/wandb/pull/6822
- feat(public-api): Added option to control number of grouped sampled runs in reports by thanos-wandb in https://github.com/wandb/wandb/pull/6840
- feat(sdk): add shared mode to enable multiple independent writers to the same run by dmitryduev in https://github.com/wandb/wandb/pull/6882
- perf(artifacts): Reduce artifact download latency via optional cache copy + threads by biaslucas in https://github.com/wandb/wandb/pull/6878
- feat(artifacts): Add partial file downloads, via directory prefix by biaslucas in https://github.com/wandb/wandb/pull/6911
- feat(integrations): Update the Diffusers Integration by soumik12345 in https://github.com/wandb/wandb/pull/6804
- feat(integrations): Update Ultralytics Integration by soumik12345 in https://github.com/wandb/wandb/pull/6796
- feat(integrations): Add Pytorch Lightning Fabric Logger by ash0ts in https://github.com/wandb/wandb/pull/6919
- feat(core): update go packages by kptkin in https://github.com/wandb/wandb/pull/6908

:hammer: Fixes

- fix(launch): Remove project and runner fields from agent config by KyleGoyette in https://github.com/wandb/wandb/pull/6818
- fix(launch): recognize deleted k8s jobs as failed by bcsherma in https://github.com/wandb/wandb/pull/6824
- fix(launch): warn of extra fields in environment block instead of erroring by bcsherma in https://github.com/wandb/wandb/pull/6833
- fix(sdk): entity override bug where ENVVAR is prioritized over kwargs by biaslucas in https://github.com/wandb/wandb/pull/6843
- fix(launch): Local container runner doesn't ignore override args by TimH98 in https://github.com/wandb/wandb/pull/6844
- fix(sdk): merge-update config with sweep/launch config by dannygoldstein in https://github.com/wandb/wandb/pull/6841
- fix(sdk): fix retry logic in wandb-core and system_tests conftest by dmitryduev in https://github.com/wandb/wandb/pull/6847
- fix(core): use RW locks in system monitor's assets management by dmitryduev in https://github.com/wandb/wandb/pull/6852
- fix(launch): set build context to entrypoint dir if it contains Dockerfile.wandb by bcsherma in https://github.com/wandb/wandb/pull/6855
- security(launch): Mount wandb api key in launch job pods from a k8s secret by TimH98 in https://github.com/wandb/wandb/pull/6722
- fix(launch): wandb job create should not look for requirements.txt if Dockerfile.wandb is next to entrypoint by bcsherma in https://github.com/wandb/wandb/pull/6861
- fix(sdk): fix \_parse_path when only id is passed to wandb.Api().run() by luisbergua in https://github.com/wandb/wandb/pull/6858
- fix(media): Update video.py: Fix fps bug by stellargo in https://github.com/wandb/wandb/pull/6887
- fix(sdk): clean up temp folders by dmitryduev in https://github.com/wandb/wandb/pull/6891
- fix(artifacts): fix long artifact paths on Windows by ArtsiomWB in https://github.com/wandb/wandb/pull/6846
- fix(sdk): Update Report API to work with pydantic2.6 by andrewtruong in https://github.com/wandb/wandb/pull/6925
- fix(launch): fetch all commits to enable checking out by sha by bcsherma in https://github.com/wandb/wandb/pull/6926
- fix(sweeps): dont swallow exceptions in pyagent by dannygoldstein in https://github.com/wandb/wandb/pull/6927
- fix(artifacts): artifact file upload progress in nexus by ibindlish in https://github.com/wandb/wandb/pull/6939
- fix(sdk): exercise caution in system monitor when rocm-smi is installed on a system with no amd gpus by dmitryduev in https://github.com/wandb/wandb/pull/6938
- fix(cli): typo in cli.py by eltociear in https://github.com/wandb/wandb/pull/6892
- fix(launch): remove deadsnakes from accelerator build step by bcsherma in https://github.com/wandb/wandb/pull/6933

:books: Docs

- docs(sdk): update sweep `docstrings` by ngrayluna in https://github.com/wandb/wandb/pull/6830
- docs(sdk): Updates the Tables reference docs. by katjacksonWB in https://github.com/wandb/wandb/pull/6880
- docs(sdk): Artifact docstrings PR by ngrayluna in https://github.com/wandb/wandb/pull/6825

New Contributors

- biaslucas made their first contribution in https://github.com/wandb/wandb/pull/6843
- stellargo made their first contribution in https://github.com/wandb/wandb/pull/6887
- timoffex made their first contribution in https://github.com/wandb/wandb/pull/6916

**Full Changelog**: https://github.com/wandb/wandb/compare/v0.16.2...v0.16.3