Download files to a specific folder
It is now possible to download files from the Hub and move them to a specific folder!
Two behaviors are possible: either create symlinks or move the files from the cache. This can be controlled with the `local_dir_use_symlinks` input parameter. The default -and recommended- value is `"auto"` which will duplicate small files to ease user experience (no symlinks when editing a file) and create symlinks for big files (save disk usage).
py
from huggingface_hub import snapshot_download
or "from huggingface_hub import hf_hub_download"
Download and cache files + duplicate small files (<5MB) to "my-folder" + add symlinks for big files
snapshot_download(repo_id, local_dir="my-folder")
Download and cache files + add symlinks in "my-folder"
snapshot_download(repo_id, local_dir="my-folder", local_dir_use_symlinks=True)
Duplicate files already existing in cache and/or download missing files directly to "my-folder"
snapshot_download(repo_id, local_dir="my-folder", local_dir_use_symlinks=False)
* Download file to specific destination by Wauplin in 1360
Documentation
Efforts to improve documentation have continued. The [guides overview](https://huggingface.co/docs/huggingface_hub/main/en/guides/overview) has been refactored to display which topics are covered (repository, upload, download, search, inference, community tab, cache, model cards, space management and integration).
Upload / Download files
The [repository](https://huggingface.co/docs/huggingface_hub/main/en/guides/repository), [upload](https://huggingface.co/docs/huggingface_hub/main/en/guides/upload) and [download](https://huggingface.co/docs/huggingface_hub/main/en/guides/download) guides have been revisited to showcase the different possibilities to manage a repository and upload/download files to/from it. The focus has been explicitly put on the HTTP endpoints rather than the git cli.
* Refactor guides section + promote HTTP over GIT by Wauplin in 1338
Integrate a library
A new guide has been added on [how to integrate any ML framework](https://huggingface.co/docs/huggingface_hub/main/en/guides/integrations) with the Hub. It explains what is meant by that and how to do it. Here is the summary table to remember:
* Write Mixins/Integration guide by Wauplin in 1362
Other
* Add repo_type to repo_info docstring by albertvillanova in 1347
New endpoints + QOL improvements
Duplicate a Space
It's now possible to [duplicate a Space](https://huggingface.co/docs/huggingface_hub/main/en/package_reference/hf_api#huggingface_hub.HfApi.duplicate_space) programmatically!
py
>>> from huggingface_hub import duplicate_space
Duplicate a Space to your account
>>> duplicate_space("multimodalart/dreambooth-training")
RepoUrl('https://huggingface.co/spaces/nateraw/dreambooth-training',...)
* :sparkles: add duplicate_space by nateraw in 1353
`delete_patterns` in `upload_folder`
New input parameter `delete_patterns` for the `upload_folder` method. It allows to delete some remote files before pushing a folder to the Hub, in a single commit. Useful when you don't exactly know which files have already been pushed. Here is an example to upload log files while deleting existing logs on the Hub:
py
api.upload_folder(
folder_path="/path/to/local/folder/logs",
repo_id="username/trained-model",
path_in_repo="experiment/logs/",
allow_patterns="*.txt", Upload all local text files
delete_patterns="*.txt", Delete all remote text files before
)
* Add delete_patterns option to upload_folder by Wauplin in 1370
List repo history
Get the repo history (i.e. all the commits) for a given revision.
py
Get initial commit on a repo
>>> from huggingface_hub import list_repo_commits
>>> initial_commit = list_repo_commits("gpt2")[-1]
Initial commit is always a system commit containing the `.gitattributes` file.
>>> initial_commit
GitCommitInfo(
commit_id='9b865efde13a30c13e0a33e536cf3e4a5a9d71d8',
authors=['system'],
created_at=datetime.datetime(2019, 2, 18, 10, 36, 15, tzinfo=datetime.timezone.utc),
title='initial commit',
message='',
formatted_title=None,
formatted_message=None
)
* Add list_repo_commits to list git history of a repo by Wauplin in 1331
Accept token in `huggingface-cli login`
`--token` and `--add-to-git-credential` option have been added to login directly from the CLI using an environment variable. Useful to login in a Github CI script for example.
huggingface-cli login --token $HUGGINGFACE_TOKEN --add-to-git-credential
* Add token and git credentials to login cli command by silvanocerza in 1372
* token in CLI login docs by Wauplin (direct commit on main)
Telemetry helper
Helper for external libraries to track usage of specific features of their package. Telemetry can be globally disabled by the user using `HF_HUB_DISABLE_TELEMETRY`.
py
from huggingface_hub.utils import send_telemetry
send_telemetry("gradio/local_link", library_name="gradio", library_version="3.22.1")
* Add utility to send telemetry by Wauplin in 1344
Breaking change
When loading a model card with an invalid `model_index` in the metadata, an error is explicitly raised. Previous behavior was to trigger a warning and ignore the model_index. This was problematic as it could lead to a loss of information. Fixing this is a breaking change but impact should be limited as the server is already rejecting invalid model cards. An optional `ignore_metadata_errors` argument (default to False) can be used to load the card with only a warning.
* Explicit raise on invalid model_index + add ignore_metadata_errors option by Wauplin in 1377
Bugfixes & small improvements
Model cards, datasets cards and space cards
A few improvements in repo cards: expose `RepoCard` as top-level, dict-like methods for `RepoCardData` object (1354), updated template and improved type annotation for metadata.
* Updating MC headings by EziOzoani in 1367
* Switch datasets type in ModelCard to a list of datasets by davanstrien in 1356
* Expose `RepoCard` at top level + few qol improvements by Wauplin in 1354
* Explicit raise on invalid model_index + add ignore_metadata_errors option by Wauplin in 1377
Misc
* Fix contrib CI for timm by Wauplin in 1346
* :ambulance: hotfix isHf test user permissions bug by nateraw in 1357
* Remove unused private methods by Wauplin in 1359
* Updates types for RepoCards fields by davanstrien in 1358
* some typos by Wauplin (direct commit on main)
* [WIP] make repr formatting narrower by davanstrien in 903
* Add ReprMixin to Repofile by Wauplin (direct commit on main)
* Fix ReprMixin for python3.7 + test by Wauplin in 1380
* fix create_commit on lowercased repo_id + add regression test by Wauplin in 1376
* FIX create_repo with exists_ok but no permission by Wauplin in 1364