The official Python client for the Huggingface Hub.
APACHE-2.0 License
Published by Wauplin over 1 year ago
It is now possible to download files from the Hub and move them to a specific folder!
Two behaviors are possible: either create symlinks or move the files from the cache. This can be controlled with the local_dir_use_symlinks
input parameter. The default -and recommended- value is "auto"
which will duplicate small files to ease user experience (no symlinks when editing a file) and create symlinks for big files (save disk usage).
from huggingface_hub import snapshot_download
# or "from huggingface_hub import hf_hub_download"
# Download and cache files + duplicate small files (<5MB) to "my-folder" + add symlinks for big files
snapshot_download(repo_id, local_dir="my-folder")
# Download and cache files + add symlinks in "my-folder"
snapshot_download(repo_id, local_dir="my-folder", local_dir_use_symlinks=True)
# Duplicate files already existing in cache and/or download missing files directly to "my-folder"
snapshot_download(repo_id, local_dir="my-folder", local_dir_use_symlinks=False)
Efforts to improve documentation have continued. The guides overview has been refactored to display which topics are covered (repository, upload, download, search, inference, community tab, cache, model cards, space management and integration).
The repository, upload and download guides have been revisited to showcase the different possibilities to manage a repository and upload/download files to/from it. The focus has been explicitly put on the HTTP endpoints rather than the git cli.
A new guide has been added on how to integrate any ML framework with the Hub. It explains what is meant by that and how to do it. Here is the summary table to remember:
It's now possible to duplicate a Space programmatically!
>>> from huggingface_hub import duplicate_space
# Duplicate a Space to your account
>>> duplicate_space("multimodalart/dreambooth-training")
RepoUrl('https://huggingface.co/spaces/nateraw/dreambooth-training',...)
delete_patterns
in upload_folder
New input parameter delete_patterns
for the upload_folder
method. It allows to delete some remote files before pushing a folder to the Hub, in a single commit. Useful when you don't exactly know which files have already been pushed. Here is an example to upload log files while deleting existing logs on the Hub:
api.upload_folder(
folder_path="/path/to/local/folder/logs",
repo_id="username/trained-model",
path_in_repo="experiment/logs/",
allow_patterns="*.txt", # Upload all local text files
delete_patterns="*.txt", # Delete all remote text files before
)
Get the repo history (i.e. all the commits) for a given revision.
# Get initial commit on a repo
>>> from huggingface_hub import list_repo_commits
>>> initial_commit = list_repo_commits("gpt2")[-1]
# Initial commit is always a system commit containing the `.gitattributes` file.
>>> initial_commit
GitCommitInfo(
commit_id='9b865efde13a30c13e0a33e536cf3e4a5a9d71d8',
authors=['system'],
created_at=datetime.datetime(2019, 2, 18, 10, 36, 15, tzinfo=datetime.timezone.utc),
title='initial commit',
message='',
formatted_title=None,
formatted_message=None
)
huggingface-cli login
--token
and --add-to-git-credential
option have been added to login directly from the CLI using an environment variable. Useful to login in a Github CI script for example.
huggingface-cli login --token $HUGGINGFACE_TOKEN --add-to-git-credential
Helper for external libraries to track usage of specific features of their package. Telemetry can be globally disabled by the user using HF_HUB_DISABLE_TELEMETRY
.
from huggingface_hub.utils import send_telemetry
send_telemetry("gradio/local_link", library_name="gradio", library_version="3.22.1")
When loading a model card with an invalid model_index
in the metadata, an error is explicitly raised. Previous behavior was to trigger a warning and ignore the model_index. This was problematic as it could lead to a loss of information. Fixing this is a breaking change but impact should be limited as the server is already rejecting invalid model cards. An optional ignore_metadata_errors
argument (default to False) can be used to load the card with only a warning.
A few improvements in repo cards: expose RepoCard
as top-level, dict-like methods for RepoCardData
object (#1354), updated template and improved type annotation for metadata.
RepoCard
at top level + few qol improvements by @Wauplin in #1354Published by Wauplin over 1 year ago
Hot-fix to remove authorization header when following redirection (using cached_download
). Fix was already implemented for hf_hub_download
but we forgot about this one. Has only a consequence when downloading LFS files from Spaces. Problem arose since a server-side change on how files are served. See https://github.com/huggingface/huggingface_hub/pull/1345.
Full Changelog: https://github.com/huggingface/huggingface_hub/compare/v0.12.0...v0.12.1
Published by Wauplin almost 2 years ago
Spaces support has been substantially enhanced. You can now:
# Assign hardware when creating the Space
api.create_repo(repo_id=repo_id, repo_type="space", space_sdk="gradio", space_hardware="cpu-upgrade")
# Configure some secrets
api.add_space_secret(repo_id=repo_id, key="HF_TOKEN", value="hf_api_***")
# Request hardware on the fly
api.request_space_hardware(repo_id=repo_id, hardware="t4-medium")
# Get Space runtime (state, hardware, sdk,...)
api.get_space_runtime(repo_id=repo_id)
Visit the docs for more details.
get_space_secrets
endpoint by @Wauplin in #1264And in bonus: Spaces now support Dockerfile natively !
api.create_repo(repo_id=repo_id, repo_type="space", space_sdk="docker")
Check out the docs for more details.
HfApi
It's now possible to list branches/tags from a repo, getting exact ref
and target_commit
.
More details in the docs.
>>> api.list_repo_refs("bigcode/the-stack", repo_type='dataset')
GitRefs(
branches=[
GitRefInfo(name='main', ref='refs/heads/main', target_commit='18edc1591d9ce72aa82f56c4431b3c969b210ae3'),
GitRefInfo(name='v1.1.a1', ref='refs/heads/v1.1.a1', target_commit='f9826b862d1567f3822d3d25649b0d6d22ace714')],
converts=[],
tags=[
GitRefInfo(name='v1.0', ref='refs/tags/v1.0', target_commit='c37a8cd1e382064d8aced5e05543c5f7753834da')
]
)
New endpoints to like a repo, unlike it and list_liked_repos.
repo_id
when creating a repoWhen using create_repo, one can provide a simple repo_name
without specifying a namespace (example: "my-cool-model"
instead of "Wauplin/my-cool-model"
). This was annoying for as one could easily know if a namespace has been added to the provided repo_id. To facilitate this, the return value of create_repo
is now an instance of RepoUrl which contains information like the endpoint, namespace, repo_id and repo_type.
By default, new branches start from main HEAD
. It's now possible to specify any branch, tag or commit id to start from.
Modelcards module is getting some adjustments to better integrate with the Hub. The scope of this work is larger than "just" huggingface_hub
and resulted in the launch of the HF Model Card Guidebook to help operationalize model cards in the ML community.
datasetcard_template
: I think linking to a GH user does not make sense anymore now that dataset repos are fully on the Hub by @julien-c in #1257REGEX_YAML_BLOCK
by @julien-c in #1285Quite some effort has been put into the documentation in the past few weeks:
huggingface_hub
is getting more and more mature but you might still have some friction if you are maintainer of a library depending on huggingface_hub
. To help detect breaking changes that would affect third-party libraries, we built a framework to run simple end-to-end tests in our CI. This is still preliminary work but the hope is make hfh
ecosystem more and more robust over time. Check out our README for more details.
Goal is to download files faster. First step has been to increase the chunk size by which the files are uploaded. Second step has been to add an optional Rust extension. This is not officially documented for now as we are internally assessing its benefits and limits. Installing and activating hf_transfer
is purely optional.
Repository
"clone_from" feature do not create the remote repository if it doesn't exist on the Hub. Please use create_repo
first before cloning it locally. The private
init argument has also been removed as it was not needed anymore.allow_regex
and ignore_regex
have been removed from snapshot_download
in favor allow_patterns
and ignore_patterns
.push_to_hub_fastai
has been removed in favor of the HTTP-based approach. Same for ModelHubMixin
, PyTorchModelHubMixin
, push_to_hub_keras
and KerasModelHubMixin
.create_repo
is now forced to use keyword-arguments. Same for metadata_eval_result
.Not really some features, not really some fixes. But definitely a quality of life improvement for users 🙂
hf://
urls + raise ValueError
if repo type is unknown by @Wauplin in #1298Published by Wauplin almost 2 years ago
Hot-fix to fix permission issues when downloading with hf_hub_download
or snapshot_download
. For more details, see https://github.com/huggingface/huggingface_hub/pull/1220, https://github.com/huggingface/huggingface_hub/issues/1141 and https://github.com/huggingface/huggingface_hub/issues/1215.
Full changelog: https://github.com/huggingface/huggingface_hub/compare/v0.11.0...v0.11.1
Published by Wauplin almost 2 years ago
HfApi
HfApi
is the central point to interact with the Hub API (manage repos, create commits,...). The goal is to propose more and more git-related features using HTTP endpoints to allow users to interact with the Hub without cloning locally a repo.
from huggingface_hub import create_branch, create_tag, delete_branch, delete_tag
create_tag(repo_id, tag="v0.11", tag_message="Release v0.11")
delete_tag(repo_id, tag="something") # If you created a tag by mistake
create_branch(repo_id, branch="experiment-154")
delete_branch(repo_id, branch="experiment-1") # Clean some old branches
create_tag
method to create tags from the HTTP endpoint by @Wauplin in #1089delete_tag
method to HfApi
by @Wauplin in #1128Making a very large commit was previously tedious. Files are now processed by chunks which makes it possible to upload 25k files in a single commit (and 1Gb payload limitation if uploading only non-LFS files). This should make it easier to upload large datasets.
from huggingface_hub import CommitOperationDelete, create_commit, delete_folder
# Delete a single folder
delete_folder(repo_id=repo_id, path_in_repo="logs/")
# Alternatively, use the low-level `create_commit`
create_commit(
repo_id,
operations=[
CommitOperationDelete(path_in_repo="old_config.json") # Delete a file
CommitOperationDelete(path_in_repo="logs/") # Delete a folder
],
commit_message=...,
)
In the future, listing models, datasets and spaces will be paginated on the Hub by default. To avoid breaking changes, huggingface_hub
follows already pagination. Output type is currently a list (deprecated), will become a generator in v0.14
.
Authentication has been revisited to make it as easy as possible for the users.
login
and logout
methodsfrom huggingface_hub import login, logout
# `login` detects automatically if you are running in a notebook or a script
# Launch widgets or TUI accordingly
login()
# Now possible to login with a hardcoded token (non-blocking)
login(token="hf_***")
# If you want to bypass the auto-detection of `login`
notebook_login() # still available
interpreter_login() # to login from a script
# Logout programmatically
logout()
# Still possible to login from CLI
huggingface-cli login
HfApi
sessionfrom huggingface_hub import HfApi
# Token will be sent in every request but not stored on machine
api = HfApi(token="hf_***")
use_auth_token
in favor of token
, everywheretoken
parameter can now be passed to every method in huggingface_hub
. use_auth_token
is still accepted where it previously existed but the mid-term goal (~6 months) is to deprecate and remove it.
use_auth_token
arg by token
everywhere by @Wauplin in #1122Previously, token was stored in the git credential store
. Can now be in any helper configured by the user -keychain, cache,...-.
# Dump all relevant information. To be used when reporting an issue.
➜ huggingface-cli env
Copy-and-paste the text below in your GitHub issue.
- huggingface_hub version: 0.11.0.dev0
- Platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.35
- Python version: 3.10.6
...
x-error-message
header if exists by @Wauplin in #1121Few improvements/fixes in the modelcard module:
model_name
in metadata_update
by @lvwerra in #1157New feature to provide a path in the cache where any downstream library can store assets (processed data, files from the web, extracted data, rendered images,...)
create_repo
identical_ok
removed in upload_file
validate_preupload_info
, prepare_commit_payload
, _upload_lfs_object
(internal helpers for the commit API)huggingface_hub.snapshot_download
is not exposed as a public module anymoreHfApi.move_repo(...)
and complete tests by @Wauplin in #1136Published by Wauplin about 2 years ago
Hot-fix to force utf-8 encoding in modelcards. See https://github.com/huggingface/huggingface_hub/pull/1102 and https://github.com/skops-dev/skops/pull/162#issuecomment-1263516507 for context.
Full Changelog: https://github.com/huggingface/huggingface_hub/compare/v0.10.0...v0.10.1
Published by Wauplin about 2 years ago
Contribution from @nateraw to integrate the work done on Modelcards and DatasetCards (from nateraw/modelcards) directly in huggingface_hub
.
>>> from huggingface_hub import ModelCard
>>> card = ModelCard.load('nateraw/vit-base-beans')
>>> card.data.to_dict()
{'language': 'en', 'license': 'apache-2.0', 'tags': ['generated_from_trainer', 'image-classification'],...}
modelcards
repo by @nateraw in #940update_metadata
by @Wauplin in #1061huggingface-cli scan-cache
and huggingface-cli delete-cache
)New commands in huggingface-cli
to scan and delete parts of the cache. Goal is to manage the cache-system the same way for any dependent library that uses huggingface_hub
. Only the new cache-system format is supported.
➜ huggingface-cli scan-cache
REPO ID REPO TYPE SIZE ON DISK NB FILES LAST_ACCESSED LAST_MODIFIED REFS LOCAL PATH
--------------------------- --------- ------------ -------- ------------- ------------- ------------------- -------------------------------------------------------------------------
glue dataset 116.3K 15 4 days ago 4 days ago 2.4.0, main, 1.17.0 /home/wauplin/.cache/huggingface/hub/datasets--glue
google/fleurs dataset 64.9M 6 1 week ago 1 week ago refs/pr/1, main /home/wauplin/.cache/
(...)
Done in 0.0s. Scanned 6 repo(s) for a total of 3.4G.
Got 1 warning(s) while scanning. Use -vvv to print details.
huggingface-cli delete-cache
command by @Wauplin in #1046delete-cache
by @Wauplin in #1063HTTP calls to the Hub have been harmonized to behave the same across the library.
Major differences are:
hf_raise_for_status
(more informative error message)hf_hub_download
.huggingface_hub
by @Wauplin in #1019create_commit
has been modified. This is a breaking change, but we hope the return type of this method was never used (quite recent and niche output type).repo_id
is now validated using @validate_hf_hub_args
(see below), a breaking change can be caused if repo_id
was previously miused. A HFValidationError
is now raised if repo_id
is not valid.push_to_hub_fastai
@validate_hf_hub_args
huggingface_hub
by @Wauplin in #1029⚠️ This is a breaking change if repo_id
was previously misused ⚠️
token
in read-only methods of HfApi
in favor of use_auth_token
by @SBrandeis in #928/models/
path for api call to update settings by @Wauplin in #1049store
in google colab by @Wauplin in #1053Published by Wauplin about 2 years ago
Hot-fix error message on gated repositories (https://github.com/huggingface/huggingface_hub/pull/1015).
Context: https://huggingface.co/CompVis/stable-diffusion-v1-4 has been widely shared in the last days but since it's a gated-repo, lots of users are getting confused by the Authentification error received. Error message is now more detailed.
Full Changelog: https://github.com/huggingface/huggingface_hub/compare/v0.9.0...v0.9.1
Published by Wauplin about 2 years ago
Huge work to programmatically interact with the community tab, thanks to @SBrandeis !
It is now possible to:
create_discussion
, create_pull_request
, merge_pull_request
, change_discussion_status
, rename_discussion
)comment_discussion
, edit_discussion_comment
)get_repo_discussions
, get_discussion_details
)See full documentation for more details.
push_to_hub
mixinspush_to_hub
mixin and push_to_hub_keras
have been refactored to leverage the http-endpoint. This means pushing to the hub will no longer require to first download the repo locally. Previous git-based version is planned to be supported until v0.12.
git
by @LysandreJik in #847parent_commit
argument for create_commit
and related functions by @SBrandeis in #916files_metadata
option to repo_info by @SBrandeis in #951upload_folder
HF_HUB_DISABLE_PROGRESS_BARS
env variable or using disable_progress_bars
/enable_progress_bars
helpers.try_to_load_from_cache
to check if a file is locally cachedupload_file
with upload_folder
in upload_folder
docstring by @mariosasko in #927hf_hub_download
for a renamed repo by @Wauplin in #983path_in_repo
optional in upload folder
by @Wauplin in #988repocard_types.py
by @julien-c in #931flake8-bugbear
+ adapt existing codebase by @Wauplin in #967Published by LysandreJik over 2 years ago
v0.8.1 introduces a new way of caching files from the Hugging Face Hub, to two methods: snapshot_download
and hf_hub_download
.
The new approach is extensively documented in the Documenting files guide and we recommend checking it out to get a better understanding of how caching works.
create_commit
APIA new create_commit
API allows users to upload and delete several files at once using HTTP-based methods. You can read more about it in this guide. The following convenience methods were also introduced:
upload_folder
: Allows uploading a local directory to a repo.delete_file
allows deleting a single file from a repo.upload_file
now uses create_commit
under the hood.
create_commit
also allows creating pull requests with a create_pr=True
flag.
None of the methods rely on Git locally.
create_commit
API by @SBrandeis in #888All modules will now be lazy-loaded. This should drastically reduce the time it takes to import huggingface_hub
as it will no longer load all soft dependencies.
hub-ci
for tests by @SBrandeis in #898metadata_update
: work on a copy of the upstream file, to not mess up the cache by @julien-c in #891Published by LysandreJik over 2 years ago
This PR adds a metadata_update function that allows the user to update the metadata in a repository on the hub. The function accepts a dict with metadata (following the same pattern as the YAML in the README) and behaves as follows for all top level fields except model-index.
Examples:
Starting from
existing_results = [{
'dataset': {'name': 'IMDb', 'type': 'imdb'},
'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.995}],
'task': {'name': 'Text Classification', 'type': 'text-classification'}
}]
new_results = deepcopy(existing_results)
new_results[0]["metrics"][0]["value"] = 0.999
_update_metadata_model_index(existing_results, new_results, overwrite=True)
[{'dataset': {'name': 'IMDb', 'type': 'imdb'},
'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.999}],
'task': {'name': 'Text Classification', 'type': 'text-classification'}}]
new_results = deepcopy(existing_results)
new_results[0]["metrics"][0]["name"] = "Recall"
new_results[0]["metrics"][0]["type"] = "recall"
[{'dataset': {'name': 'IMDb', 'type': 'imdb'},
'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.995},
{'name': 'Recall', 'type': 'recall', 'value': 0.995}],
'task': {'name': 'Text Classification', 'type': 'text-classification'}}]
new_results = deepcopy(existing_results)
new_results[0]["dataset"] = {'name': 'IMDb-2', 'type': 'imdb_2'}
[{'dataset': {'name': 'IMDb', 'type': 'imdb'},
'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.995}],
'task': {'name': 'Text Classification', 'type': 'text-classification'}},
{'dataset': ({'name': 'IMDb-2', 'type': 'imdb_2'},),
'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.995}],
'task': {'name': 'Text Classification', 'type': 'text-classification'}}]
Published by LysandreJik over 2 years ago
Disclaimer: This release was initially released with advertised support for #844. It was not released in this release and will be in v0.7.
v0.6.0 introduces downstream (download) and upstream (upload) support for the fastai libraries. It supports fastai versions above 2.4.
The integration is detailed in the following blog.
Repository
Binary files are now rejected by default by the Hub. v0.6.0 introduces automatic binary file tracking through the auto_lfs_track
argument of the Repository.git_add
method. It also introduces the Repository.auto_track_binary_files
method which can be used independently of other methods.
skip_lfs_file
is now added to mixinsThe parameter skip_lfs_files
is now added to the different mixins. This will enable pushing files to the hub without first downloading the files above 10MB. This should drammatically reduce the time needed when updating a modelcard, a configuration file, and others.
The support for Keras model is greatly improved through several additions:
save_pretrained_keras
method now accepts a list of tags that will automatically be added to the repository.hf_api
a bit and add support for Spaces by @julien-c in #792Published by LysandreJik over 2 years ago
This is a patch release fixing a breaking backward compatibility issue.
Linked PR: https://github.com/huggingface/huggingface_hub/pull/822
Published by LysandreJik over 2 years ago
Version v0.5.0 is the first version which features an API reference. It is still a work in progress with features lacking, some images not rendering, and a documentation reorg coming up, but should already provide significantly simpler access to the huggingface_hub
API.
The documentation is visible here.
The list_models
and list_datasets
methods have been improved in several ways.
These two methods now accept the token
keyword to specify your token. Specifying the token will include your private models and datasets in the returned list.
These two methods now accept the cardData
boolean argument. If set to True
, the modelcard metadata will also be returned when using these two methods.
The list_models
method now also accepts an emissions_trehsholds
parameter to filter by carbon emissions.
The Keras serialization and upload methods have been worked on to provide better support for models:
push_to_hub_keras
log_dir
parameter for TensorBoard logs, which will automatically spawn a TensorBoard instance on the Hub.include_optimizer
parameter to push_to_hub_keras()
by @merveenoyan in https://github.com/huggingface/huggingface_hub/pull/616
test_keras_integration.py
by @merveenoyan in https://github.com/huggingface/huggingface_hub/pull/761
A contributing guide is now available for the huggingface_hub
repository. For any and all information related to contributing to the repository, please check it out!
Read more about it here: CONTRIBUTING.md.
The huggingface_hub
GitHub repository has several checks to ensure that the code respects code quality standards. Opt-in pre-commit hooks have been added in order to make it simpler for contributors to leverage them.
Read more about it in the aforementionned CONTRIBUTING guide.
Repositories can now be renamed and transferred programmatically using move_repo
.
⛔ The following methods have now been removed following a deprecation cycle
list_repos_objs
The list_repos_objs
and the accompanying CLI utility huggingface-cli repo ls-files
have been removed.
The same can be done using the model_info
and dataset_info
methods.
list_repos_objs
and huggingface-cli repo ls-files
by @julien-c in https://github.com/huggingface/huggingface_hub/pull/702
Python 3.6 support is now dropped as end of life. Using Python 3.6 and installing huggingface_hub
will result in version v0.4.0 being installed.
⚠️ Items below are now deprecated and will be removed in a future version
post_method
is only executed once by @sgugger in https://github.com/huggingface/huggingface_hub/pull/676
settings/token
to settings/tokens
by @ronvoluted in https://github.com/huggingface/huggingface_hub/pull/699
hf_api
, logging documentation by @LysandreJik in https://github.com/huggingface/huggingface_hub/pull/748
Hf.Api.delete_repo
by @adrinjalali in https://github.com/huggingface/huggingface_hub/pull/783
HF_ENDPOINT
. by @Narsil in https://github.com/huggingface/huggingface_hub/pull/819
Full Changelog: https://github.com/huggingface/huggingface_hub/compare/v0.4.0...v0.5.0
Published by LysandreJik over 2 years ago
This PR introduces the ability to fetch all available tags for models or datasets and returns them as a nested namespace object, for example:
>>> from huggingface_hub import HfApi
>>> api = HfApi()
>>> tags = api.get_model_tags()
>>> print(tags)
Available Attributes:
* benchmark
* language_creators
* languages
* licenses
* multilinguality
* size_categories
* task_categories
* task_ids
>>> print(tags.benchmark)
Available Attributes:
* raft
* superb
* test
With a goal of adding more tab-completion to the library, this PR introduces two objects:
DatasetSearchArguments
ModelSearchArguments
These two AttributeDictionary
objects contain all the valid information we can extract from a model as tab-complete parameters. We also include the author_or_organization
and dataset
(or model
) _name
as well through careful string splitting.
This PR introduces a new way to search the hub: the ModelFilter
class.
It is a simple Enum at first to the user, allowing them to specify what they want to search for, such as:
f = ModelFilter(author="microsoft", model_name="wavlm-base-sd", framework="pytorch")
From there, they can pass in this filter to the new list_models_by_filter
function in HfApi
to search through it:
models = api.list_modes(filter=f)
The API may then be used for complex queries:
args = ModelSearchArguments()
f = ModelFilter(framework=[args.library.pytorch, args.library.TensorFlow], model_name="bert", tasks=[args.pipeline_tag.Summarization, args.pipeline_tag.TokenClassification])
api.list_models_from_filter(f)
This PR introduces a way to limit the files that will be fetched by the snapshot_download
. This is useful when you want to download and cache an entire repository without using git, and that you want to skip files according to their filenames.
espnet
. by @Narsil in https://github.com/huggingface/huggingface_hub/pull/542
Full Changelog: https://github.com/huggingface/huggingface_hub/compare/v0.2.1...v0.4.0
Published by LysandreJik over 2 years ago
This is a patch release fixing an issue with the notebook login.
Published by LysandreJik over 2 years ago
Version v0.2.0 introduces the access token compatibility with the hub. It offers the access tokens as the main login handler, with the possibility to still login with username/password when doing [Ctrl/CMD]+C on the login prompt:
The notebook login is adapted to work with the access tokens.
The Repository
class now has an additional parameter, skip_lfs_files
, which allows cloning the repository while skipping the large file download.
https://github.com/huggingface/huggingface_hub/pull/472
snapshot_download
The snapshot_download
method can now take local_files_only
as a parameter to enable leveraging previously downloaded files.
Published by LysandreJik almost 3 years ago
Full Changelog: https://github.com/huggingface/huggingface_hub/compare/v0.1.1...v0.1.2
Published by LysandreJik almost 3 years ago
create_repo
for Repository.clone_from
by @sgugger in https://github.com/huggingface/huggingface_hub/pull/459
Full Changelog: https://github.com/huggingface/huggingface_hub/compare/v0.1.0...v0.1.1
Published by LysandreJik almost 3 years ago
Version v0.1.0 is the first minor release of the huggingface_hub
package, which promises better stability for the incoming versions. This update comes with big quality of life improvements.
Previously, most methods of the HfApi
class required the token to be explicitly passed. This is changed in this version, where it defaults to the token stored in the cache. This results in a re-ordering of arguments, but backward compatibility is preserved in most cases. Where it is not preserved, an explicit error is thrown.
HfApi
by @LysandreJik in https://github.com/huggingface/huggingface_hub/pull/388
The HfApi
class now exposes its methods through the hf_api
file, reducing the friction to access these helpers. See the example below:
# Previously
from huggingface_hub import HfApi
api = HfApi()
user = api.whoami()
# Now
from huggingface_hub.hf_api import whoami
user = whoami()
The HfApi
can still be imported and works as before for backward compatibility.
list_repo_files
util by @sgugger in https://github.com/huggingface/huggingface_hub/pull/395
Offers a list_repo_files
to ... list the repo files! Supports both model repositories and dataset repositories
model-index
, with proper typing by @julien-c in https://github.com/huggingface/huggingface_hub/pull/382
Offers a metadata_eval_result
in order to generate a YAML block to put in model cards according to evaluation results.
Adds a list_metrics method to HfApi!
Adds a git_prune
method to the Repository
class. This prunes local files which are unneeded as already pushed to a remote.
It adds the argument auto_lfs_prune
to git_push
and the commit
context-manager for simpler handling.
datasets
' push_to_hub
method by @LysandreJik in https://github.com/huggingface/huggingface_hub/pull/415
Full Changelog: https://github.com/huggingface/huggingface_hub/compare/v0.0.19...v0.1.0