Skip to content

Commit

Permalink
Update datasets-download-stats.md
Browse files Browse the repository at this point in the history
  • Loading branch information
lhoestq authored Oct 24, 2024
1 parent f6df83b commit ec0aeca
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions docs/hub/datasets-download-stats.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Datasets Download Stats

## How are download stats generated for datasets?
## How are downloads counted for datasets ?

Counting the number of downloads for datasets is not a trivial task, as a single dataset repository might contain multiple files, from multiple subsets and splits (e.g. train/validation/test) and sometimes with many files in a single split. To solve this issue and avoid counting one person's download multiple times, we treat all files downloaded by a user within a 5-minute window as a single dataset download. This counting happens automatically on our servers when files are downloaded (through GET or HEAD requests), with no need to collect any user information or make additional calls.
Counting the number of downloads for datasets is not a trivial task, as a single dataset repository might contain multiple files, from multiple subsets and splits (e.g. train/validation/test) and sometimes with many files in a single split. To solve this issue and avoid counting one person's download multiple times, we treat all files downloaded by a user (based on their IP address) within a 5-minute window as a single dataset download. This counting happens automatically on our servers when files are downloaded (through GET or HEAD requests), with no need to collect any user information or make additional calls.

## Before Setpember 2024
## Before September 2024

The Hub used to provide download stats only for the datasets loadable via the `datasets` library. To determine the number of downloads, the Hub previously counted every time `load_dataset` was called in Python, excluding Hugging Face's CI tooling on GitHub. No information was sent from the user, and no additional calls were made for this. The count was done server-side as we served files for downloads. This means that:

Expand Down

0 comments on commit ec0aeca

Please sign in to comment.