LibriVox Mirror: A Public-Domain Audiobook Dataset on Hugging Face
LibriVox volunteers record public-domain books and give the audio away. The Internet Archive hosts it. They have been doing this for more than 20 years.1
I wanted to use the collection in large audio jobs. The Internet Archive is an excellent archive, but repeatedly walking half a million MP3s is a rough input pipeline.
So that’s what LibriVox Mirror is for. It keeps the original MP3s, packs each book into a streamable WebDataset TAR, and adds typed Parquet indexes for books and audio segments.2
I guess the main point is convenience: the mirror gives you LibriVox already organized and fast to stream from Hugging Face, without another crawl through the Internet Archive.
At the time of this build: 21,797 books, 494,821 segments, 133,073 hours, and 7.65 TB.3
Here is the collection by Internet Archive publication date, in a GitHub-esque contribution graph.4
| Year | Segments |
|---|---|
| 2005 | 282 |
| 2006 | 5491 |
| 2007 | 14648 |
| 2008 | 17346 |
| 2009 | 22477 |
| 2010 | 22262 |
| 2011 | 23326 |
| 2012 | 23204 |
| 2013 | 23680 |
| 2014 | 20865 |
| 2015 | 23165 |
| 2016 | 24569 |
| 2017 | 27607 |
| 2018 | 26023 |
| 2019 | 26246 |
| 2020 | 32717 |
| 2021 | 32168 |
| 2022 | 29702 |
| 2023 | 29390 |
| 2024 | 27858 |
| 2025 | 23366 |
| 2026 | 18429 |
Every sample keeps its LibriVox project, exact Internet Archive source file, upstream checksums, and a mirror SHA-256.5 Records with missing or contradictory source files are quarantined instead of guessed.6
A scheduled sync picks up new books as LibriVox publishes them. The dataset is on Hugging Face, and the mirroring tool is open source.
Footnotes
- LibriVox accepts readers without auditions or prior recording experience. The project says it has no paid staff, board, or offices. See how volunteering works and its note to donors. ↩
- The mirror’s compilation, curation, normalized metadata, indexes, and documentation are CC BY 4.0. The original LibriVox audio remains public domain in the United States and is not relicensed by the mirror. See the dataset’s license and attribution. ↩
- Book, segment, and duration totals come from the mirror’s versioned
state/sync.json. Storage comes from Hugging Face repository usage. Both are fetched at build time. ↩ - The grid reads each book’s exact
section_countandarchive_metadata.publicdate, then groups those segments into seven-day blocks and three-hour UTC windows. It measures when the containing book became public, not when somebody narrated it. The data is refreshed at build time, with a last-known snapshot as fallback. ↩ - The mirror is a downstream cache, not a replacement for either project. The source links and checksums make every file traceable to the original. ↩
- Typical failures are a missing Archive item or metadata naming an MP3 that does not exist. ↩