LibriVox Mirror: A Public-Domain Audiobook Dataset on Hugging Face
LibriVox volunteers record public-domain books and give the audio away. The Internet Archive hosts it. They have been doing this for more than 20 years.1
I wanted to use the collection in large audio jobs. The Internet Archive is an excellent archive, but repeatedly walking half a million MP3s is a rough input pipeline.
So that’s what LibriVox Mirror is for. It keeps the original MP3s, packs each book into a streamable WebDataset TAR, and adds typed Parquet indexes for books and audio segments.
I guess the main point is convenience: the mirror gives you LibriVox already organized and fast to stream from Hugging Face, without another crawl through the Internet Archive.
At the time of this build: 21,644 books, 491,026 segments, 131,973 hours, and 7.58 TB.2
Here is the collection by Internet Archive publication date, in a GitHub-esque contribution graph.3
| Year | Segments |
|---|---|
| 2005 | 282 |
| 2006 | 5491 |
| 2007 | 14648 |
| 2008 | 17346 |
| 2009 | 22477 |
| 2010 | 22262 |
| 2011 | 23326 |
| 2012 | 23204 |
| 2013 | 23680 |
| 2014 | 20865 |
| 2015 | 23165 |
| 2016 | 24569 |
| 2017 | 27607 |
| 2018 | 24963 |
| 2019 | 25928 |
| 2020 | 32614 |
| 2021 | 32147 |
| 2022 | 29702 |
| 2023 | 29390 |
| 2024 | 27858 |
| 2025 | 23338 |
| 2026 | 16164 |
Every sample keeps its LibriVox project, exact Internet Archive source file, upstream checksums, and a mirror SHA-256.4 Records with missing or contradictory source files are quarantined instead of guessed.5
A scheduled sync picks up new books as LibriVox publishes them. The dataset is on Hugging Face, and the mirroring tool is open source.
Footnotes
- LibriVox accepts readers without auditions or prior recording experience. The project says it has no paid staff, board, or offices. See how volunteering works and its note to donors. ↩
- Book, segment, and duration totals come from the mirror’s versioned
state/sync.json. Storage comes from Hugging Face repository usage. Both are fetched at build time. ↩ - The grid reads each book’s exact
section_countandarchive_metadata.publicdate, then groups those segments into seven-day blocks and three-hour UTC windows. It measures when the containing book became public, not when somebody narrated it. The data is refreshed at build time, with a last-known snapshot as fallback. ↩ - The mirror is a downstream cache, not a replacement for either project. The source links and checksums make every file traceable to the original. ↩
- Typical failures are a missing Archive item or metadata naming an MP3 that does not exist. ↩