Datasets Files

Hub of ready-to-use datasets for ML models

This is an exact mirror of the Datasets project, hosted at https://github.com/huggingface/datasets. SourceForge is not affiliated with Datasets. For more information, see the SourceForge Open Source Mirror Directory.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
4.3.0 source code.tar.gz	2025-10-23	2.1 MB	0
4.3.0 source code.zip	2025-10-23	2.2 MB	2
README.md	2025-10-23	1.7 kB	0
Totals: 3 Items		4.3 MB	2

Dataset Features

Enable large scale distributed dataset streaming:

Keep hffs cache in workers when streaming by @lhoestq in https://github.com/huggingface/datasets/pull/7820
Retry open hf file by @lhoestq in https://github.com/huggingface/datasets/pull/7822

These improvements require huggingface_hub>=0.36.0 to take full effect

What's Changed

fix conda deps by @lhoestq in https://github.com/huggingface/datasets/pull/7810
Add pyarrow's binary view to features by @delta003 in https://github.com/huggingface/datasets/pull/7795
Fix polars cast column image by @CloseChoice in https://github.com/huggingface/datasets/pull/7800
Allow streaming hdf5 files by @lhoestq in https://github.com/huggingface/datasets/pull/7814
Fix batch_size default description in to_polars docstrings by @albertvillanova in https://github.com/huggingface/datasets/pull/7824
docs: document_dataset PDFs & OCR by @ethanknights in https://github.com/huggingface/datasets/pull/7812
Add custom fingerprint support to from_generator by @simonreise in https://github.com/huggingface/datasets/pull/7533
picklable batch_fn by @lhoestq in https://github.com/huggingface/datasets/pull/7826

New Contributors

@delta003 made their first contribution in https://github.com/huggingface/datasets/pull/7795
@CloseChoice made their first contribution in https://github.com/huggingface/datasets/pull/7800
@ethanknights made their first contribution in https://github.com/huggingface/datasets/pull/7812
@simonreise made their first contribution in https://github.com/huggingface/datasets/pull/7533

Full Changelog: https://github.com/huggingface/datasets/compare/4.2.0...4.3.0

Source: README.md, updated 2025-10-23

Other Useful Business Software

Simple, Secure Domain Registration Icon

Simple, Secure Domain Registration

Get your domain at wholesale price. Cloudflare offers simple, secure registration with no markups, plus free DNS, CDN, and SSL integration.

Register or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.

Sign up for free

MongoDB Atlas runs apps anywhere Icon

MongoDB Atlas runs apps anywhere

Deploy in 115+ regions with the modern database for every enterprise.

MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.

Start Free

Recommended Projects

SDGym
The Synthetic Data Gym (SDGym) is a benchmarking framework for modeling and generating synthetic data. Measure performance and memory usage across different synthetic data modeling techniques – classical statistics, deep learning and more! The SDGym library integrates with the Synthetic Data...
PromptSource
PromptSource is a toolkit for creating, sharing and using natural language prompts. Recent work has shown that large language models exhibit the ability to perform reasonable zero-shot generalization to new tasks. For instance, GPT-3 demonstrated that large language models have strong zero- and...
YouTube-8M
youtube-8m is Google’s open source starter code and reference implementation for training and evaluating machine learning models on the YouTube-8M dataset, one of the largest video understanding datasets publicly released. The repository provides a complete pipeline for video-level and...
Taipy
From simple pilots to production-ready web applications in no time. No more compromise on performance, customization, and scalability. Taipy enhances performance with caching control of graphical events, optimizing rendering by selectively updating graphical components only upon interaction....
Apache Spark
Apache Spark is a unified engine for large-scale data processing, offering APIs for batch jobs, streaming, machine learning, and graph computation. It builds on resilient distributed datasets (RDDs) and the newer DataFrame/Dataset abstractions to provide fault-tolerant, in-memory computation...