ModelScope Files

English Version

• Added Hub download progress callback: Supports custom callback functions for download progress updates. https://github.com/modelscope/modelscope/pull/1380
• Added environment variable configuration: Introduced the SIZE_THRESHOLD_TO_ENFORCE_LFS environment variable to configure the file size threshold for enforcing LFS usage. https://github.com/modelscope/modelscope/pull/1383
• Added wildcard batch deletion: Supports batch deletion of files in model or dataset repositories using wildcards. https://github.com/modelscope/modelscope/pull/1384
• Optimized VAE processing: Improved VAE chunking workflow and reduced VRAM consumption. https://github.com/modelscope/modelscope/pull/1378
• Added Hub upload environment variables: Added environment variable configurations for Hub uploads. https://github.com/modelscope/modelscope/pull/1389
• Compatibility improvement: Added compatibility support for datasets library version 3.6. https://github.com/modelscope/modelscope/pull/1393
• Bug fix: Resolved an issue where AutoProcessor and AutoTokenizer might download incomplete files when using from_pretrained. https://github.com/modelscope/modelscope/pull/1398

中文版本

• 新增 Hub 下载进度回调支持： 支持自定义下载进度更新回调函数。 https://github.com/modelscope/modelscope/pull/1380
• 新增环境变量配置： 添加 SIZE_THRESHOLD_TO_ENFORCE_LFS 环境变量，用于配置强制使用 LFS 的文件大小阈值。 https://github.com/modelscope/modelscope/pull/1383
• 新增通配符批量删除功能： 支持使用通配符批量删除模型或数据集仓库中的文件。 https://github.com/modelscope/modelscope/pull/1384
• 优化 VAE 处理： 优化 VAE 分块处理流程并减少显存占用。https://github.com/modelscope/modelscope/pull/1378
• 新增 Hub 上传环境变量配置： 增加 Hub 上传相关的环境变量配置选项。 https://github.com/modelscope/modelscope/pull/1389
• 兼容性提升： 兼容 datasets 库 3.6 版本。 https://github.com/modelscope/modelscope/pull/1393
• 问题修复： 修复 AutoProcessor 和 AutoTokenizer 使用 from_pretrained 方法时可能出现的文件下载不完整问题。https://github.com/modelscope/modelscope/pull/1398

What's Changed

• [hub] support ProgressCallback by @Jintao-Huang in https://github.com/modelscope/modelscope/pull/1380
• Add env SIZE_THRESHOLD_TO_ENFORCE_LFS by @wangxingjun778 in https://github.com/modelscope/modelscope/pull/1383
• Feat deletefiles by @Koko-ry in https://github.com/modelscope/modelscope/pull/1384
• Merge release 1.27 by @hjh0119 in https://github.com/modelscope/modelscope/pull/1392
• perf(vae): optimize tiling performance and memory usage by @MengAiDev in https://github.com/modelscope/modelscope/pull/1378
• Add upload check envs by @wangxingjun778 in https://github.com/modelscope/modelscope/pull/1389
• Upgrade datasets by @wangxingjun778 in https://github.com/modelscope/modelscope/pull/1393
• Fix ignore files of AutoProcessor by @tastelikefeet in https://github.com/modelscope/modelscope/pull/1398

New Contributors

• @Koko-ry made their first contribution in https://github.com/modelscope/modelscope/pull/1384
• @hjh0119 made their first contribution in https://github.com/modelscope/modelscope/pull/1392
• @MengAiDev made their first contribution in https://github.com/modelscope/modelscope/pull/1378

Full Changelog: https://github.com/modelscope/modelscope/compare/v1.27.1...v1.28.0