Major Updates

📊 Stats: 10 PRs merged, from 6 contributors

📈 Code diff: 125 files changed, with 11,287 insertions and 766 deletions

🧑‍🤝‍🧑 New OPs — Added 9 human-centric video understanding operators (human track extraction, active-speaker detection, audio ASR, speech emotion & age/gender detection, face demographic & attribute/emotion captioning, face-ratio filtering) for building HumanVBench-style pipelines. (These OPs need third-party patches/models under thirdparty/humanvbench_models/ and require a source install for now.) [#938] [#1013]

⚡ Batch-local stage fusion — New FusedSequentialBatchOp runs a list of batch-local sub-operators sequentially inside a single dataset map stage, reducing scheduler/stage overhead for mapper/filter chains. Wired through DefaultExecutor, RayExecutor, and PartitionedRayExecutor. [#1004]

New OPs

• 9 HumanVBench human-centric video OPs — 8 mappers (video_human_tracks_extraction_mapper, video_active_speaker_detect_mapper, video_audio_ASR_mapper, video_audio_speech_emotion_mapper, video_audio_detect_age_gender_mapper, video_human_tracks_face_demographic_mapper, video_captioning_face_attribute_emotion_mapper, video_captioning_from_human_tracks_mapper) + 1 filter (video_face_ratio_filter). [#938] [#1013]
• fused_sequential_batch_op (FusedSequentialBatchOp). [#1004]

Fixed Bugs

• HumanVBench ops runtime compatibility: scenedetect open_video API migration, SenseVoiceSmall model resolution via ModelScope, output-directory creation on init, and declared runtime deps. [#1013]
• Ray deduplicator shared state: share dedup backend state across map_batches tasks, reuse actor handles, materialize stats for stateful backends. [#978]
• Ray checkpoint relative path: resolve work_dir/checkpoint_dir to absolute paths in PartitionedRayExecutor. [#979]
• clean_html_mapper on null text: no longer errors on null/None text. [#1001]
• Boolean stat columns in ColumnWiseAnalysis: dj-analyze no longer crashes on boolean stat columns. [#1003]
• ARM64 (aarch64) install unblocked: precise decord/torchcodec platform markers restrict them to platforms with pre-built wheels (Linux x86_64, Windows x86_64, macOS arm64). [#998]

Enhancements

• Test coverage & cleanup: expanded utility/model-handling tests, plus assorted cleanup. [#1002] [#1007]

Acknowledgements

• @macroguo-ghy fixed Ray deduplicator shared-state handling. [#978]
• @wenhaozhao011-cmd fixed boolean stat column handling in ColumnWiseAnalysis. [#1003]
• @bglglzd contributed code/documentation cleanup. [#1007]

New Contributors

• @bglglzd made their first contribution in [#1007]

Full Changelog: https://github.com/datajuicer/data-juicer/compare/v1.5.3...v1.5.4