Highlight - Transforming KeyPoints and Rotated boxes!

📦 This release is introducing two highly popular feature requests: Transforms support for KeyPoints and Rotated Bounding Boxes! * Rotated Bounding Boxes provide a tighter fit and alignment with rotated and elongated objects, which improves the localization, reduces overlap in densely packed images, and improves isolation of objects in crowded scenes. * KeyPoints offer a robust and accurate way to identify and locate specific points of interest within an image or video frame. These features aim at improving developer experience to implement use cases, including detecting & tracking objects, estimating pose, analyzing facial expressions, and creating augmented reality experiences.

We illustrated the use of Rotated Bounding Boxes below. You can expect keypoints and rotated boxes to work with all existing torchvision transforms in torchvision.transforms.v2. You can find some examples on how to use those transformations in our Transforms on Rotated Bounding Boxes tutorials.

[!NOTE] These features are released in BETA status. The API are unlikely to change, but we may have some rough edges and we may make some slight bug fixes in future releases. Please let us know if you encounter any issue!

Detailed changes

New Features

[transforms] Added support for BoundingBoxes formats and transforms (#9104, [#9084], [#9095], [#9138]) [transforms] Added the KeyPoints to TVTensor and support for transforms (#8817)

Improvements

[utils] Add label background to draw_bounding_boxes (#9018) [MPS] Add deformable conv2d kernel support on MPS (#9017, [#9115]) [documentation] Various documentation improvements (#9063, [#9119], [#9083], [#9105], [#9106], [#9145]) [code-quality] Bunch of code quality improvements (#9087, [#9093], [#8814], [#9035], [#9120], [#9080], [#9027], [#9062], [#9117], [#9024], [#9032])

Bug Fixes

[datasets] Fix COCO dataset to avoid issue when copying the dataset results (#9107) [datasets] Raise error when download=True for LFW dataset, which is not available for download anymore [#9040]) [tv_tensors] Add error message when setting 1D tensor ToImage() (#9114) [io] Warn when webp is asked to decode into grayscale (#9101)

Contributors

🎉 We're grateful for our community, which helps us improve Torchvision by submitting issues and PRs, and providing feedback and suggestions. The following persons have contributed patches for this release: @AlannaBurke, @Alexandre-SCHOEPP, @atalman, @AntoineSimoulin, @BoyuanFeng, @cyyever, @elmuz, @emmanuel-ferdman, @hmk114, @Isalia20, @NicolasHug, @malfet, @chengolivia, @RhutvikH, @hvaara, @scotts, @alinpahontu2912, @tsahiasher, and @youcefouadjer.