Search Results for "3d map generator"

Map-Anything is a universal, feed-forward transformer for metric 3D reconstruction that predicts a scene’s geometry and camera parameters directly from visual inputs. Instead of stitching together many task-specific models, it uses a single architecture that supports a wide range of 3D tasks—multi-image structure-from-motion, multi-view stereo, monocular metric depth, registration, depth completion, and more.

...As a general-purpose monocular depth backbone, Depth Pro slots into 3D reconstruction, relighting, and scene understanding workflows that benefit from metric predictions.

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch. They combine pseudo-3d convolutions (axial convolutions) and temporal attention and show much better temporal fusion. The pseudo-3d convolutions isn't a new concept. It has been explored before in other contexts, say for protein contact prediction as "dimensional hybrid residual networks". The gist of the paper comes down to, take a SOTA text-to-image model (here they use DALL-E2, but the same learning points would easily apply to Imagen), make a few minor modifications for attention across time and other ways to skimp on the compute cost, do frame interpolation correctly, get a great video model out. ...

Minigrid is a lightweight, minimalistic grid-world environment library for reinforcement learning (RL) research. It provides a suite of simple 2D grid-based tasks (e.g., navigating mazes, unlocking doors, carrying keys) where an agent moves in discrete steps and interacts with objects. The design emphasizes speed (agents can run thousands of steps per second), low dependency overhead, and high customizability — making it easy to define new maps, new tasks, or wrappers. It supports the...

...DensePose is widely used in augmented reality, motion capture, virtual try-on, and visual effects applications because it enables real-time 3D human mapping from 2D inputs. The model architecture builds on Mask R-CNN, using additional regression heads to predict UV coordinates that map image pixels to 3D surfaces.

VideoPose3D is a deep learning framework that reconstructs 3D human poses from 2D keypoint sequences extracted from videos. It builds on top of convolutional and temporal networks that map 2D joint coordinates over time to consistent 3D skeletons, enabling robust motion capture without specialized sensors. The model is trained on large motion capture datasets and can generalize well to unseen environments by leveraging temporal context for smoothing and error correction. ...