Search Results for "transformer"

...Natural-language instructions can drive subject edits, local changes, background replacement, style transfer, motion changes, and reference-guided transformations. Its architecture combines a multimodal condition encoder, causal video VAE, and a 16B-parameter multimodal diffusion transformer. Autoregressive diffusion and bounded KV-state inference are used to keep computation stable across long streams. The released deployment reaches high-throughput 720p editing and also supports real-time operation on selected consumer GPUs. Updated checkpoints improve identity preservation, reference conditioning, and temporal consistency for reference-image-guided editing.

ProPainter is a video inpainting project based on the ICCV 2023 paper “Improving Propagation and Transformer for Video Inpainting.” It is designed to remove objects, complete missing regions, and fill masked areas in videos while preserving temporal consistency. The project accepts video input or split frames along with frame-wise masks that define the areas to reconstruct. It provides pretrained models, example inputs, inference scripts, and an interactive demo workflow for object removal. ...

TimeSformer is a vision transformer architecture for video that extends the standard attention mechanism into spatiotemporal attention. The model alternates attention along spatial and temporal dimensions (or designs variants like divided attention) so that it can capture both appearance and motion cues in video. Because the attention is global across frames, TimeSformer can reason about dependencies across long time spans, not just local neighborhoods.