Search Results for "state-thread"

...To support training, the team built a scalable data engine that automatically curates large video datasets with camera pose estimation and metric depth prediction. As a result, Voyager delivers state-of-the-art performance on world exploration benchmarks while maintaining photometric, style, and 3D consistency.

...Additionally, we will try adding an extra linear attention on the main branch as well as self-conditioning in the pixel space. The insight of being able to self-condition on any hidden state of the network as well as the newly proposed sigmoid noise schedule are the two main findings.

text-to-video-synthesis-colab is a collection of Google Colab notebooks that enable users to generate videos from text prompts using state-of-the-art AI models. It integrates multiple open-source models and pipelines, allowing users to experiment with different approaches to text-to-video generation. The notebooks are preconfigured for cloud execution, making them accessible without requiring local GPU setups. Users can generate, upscale, and refine videos directly from natural language descriptions. ...

Implementation of NÜWA, state of the art attention network for text-to-video synthesis, in Pytorch. It also contains an extension into video and audio generation, using a dual decoder approach. It seems as though a diffusion-based method has taken the new throne for SOTA. However, I will continue on with NUWA, extending it to use multi-headed codes + hierarchical causal transformer.