VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models
OTHER License
Please Join us and create your own film on Discord/Floor33.
π€π€π€ VideoCrafter is an open-source video generation and editing toolbox for crafting video content. It currently includes the Text2Video and Image2Video models:
Click the GIF to access the high-resolution video.
π₯ You are highly recommended to try our dedicated I2V model DynamiCrafter: Higher resolution, Better Dynamics, More Coherence!!!
[2024.02.05]: π₯π₯ Release new I2V model with the resolution of 640x1024 of VideoCrafter1/DynamiCrafter.
[2024.01.26]: Release the 512x320 checkpoint of VideoCrafter2.
[2024.01.18]: Release the VideoCrafter2 and Tech Report!
[2023.10.30]: Release VideoCrafter1 Technical Report!
[2023.10.13]: Release the VideoCrafter1, High Quality Video Generation!
[2023.08.14]: Release a new version of VideoCrafter on Discord/Floor33. Please join us to create your own film!
[2023.04.18]: Release a VideoControl model with most of the watermarks removed!
[2023.04.05]: Release pretrained Text-to-Video models, VideoLora models, and inference code.
T2V-Models | Resolution | Checkpoints |
---|---|---|
VideoCrafter2 | 320x512 | Hugging Face |
VideoCrafter1 | 576x1024 | Hugging Face |
VideoCrafter1 | 320x512 | Hugging Face |
I2V-Models | Resolution | Checkpoints |
---|---|---|
VideoCrafter1 | 640x1024 | Hugging Face |
VideoCrafter1 | 320x512 | Hugging Face |
conda create -n videocrafter python=3.8.5
conda activate videocrafter
pip install -r requirements.txt
model.ckpt
in checkpoints/base_512_v2/model.ckpt
. sh scripts/run_text2video.sh
model.ckpt
in checkpoints/i2v_512_v1/model.ckpt
. sh scripts/run_image2video.sh
python gradio_app.py
π VideoCrafter2 Tech report: VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models
π VideoCrafter1 Tech report: VideoCrafter1: Open Diffusion Models for High-Quality Video Generation
The technical report is currently unavailable as it is still in preparation. You can cite the paper of our image-to-video model and related base model.
@misc{chen2024videocrafter2,
title={VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models},
author={Haoxin Chen and Yong Zhang and Xiaodong Cun and Menghan Xia and Xintao Wang and Chao Weng and Ying Shan},
year={2024},
eprint={2401.09047},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{chen2023videocrafter1,
title={VideoCrafter1: Open Diffusion Models for High-Quality Video Generation},
author={Haoxin Chen and Menghan Xia and Yingqing He and Yong Zhang and Xiaodong Cun and Shaoshu Yang and Jinbo Xing and Yaofang Liu and Qifeng Chen and Xintao Wang and Chao Weng and Ying Shan},
year={2023},
eprint={2310.19512},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@article{xing2023dynamicrafter,
title={DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors},
author={Jinbo Xing and Menghan Xia and Yong Zhang and Haoxin Chen and Xintao Wang and Tien-Tsin Wong and Ying Shan},
year={2023},
eprint={2310.12190},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@article{he2022lvdm,
title={Latent Video Diffusion Models for High-Fidelity Long Video Generation},
author={Yingqing He and Tianyu Yang and Yong Zhang and Ying Shan and Qifeng Chen},
year={2022},
eprint={2211.13221},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Our codebase builds on Stable Diffusion. Thanks the authors for sharing their awesome codebases!
We develop this repository for RESEARCH purposes, so it can only be used for personal/research/non-commercial purposes.