alibaba / alibaba/Tora

Welcome to use OpenS2V-Nexus to evaluate and train your latest models

Open
#42 1 comment 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.2k
Forks
61
PR merge metrics
No merged PRs in 30d

Description

@Leojc @zhenghao977 Thanks for great work **(Tora2)** ! We recently tackle the core challenges of ​​Subject-to-Video Generation (S2V)​​ by systematically building the first complete infrastructure—featuring an evaluation benchmark and a million-scale dataset! ✨Welcom to try it!

🧠 Introducing **​​OpenS2V-Eval​​**—the first ​​fine-grained S2V benchmark​​, with ​​180 multi-domain prompts + real/synthetic test pairs​​. We propose ​​NexusScore​​, ​​NaturalScore​​, and ​​GmeScore​​ to precisely quantify model performance across ​​subject consistency, naturalness, and text alignment​​ ✔

📊 Using this framework, we conduct a ​​comprehensive evaluation of 18 leading S2V models​​, revealing their strengths/weaknesses in complex scenarios!

🔥 ​​**OpenS2V-5M** dataset​​ now available! A ​​5.4M 720P HD​​ collection of ​​subject-text-video triplets​​, enabled by ​​cross-video association segmentation + multi-view synthesis​​ for ​​diverse subjects & high-quality annotations​​ 🚀

​​All resources open-sourced​​: Paper, Code, Data, and Evaluation Tools 📄
Let's advance S2V research together! 💡

🔗 ​​Links​​:
Code: https://github.com/PKU-YuanGroup/OpenS2V-Nexus
Project: https://pku-yuangroup.github.io/OpenS2V-Nexus
LeaderBoard: https://huggingface.co/spaces/BestWishYsh/OpenS2V-Eval
OpenS2V-5M: https://huggingface.co/datasets/BestWishYsh/OpenS2V-5M

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue points to OpenS2V-Nexus, its project page, leaderboard, and OpenS2V-5M dataset, but names no files, tests, or entry points in the Tora repository. Review the linked resources first; no concrete change or completion criteria are specified, so the work would need clarification before it can be started.

Written by the indexing model from the issue text.

Assessment

Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
10/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.