Chenliang Xu Research Group

University of Rochester

How can video understanding methods link vision and language to describe actions and narratives in moving images? The group develops methods for video segmentation and fine-grained actor-action segmentation to separate actors and actions across frames using computer vision algorithms. Researchers build activity recognition and video storytelling models that combine visual features with natural language representations to generate coherent descriptions of video content. The group explores cross-modal audio-visual generation to synthesize complementary modalities for richer video understanding using deep multimodal learning. Recent projects include work on multimodal vision-and-x modeling and trustworthy AI for video tasks.