Vatsal Agarwal

I am a fifth year Ph.D student in the department of Computer Science at the University of Maryland (UMD), advised by Professor Abhinav Shrivastava. My research lies at the intersection of computer vision and deep learning.

I completed my undergrad in Computer Science at the University of Maryland in 2022. During my undergrad, I worked as a research assistant at the National Institutes of Health Clinical Center to develop deep learning models for medical imaging. I also interned as a machine learning engineer at DeepHealth.

In the past, I have been fortunate to have worked with Youbao Tang, Bill Lotter, Ronald Summers, Suhas Srinivasan.

Email / CV / Scholar / Github

Research

My research focuses on building scalable memory systems for long-context intelligence. I am currently developing memory that lets multimodal models process and reason over video streams from tens of minutes to several hours long. Previously, I worked on multimodal image understanding and panoptic image segmentation.

	MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos Arushi Goel, Sreyan Ghosh, Vatsal Agarwal, Nishit Anand, …, Abhinav Shrivastava, …, Wei Ping NeurIPS 2026 Project Page \| Paper We introduce MMOU, a benchmark of 20,000 manually annotated questions over 11,877 long, real-world videos that evaluates joint visual, audio, and textual understanding and reasoning across 13 skill categories.
	Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memory Vatsal Agarwal, Saksham Suri, Matthew Gwilliam , Pulkit Kumar, Abhinav Shrivastava NeurIPS 2026 Project Page \| Paper We propose a unified encoding and retrieval strategy for accurate streaming video understanding with higher token-budgets.
	DiffVLM: Towards Text-Aware Multimodal Understanding via Stable Diffusion Vatsal Agarwal, Matthew Gwilliam , Eshan Verma , Daniel Ulbricht , Abhinav Shrivastava WACV 2027 Project Page \| Paper We explore the potential of text-to-image diffusion models as visual encoders for MLLMs. Specifically, wen pass a question to a pre-trained diffusion model to obtain question-aware features.
	MAPS: Memory Augmented Panoptic Segmentation Vatsal Agarwal, Saksham Suri, Max Ehrlich, Abhinav Shrivastava ICCV 2025 Workshop: MemVis: The 1st Workshop on Memory and Vision (oral) Paper We explore the potential of explicit memory capturing scene-object associations for panoptic segmentation.
	Do Text-free Diffusion Models Learn Discriminative Visual Representations? Matthew Gwilliam* , Soumik Mukhopadhyay, Vatsal Agarwal^✝, Yosuke Yamaguchi^✝, Namitha Padmanabhan, Archana Swaminathan, Tianyi Zhou, Abhinav Shrivastava ECCV 2024* Project Page \|Paper Explore diffusion models as unified unsupervised image representation learning models for many recognition tasks. Propose DifFormer and DifFeed, novel mechanisms for fusing diffusion features for image classification.
	Diffusion Models Beat GANs on Image Classification Matthew Gwilliam* , Soumik Mukhopadhyay, Vatsal Agarwal, Namitha Padmanabhan, Archana Swaminathan, Tianyi Zhou, Abhinav Shrivastava preprint only* Project Page \|Paper Show the possible utility of diffusion models as unified unsupervised image representation learners.
	Coarse-to-Fine Human Mesh Recovery with Transformers Vatsal Agarwal, Mara Levy, Max Ehrlich, Youbao Tang, Ning Zhang, Abhinav Shrivastava ECCV 2024 Workshop: Towards a Complete Analysis of People: Fine-Grained Understanding for Real-World Applications Paper Build efficient Transformer design for non-parametric human mesh recovery with coarse-to-fine pipeline.
	Weakly Supervised Lesion Co-segmentation on CT Scans Vatsal Agarwal, Youbao Tang, Jing Xiao, Ronald M. Summers IEEE International Symposium on Biomedical Imaging (ISBI), 2020 Paper Conducted thorough investigation on impact of different attention mechanisms for lesion segmentation.
	Weakly-Supervised Lesion Segmentation on CT Scans using Co-Segmentation Vatsal Agarwal, Youbao Tang, Jing Xiao, Ronald M. Summers SPIE Medical Imaging, 2020 Paper Developed attention-based co-segmentation model and applied it to task of lesion segmentation.

Template credits