Uploaded August 2021 | Updated September 2026, 10 hours ago
This video elaborates on key ideas from the paper "Towards General Purpose Vision Systems." Specifically, you will learn:
- what constitutes a "general purpose" vision system and how they differ from current "N-purpose" systems
- what are the "tenets" of general purpose models
- the notion of "skill-concept factorization" and its significance in the evaluation of general purpose systems
- role of "foundation models" in building "general purpose" systems
- design and evaluation of GPV-1, a task-agnostic vision-language architecture, on the new COCO-SCE dataset as a step towards general purpose vision
--------------------------------------------------------------------------------------------------------------------------------------------
Arxiv: arxiv.org/abs/2104.00743
Demo: vision-explorer.allenai.org/general_purpose_vision
Code: github.com/allenai/gpv-1
Qualitative Results (randomly sampled): prior.allenai.org/projects/gpv
--------------------------------------------------------------------------------------------------------------------------------------------
Outline:
0:00 - Introduction
0:18 - State of Computer Vision (Single-Purpose & Multi-Purpose)
2:16 - N-Purpose to General Purpose
2:43 - Foundation Models vs General Purpose Models
3:23 - 3 Tenets of GPV
3:48 - Tenet 1: Generality of Architecture
6:27 - GPV-1 (Architecture & Learning)
9:28 - Tenet 2: Generality of Concepts across Skills
9:40 - Skill-Concept Factorization
10:46 - COCO-SCE: A benchmark to evaluate Skill-Concept Generalization
11:33 - Evaluating GPV-1 on COCO-SCE
13:08 - Tenet 3: Generality of Learning (Sample Efficiency & Retention when learning new tasks)
13:43 - Analysis on Referring Expression Comprehension
14:55 - Foundation Models in Vision and relation to GPVs (following up on 2.43)
15:49 - Limitations of GPV-1
16:18 - Takeaways
16:47 - Links to Demo, Qualitative Results, and Code
This video elaborates on key ideas from the paper "Towards General Purpose Vision Systems." Specifically, you will learn:
- what constitutes a "general purpose" vision system and how they differ from current "N-purpose" systems
- what are the "tenets" of general purpose models
- the notion of "skill-concept factorization" and its significance in the evaluation of general purpose systems
- role of "foundation models" in building "general purpose" systems
- design and evaluation of GPV-1, a task-agnostic vision-language architecture, on the new COCO-SCE dataset as a step towards general purpose vision
--------------------------------------------------------------------------------------------------------------------------------------------
Arxiv: arxiv.org/abs/2104.00743
Demo: vision-explorer.allenai.org/general_purpose_vision
Code: github.com/allenai/gpv-1
Qualitative Results (randomly sampled): prior.allenai.org/projects/gpv
--------------------------------------------------------------------------------------------------------------------------------------------
Outline:
0:00 - Introduction
0:18 - State of Computer Vision (Single-Purpose & Multi-Purpose)
2:16 - N-Purpose to General Purpose
2:43 - Foundation Models vs General Purpose Models
3:23 - 3 Tenets of GPV
3:48 - Tenet 1: Generality of Architecture
6:27 - GPV-1 (Architecture & Learning)
9:28 - Tenet 2: Generality of Concepts across Skills
9:40 - Skill-Concept Factorization
10:46 - COCO-SCE: A benchmark to evaluate Skill-Concept Generalization
11:33 - Evaluating GPV-1 on COCO-SCE
13:08 - Tenet 3: Generality of Learning (Sample Efficiency & Retention when learning new tasks)
13:43 - Analysis on Referring Expression Comprehension
14:55 - Foundation Models in Vision and relation to GPVs (following up on 2.43)
15:49 - Limitations of GPV-1
16:18 - Takeaways
16:47 - Links to Demo, Qualitative Results, and Code
