Every session is captured simultaneously from two synchronized head-mounted cameras, paired with two exocentric views as well as dense scene scans and high-resolution object scans — enabling a new class of multi-view egocentric tasks.
A short walkthrough of CoMind: dual-view egocentric capture, scene and object scans, and the three benchmark tasks.
Every session is recorded by two participants wearing synchronized capture glasses. Both streams are frame-aligned, hardware-timestamped, and come with shared scene & object scans.
Beyond video, every session ships with the 3D context it was captured in.
Photogrammetric scans of each recording environment, delivered as point clouds and aligned to ego-camera coordinates for direct reprojection.
Interacted objects are independently scanned at sub-millimeter resolution, enabling applications involving 6-DoF pose extraction and rendering-based evaluation.
Tasks constructed using realistic social scenarios, requiring complex reasoning about social cues in the provided speech transcripts and context frames.
Given two synchronized ego views at a moment of joint attention, the model predicts the category of the jointly attended object, its bounding box in the left and right views, and the type of social cue that can be used to infer which object is being attended to.
From context frames and transcribed audio together with a prediction frame, the model predicts the noun and verb of the action to be performed, the interacted object's bounding box, and the type of social cue that can be used to infer the next interacted object.
From context frames and transcribed audio together with a prediction frame showing a moment directly preceding a handover event, the model predicts the delivery flow (who hands to whom), the object category and bounding box in the view of the handing participant, the initiator, and the cue type.
Evaluate your model on the CoMind test set. Submit your test-set predictions for a task through its form below — we score them against the held-out annotations and add your entry here. All metrics are fractions in [0, 1]; higher is better (↑).
| # | Method | L-IoU@0.5 ↑ | R-IoU@0.5 ↑ | Cue Type ↑ | Cat. (L1) ↑ |
|---|---|---|---|---|---|
| No submissions yet. Be the first to appear here. | |||||
| # | Method | IoU@0.5 ↑ | Cue Type ↑ | Act. Verb ↑ | Act. Noun (L1) ↑ |
|---|---|---|---|---|---|
| No submissions yet. Be the first to appear here. | |||||
| # | Method | IoU@0.5 ↑ | Del. Flow ↑ | Initiator ↑ | Init. Type ↑ | Cat. (L1) ↑ | TTH ↑ |
|---|---|---|---|---|---|---|---|
| No submissions yet. Be the first to appear here. | |||||||
CoMind is distributed through a Python download script.
Run it with the --help flag for usage, all options, and the full list of dataset components.
@inproceedings{gavryushin2026eccv,
author = {Gavryushin, Alexey and Zhang, Dingxi and Huang, Zhao and Delitzas, Alexandros and Chen, Jiaqi and Ellis, Ben and Z{\"o}llner, Cedric and Patel, Manthan and Kaufmann, Manuel and Pollefeys, Marc and Wang, Xi},
title = {{CoMind}: Understanding Collaborative Human Activity from Multiple Minds and Views},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2026},
publisher = {Springer}
}