Published June 7, 2021 | Version 1.0
Dataset Open

Caltech Mouse Social Interactions (CalMS21) Dataset

  • 1. ROR icon California Institute of Technology
  • 2. ROR icon Northwestern University

Description

Multi-agent behavior modeling aims to understand the interactions that occur between agents. We present a multi-agent dataset from behavioral neuroscience, the Caltech Mouse Social Interactions (CalMS21) Dataset. Our dataset consists of trajectory data of social interactions, recorded from videos of freely behaving mice in a standard resident-intruder assay. To help accelerate behavioral studies, the CalMS21 dataset provides benchmarks to evaluate the performance of automated behavior classification methods in three settings: (1) for training on large behavioral datasets all annotated by a single annotator, (2) for style transfer to learn inter-annotator differences in behavior definitions, and (3) for learning of new behaviors of interest given limited training data. The dataset consists of 6 million frames of unlabeled tracked poses of interacting mice, as well as over 1 million frames with tracked poses and corresponding frame-level behavior annotations. The challenge of our dataset is to be able to classify behaviors accurately using both labeled and unlabeled tracking data, as well as being able to generalize to new settings.

Files

task1_videos_mp4.zip
Files (109.8 GB)
Name Size
md5:790b2ff6054c0889c3b0112b3a12eacc
28.3 GB Preview Download
md5:b771c359e96a2f6576151e695166652b
74.2 GB Preview Download
md5:c97e87e13e77ffb80c073e05c05a4683
912.1 MB Preview Download
md5:068f0362ea9879a4051b66bd60656c29
4.4 kB Download
md5:ad3faeae6835747fd30fcc0c8e83ad2e
10.9 kB Preview Download
md5:df59df02d069bab1cfc376cdc1a3925b
556.2 MB Preview Download
md5:ae250529ec80fb268aa068b5bc35bda1
9.2 kB Preview Download
md5:2a8b758dfff719a7310a24fa5a7a2435
1.8 GB Preview Download
md5:35ab3acdeb231a3fe1536e38ad223b2e
3.5 GB Preview Download
md5:8a02654fddae28614ee24a6a082261b8
457.4 MB Preview Download

Other

Our trajectory dataset is available in json format. Please download and unzip the zip files for each task to access the data. Our video datasets are available as mp4 format (task1_videos_mp4.zip) or seq format (task1_videos_seq.zip). Additionally, we release the pre-computed MARS features on our dataset in task1_MARS_features.zip. The readme included for download with the dataset provides a more detailed description of the data format, and a script to read json files and convert them to npy files (Python) is included. See our website for more info: https://sites.google.com/view/computational-behavior/our-datasets/calms21-dataset See our paper for more details on the dataset and benchmarks: https://arxiv.org/abs/2104.02710

Other

Please cite the following paper if you find the dataset useful: https://arxiv.org/abs/2104.02710 BibTeX: @article{calms21, title={The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions}, author={Sun, Jennifer J and Karigo, Tomomi and Chakraborty, Dipam and Mohanty, Sharada P and Wild, Benjamin and Sun, Quan and Chen, Chen and Anderson, David J and Perona, Pietro and Yue, Yisong and Kennedy, Ann}, journal={arXiv preprint arXiv:2104.02710}, year={2021} }

Additional details

Created:
September 13, 2022
Modified:
November 28, 2022