Caltech Mouse Social Interactions (CalMS21) Dataset

Description

Multi-agent behavior modeling aims to understand the interactions that occur between agents. We present a multi-agent dataset from behavioral neuroscience, the Caltech Mouse Social Interactions (CalMS21) Dataset. Our dataset consists of trajectory data of social interactions, recorded from videos of freely behaving mice in a standard resident-intruder assay. To help accelerate behavioral studies, the CalMS21 dataset provides benchmarks to evaluate the performance of automated behavior classification methods in three settings: (1) for training on large behavioral datasets all annotated by a single annotator, (2) for style transfer to learn inter-annotator differences in behavior definitions, and (3) for learning of new behaviors of interest given limited training data. The dataset consists of 6 million frames of unlabeled tracked poses of interacting mice, as well as over 1 million frames with tracked poses and corresponding frame-level behavior annotations. The challenge of our dataset is to be able to classify behaviors accurately using both labeled and unlabeled tracking data, as well as being able to generalize to new settings.

Files

Other

Our trajectory dataset is available in json format. Please download and unzip the zip files for each task to access the data. Our video datasets are available as mp4 format (task1_videos_mp4.zip) or seq format (task1_videos_seq.zip). Additionally, we release the pre-computed MARS features on our dataset in task1_MARS_features.zip. The readme included for download with the dataset provides a more detailed description of the data format, and a script to read json files and convert them to npy files (Python) is included. See our website for more info: https://sites.google.com/view/computational-behavior/our-datasets/calms21-dataset See our paper for more details on the dataset and benchmarks: https://arxiv.org/abs/2104.02710

Other

Please cite the following paper if you find the dataset useful: https://arxiv.org/abs/2104.02710 BibTeX: @article{calms21, title={The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions}, author={Sun, Jennifer J and Karigo, Tomomi and Chakraborty, Dipam and Mohanty, Sharada P and Wild, Benjamin and Sun, Quan and Chen, Chen and Anderson, David J and Perona, Pietro and Yue, Yisong and Kennedy, Ann}, journal={arXiv preprint arXiv:2104.02710}, year={2021} }

Additional details