AI method recovers missing movement data to enhance behavioral analysis

Scientists learn about the brain’s inner workings by studying what animals or people do, how they move, react, and make choices. Behavior is complex, as animals and humans can move in countless different manners. Yet, neuroscience studies have heavily relied on constrained, simplistic behaviors, which are easier to analyze and quantify.

Now, recording and tracking technologies offer scientists access to the movements of body parts of freely behaving animals at sub-second and millimeter scales. These detections of animal body parts often contain missing data that can hinder the analysis of both behavior and neuroscience experiments.

New method for recovering missing data

A research team led by Professor Dr. Katarzyna Bozek from the Center for Molecular Medicine Cologne (CMMC) and the CECAD Cluster of Excellence on Aging Research designed a method for recovering missing data in recordings of animal behavior.

The study “Deep Imputation for Skeleton data (DISK) for behavioral science” was published in Nature Methods. Collaborations with research labs at the Okinawa Institute of Science and Technology (OIST) in Japan, Vrije Universiteit Amsterdam, and the Salk Institute for Biological Studies in the U.S. provided behavior experimental data of mice and zebrafish.

The Deep Imputation for Skeleton Data method (DISK), is based on a Transformer neural network architecture and operates across species, ranging from insects to fish to rodents. The method can recover and fill in missing data across all parts of the body and provide real-time estimations of the quality of the data recovery.

Benefits and applications of DISK

“We aimed to make the tool usable by a broad community of behavioral researchers, without relying on prior knowledge of the species, the number of animals, or any specific information about the behavioral task,” said Dr. France Rose, the first author of the study. Researchers using this new method can take full advantage of their behavioral experimental data, which are usually expensive and time-consuming to generate.

The researchers demonstrated that DISK can improve the robustness of scientific results of such experiments, e.g., the statistical power of comparing step dynamics between two mice groups under different pharmacological treatments. In addition to imputation, DISK learns meaningful representations of movement sequences that capture motion features such as speed and direction, as well as the performed actions, such as walking or climbing.

DISK will allow behavioral researchers to improve their data quality and analyses. “We also believe that the novel way of quantifying behavior using Transformer neural networks will find broad applications in this field of research,” said Bozek.