Identifying autism motor deficits in infants using computer vision

ABSTRACT Recent work suggests that motor differences in infancy can predict subsequent ASD diagnosis, with the promise of earlier identification of ASD, and faster referral to early intervention than is currently possible. However objectively quantifying these differences has proved challenging. Here we will address this question by leveraging dramatic recent progress in deep learning-driven image processing, combined with emerging techniques in the new field of computational ethology. As a proof of concept, we will apply these techniques to a rich, longitudinal video dataset of semi-structured behavioral assessments on the Autism Observation Scale for Infants (AOSI) from all four sites of the Infant Brain Imaging Study (IBIS). Together this includes videos of more than 400 infants. In Aim 1 we will use OpenPose, a recently developed algorithm for human pose estimation, to automatically extract the location of key infant body joints from each frame of these videos, and then use customized deep-learning approaches to track the movement of the infant from frame to frame. In Aim 2 we will leverage the 24-month diagnostic status and familial liability data to validate the multidimensional computer vision-extracted kinematic data obtained in Aim 1 as ASD motor-function biomarkers. In Aim 3 we will apply unsupervised computational ethology techniques to reveal signatures of ASD in fine motor behavior that will generalize beyond the AOSI paradigm. This will result in a quantitative, precise and naturalistic description of infant motor behavior suitable for predicting ASD diagnosis and severity. By bringing together a computational neuroscientist expert in image processing and behavioral analysis, a clinician-scientist expert in the early development and assessment of autism, and an expert in infant motor development and infant sibling studies for ASD, this work will provide an unbiased and scalable approach to quantifying and categorizing ASD motor function deficits across development, thus critically facilitating high quality and accessible early ASD diagnosis.