Imagenet performance correlates with pose estimation robustness and generalization on out-of-domain data

Abstract

Neural networks are highly effective tools for pose estimation. However, robustness to outof-domain data remains a challenge, especially for small training sets that are common for realworld applications. Here, we probe the generalization ability with three architecture classes (MobileNetV2s, ResNets, and EfficientNets). We developed a novel dataset of 30 horses that allowed for both “within-domain” and “out-of-domain”(unseen horse) benchmarking-this is a crucial test for robustness that current human pose estimation benchmarks do not directly address. We show that better ImageNet-performing architectures perform better on both within-and out-of-domain data if they are first pretrained on ImageNet. Our results demonstrate that transfer learning is beneficial for out-of-domain robustness.Pose estimation is an important tool for measuring behavior, and thus widely used in technology, medicine and biology (Ostrek et al., 2019; Maceira-Elvira et al., 2019; Mathis & Mathis, 2020). Due to innovations in both deep learning algorithms (Insafutdinov et al., 2017; Cao et al., 2017; He et al., 2017; Kreiss et al., 2019; Ning et al., 2020; Cheng et al., 2020) and large-scale datasets (Lin et al., 2014; Andriluka et al., 2014; 2018) pose estimation on humans has gotten very powerful. However, typical human pose estimation benchmarks, such as MPII pose and COCO (Lin et al., 2014; Andriluka et al., 2014; 2018), contain many different individuals (> 10k) in different contexts, but only very few example postures per individual. In real world application of pose estimation, users want to estimate the location of userdefined bodyparts by only labeling a few hundred …