Hole-in-the-wall: Perception of 3D shape and affordances from static images in humans and machines

Abstract

One popular toy for toddlers involves sorting block shapes into their respective holes. While toddlers require trial-and-error actions to sort blocks correctly, adults can rapidly see the appropriate solution through visual inspection alone. This feat requires an understanding of 3D shape and mental rotation. We study this task in a simplified vision-only setting by generating “shapes” of varying complexity using square matrices filled with connected binary regions, and “holes” by taking the negative region.“Fits” and “doesn’t fit” conditions are created while ensuring that shapes do not match exactly and that the total filled area is the same in both conditions. These matrices are rendered into black-and-white images (“bw”) and into more realistic rendered scenes. Human observers performed a single-interval fits/doesn’t fit task for two complexity levels for bw and rendered scenes. Performance was high for both bw (average …