Analyzing Punch

The Motivation of Our Punch Project

Our goal is to democratize animal behavior analysis using AI-powered tools. While casual observation of Punch's videos offers intuitive impressions of his behavior, these impressions are not supported by quantitative data. Generating such data requires systematic behavioral analysis of video recordings, which is not only time-consuming but also technically demanding.

The open-source, AI-powered tool LabGym is well suited to address this challenge due to its flexibility and accessibility. We therefore set out to train AI models using LabGym to analyze Punch's behaviors from publicly available videos recorded by zoo visitors on their cell phones.

This project also presents unique technical challenges. Rather than a single continuous recording, the available data consists of a large collection of short, disconnected clips taken at different times, from different cell phones, and by different zoo visitors. The framing shifts, the camera moves, the quality varies, and many clips only capture part of what is happening. We developed approaches to address these challenges.

Even with that variability, certain behaviors appear again and again. While not identical from clip to clip, these behaviors are consistently recognizable. Once that becomes clear, the footage can be organized, compared, and labeled in a way that holds up across clips, even without controlled conditions.

Punch's case is intriguing in that sense. Although the videos were not collected for scientific analysis, they nevertheless support it. The same openness that allowed these videos to spread widely also makes them accessible as data, which can be used by LabGym to help us better understand our friend Punch.