Application of a computer vision technique to animal-borne video data: extraction of head movement to understand sea turtles’visual assessment of surroundings

Abstract

[Background] An animal-borne video recording system has recently been developed to study the behavior of free-ranging animals. In contrast to other types of sensor data (i.e., acceleration), video images offer the advantage of directly acquiring information without analysis. However, most previous findings have only been obtained through visual observation of image data. Here, we demonstrate a new method of data analysis for animal-borne videos using a computer vision technique referred to as template matching. As a case study, we tracked the horizontal head movements of green turtles (Chelonia mydas) to investigate how they move their heads to look around the underwater environment. [Results] Template matching allowed tracking of head movements with high accuracy (0.34 ± 0.12 % and 0.52 ± 0.29 % of the root-mean-square error on the x- and y-coordinates, respectively), high true (86.2 ± 8.1 %), and low false extraction rates (6.6 ± 8.4 %). However the program sometimes failed because the turtle’s head would move out of range of the video. During cruising swimming, green turtles did not significantly move their heads to one side, moving with a ratio of 50.5:49.5 (left: right). Green turtles moved their heads from side to side more widely and more slowly before (12.0 ± 4.6 point and 0.25 ± 0.03 Hz, respectively) and after taking a breath (27.5 ± 2.9 point and 0.27 ± 0.03 Hz) compared to during cruising swimming (8.4 ± 3.8 point and 0.32 ± 0.01 Hz). Before feeding, turtles moved their heads slowly (0.23 ± 0.03 Hz) and narrowly (9.3 ± 3.6 point). Our combined approach using video and gyro loggers revealed that when making a turn, turtles always turned their heads to the side 1.38 ± 0.77 s before turning their body. [Conclusions] Our method enables researchers to quantitatively extract information regarding vision cognition and behavioral responses in green turtles in the wild that could not otherwise be obtained from other sensors used in previous studies. This new method using a combination of computer vision and bio-logging (e.g., gyroscope) can serve as a powerful tool in animal behavior and ecological studies.