Automated video identification of marine species (AVIMS) - new application: report

Introduction

This report describes the system called the Automated Video Identification of Marine Species (AVIMS) that was developed by the authors as part of the Scottish Government contract, Ref: CASE/216380.

The objective of this project was to develop an automated video analysis application with a user-friendly interface which could be used by Marine Directorate biologists and stakeholders’ non-specialist staff, without the need for coding/computer science expertise. The requirement was that only open-source algorithms are to be used.

The project was meant to build upon a previous project funded by the Scottish Government - Automated Identification of Fish and Other Aquatic Life in Underwater Video (Blowers et al., 2020), which reviewed and recommended various approaches to automated image analysis in underwater video. The immediate aim of this project was to produce a software tool to allow the Marine Directorate to make cost-effective use of the video data which is used to underpin scientific advice for Ministers.

Marine Directorate collects a large variety of underwater video for a number of different purposes. The data comes from cameras which can be towed (e.g. images of the seabed) or fixed (e.g. attached to drop-frames or trawl nets, or deployed at underwater turbines or in-river fish counters).

Analyses of these video data is time-consuming, often requires a skilled taxonomist and hence constitutes a significant draw on resources. The high cost of the analyses of this large amount of data often results in situations where only a subset of the available video data can be fully analysed. Consequently, an automated video analysis software performing the above tasks would be highly desirable as it would reduce the costs of the existing activities and allow for the analysis of all available data. It is expected that due to a steady and fast improvements in the new sensor/camera technology and their decreasing costs, the amount of video data available will only increase making the current processing bottlenecks even more acute.

To achieve that goal, the Scottish Government funded an earlier piece of work in this area (Blowers et al., 2020) where the authors reviewed current image and video analysis methods and how these can be applied to different types of video footage and data extraction requirements used by the Marine Directorate. The authors also made recommendations for how video analysis could be automated using open-source machine learning algorithms. Our work builds on some of the recommendations from that work as well as on our own expertise in the field of computer vision and software development.

The following three sections describe the Methodology and Design of the AVIMS web application, the computer vision approach we have chosen, the hardware and software specification, the datasets and experiments, and the other resources.