Automated video identification of marine species (AVIMS) - new application: report

Object tracking

We noted the challenges posed by the requirement of accurate object tracking. Blowers, Evans and McNally (2020) found that inaccuracies in this component resulted in overcounting. We also noted the suggestion that the DeepSORT algorithm might provide a good solution to our object tracking problem.

Accurate object tracking is dependent on accurate input from the object detector (Bewley et al. 2016); a prior stage in the inference pipeline. Inaccurate predictions from the detector are likely to negatively impact tracking performance. Furthermore, we note that the DeepSORT algorithm uses an association metric model for the purpose of object re-identification. It is used to estimate the likelihood that detections from different frames represent the same object. This model is trained using an annotated object tracking dataset. An object tracking dataset consists of annotated videos rather than single images and that each object must be annotated in every frame in which it can be seen. We note that annotating all frames in a video substantially increases the amount of manual effort required. While it is possible that the annotation tool forming the part of our web application could be extended for this purpose, the human effort required to annotate enough videos to train a sufficiently accurate model can make this approach practically infeasible. This is the reason why we have chosen the SORT algorithm (Bewley et al., 2016) for our object tracker; effectively DeepSORT without the association metric model. SORT takes the predictions of our object detection model as input and joins the per-frame predictions into object tracks. We use our own implementation of SORT. We use the Kalman filter implementation provided by the filterpy library.