Using Mask R-CNN for Underwater Fish Instance Segmentation as Novel Objects: A Proof of Concept

Instance Segmentation in general deals with detecting, segmenting and classifying individual instances of objects in an image. Underwater instance segmentation methods often involve aquatic animals like fish as the things to be detected. In order to train deep learning models for instance segmentation in an underwater environment, rigorous human annotation in form of instance segmentation masks with labels is usually required, since the aquatic environment poses challenges due to dynamic background patterns and optical distortions. However, annotating instance segmentation masks on images is especially time- and cost-intensive compared to classification tasks. Here we show an unsupervised instance learning and segmentation approach that introduces a novel class, e.g., ""fish"" to a pre-trained Mask R-CNN model using its own detection and segmentation capabilities in underwater images. Our results demonstrate a robust detection and segmentation of underwater fish in aquaculture without the need for human annotations. This proof of concept shows that there is room for novel objects within trained instance segmentation models in the paradigm of supervised learning.