Quantification of visual blockage at culverts using deep learning based computer vision models

Blockage of culverts by debris material is reported as main cause of urban flash floods. Extraction of blockage information using intelligent video analytic (IVA) algorithms can prove helpful in making timely maintenance-related decisions toward avoiding flash floods. Having known the percentage of visual blockage at culverts can help better prioritise the maintenance of highly blocked culvert sites. This article proposes a deep learning-based segmentation-classification pipeline where visible culvert openings are segmented at the first stage and classified into one of four percentage visual blockage classes at the second stage. Images of Culverts and Blockage (ICOB) and Visual Hydraulics-Lab Blockage Dataset (VHD) dataset were used to train the deep learning models. From the results, Mask R-CNN (ResNet50 backbone) achieved the best segmentation performance (i.e. mAP@75 of 77.2%), while NASNet achieved the best classification performance (i.e. 81.2% test accuracy). To demonstrate the implication, a potential visual blockage monitoring use-case has been proposed.