Computer Vision: River Segmentation
about
Brooklyn Atlantis Citizen Science is a project headed by Jeff Laut, a PhD fellow at New York University, that aims to monitor the Brooklyn Gowanus canal using the help of citizen science. The team built an Aquatic Robotic Vehicle (ARV) that acquires pictures of the canal and the water temperature, conductivity, dissolved oxygen, and pH of the canal's water. The picture taken of the canal is later published in a public domain where the public can help by tagging any recognizable object in the picture.
To gather greater amount of data for analysis, the team seeks to develop an ambitious autonomous navigation system for the ARV using only the onboard monocular webcam as its perception and range sensor. To do so, I was tasked to develop a water segmentation algorithm that would identify canal shoreline using the onboard camera.
my role
Originally envisioned as a simple exercise of edge and line detection, the river shoreline identification task possesses significantly difficult due to the large variation in canal environment appearance. The reflection on the river would often cause false positives in geometric based approach.
To robustly segment canal water from the canal walls, I conducted extensive literature review and decided to implement a variant of the entropy and texture based segmentation algorithm outlined in the paper Water Detection with Segmentation Guided Dynamic Texture Recognition.
The Algorithm
The algorithm is outlined as follows: first the motion entropy of the water is extracted by tracking the feature movement, with the more complex moving features tagged as water and the less complex moving features as land. The tags are then propagated using texture cues.
The motion entropy is calculated by first randomly initialize 250 motion trackers and tracking them using Optical Flow algorithm. By tracking the path of the individual trackers, we can calculate the randomness/complexity of its motion. The trackers are then divided based on its path entropy, with highly entropic markers representing water.
The texture segmentation of the algorithm follows a two-stage process. The first stage is a K-means (k = 32) clustering based on appearance descriptors encompassing the Law’s Texture Energy Measure and the c1c2c3 shadow invariant color space of the image. For the second stage of segmentation process, the statistics of a larger area are learned and clustered based on the similarity using K-means (k=16) clustering method. Finally, the water tag obtained from the motion entropy is propagated using the clustered texture.
