FRNN: Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes

Description

This paper proposes a segmentic segmentation method which is a combination of a U-Net and a ResNet. The network has two processing streams. The residual stream (blue) stays at the full image resolution, the pooling stream (red) undergoes a sequence of pooling and unpooling operations. The two processing streams are coupled using full-resolution residual units (FRRUs).

While RU stands for a normal residual unit as in ResNet, the FRRU has two streams : the residual stream, are computed by adding successive residuals, while the features on the other stream, the pooling stream, are the direct result of a sequence of convolution and pooling operations applied to the input.

The overall network can be summarized as follows:

Loss

The authors implemented an interesting “bootstrapped cross-entropy loss” which focuses only on the K most difficult pixels.

Results

NOTE : SegNet is very similar to a Unet.