Introduction

This is my “best paper” of the CVPR 2019 edition. It revisits the notion of robust regression loss function \(\rho(.)\),

\[\arg \min_\theta \rho \left( f_\theta(x)-y \right )\]

as it proposes a new general loss function which is a superset of many existing losses. The parameters of this loss can be automatically adapted to fit the true nature of the problem.

Methods

\(\rho(.)\) can be an L2 or an L1 function. However, it can also be a robust function like a Welsch loss, a Geman-McClure loss, or a Cauchy loss. The author shows that all these functions are special cases of a more general fobust function

where \(\alpha\) is a shape parameter. By playing with \(\alpha\), one can recover the various loss functions mentioned before.

Although \(\alpha\) can be seen as an extra hyperparameter that needs to be tuned, one can define it as a trainable parameter. To do so, the author encapsulates the loss into a PDF formulation:

where

so the optimization function becomes

\[\arg \min_{\theta,\alpha} -log(p(x|\alpha) = \rho \left( x,\alpha \right ) + log(Z(\alpha))\]

where \(log(Z(\alpha))\) is some complicated analytical function which the author approximates with a cubic spline function.

Having \(Z(.)\) in the loss function is important because without it, the system would trivially minimize the cost of outliers by setting \(\alpha\) to be as small as possible. However, as shown in Fig.2, reducing \(\alpha\) will decrease the loss of outlisers but will increase the loss of inliers.

Results

Results are quite convincing on various applications like VAEs

and monocular depth estimation

Video

The author recorded a nice video. Don’t miss it!