Balanced L1 Loss: A Comprehensive Overview
In the field of machine learning, one of the major tasks is object detection. Object detection is identifying the location and type of objects within an image. To solve these classification and localization problems simultaneously, a loss function called Balanced L1 Loss is used. This loss function is a modified version of the Smooth L1 loss designed for object detection tasks.
The Objective Function
The objective function of Balanced L1 loss is defined as:
$$ L\_{p,u,t\_{u},v} = L\_{cls}\left(p, u\right) + \lambda\left[u \geq 1\right]L\_{loc}\left(t^{u}, v\right) $$
The equation has two parts: $L\_{cls}$ and $L\_{loc}$. The former denotes the objective function for classification, while the latter denotes the objective function for the localization of the object within an image. The predictions and targets in $L\_{cls}$ are denoted as $p$ and $u$, respectively. $t\_{u}$ is the corresponding regression result for the class $u$, while $v$ is the regression target. The value of $\lambda$ is used to tune the loss weight under multi-task learning.
Inliers and Outliers
The samples with a loss greater than or equal to 1.0 are called outliers, while the other samples are called inliers. A natural solution for balancing the involved tasks is to tune the loss weights of the objective functions, but directly raising the weight of the localization objective function will make the model more sensitive to outliers. These outliers, which can be regarded as hard samples, will produce excessively large gradients that are harmful to the training process. The inliers, which can be regarded as the easy samples, contribute less gradient to the overall gradient compared with outliers. To overcome these issues, the authors introduced the balanced L1 loss.
The Idea Behind Balanced L1 loss
The balanced L1 loss is derived from the conventional smooth L1 loss. In smooth L1 loss, an inflection point is set to separate inliers from outliers, and large gradients produced by outliers are clipped with a maximum value of 1. The key idea of balanced L1 loss is to promote the crucial regression gradients, i.e., gradients from inliers (accurate samples), to rebalance the involved samples and tasks, thus achieving a more balanced training within classification, overall localization, and accurate localization.
The localization loss $L\_{loc}$ using balanced L1 loss can be defined as:
$$ L\_{loc} = \sum\_{i\in{x,y,w,h}}L\_{b}\left(t^{u}\_{i}-v\_{i}\right) $$
The Figure below shows that balanced L1 loss increases the gradients of inliers under the control of a factor denoted as $\alpha$. A small $\alpha$ increases more gradient for inliers, but the gradients of outliers are not influenced. Furthermore, an overall promotion magnification controlled by $\gamma$ is also brought in for tuning the upper bound of regression errors, which can help the objective function better balance involved tasks. The two factors that control different aspects are mutually enhanced to achieve balanced training. $b$ is used to ensure $L\_{b}\left(x=1\right)$ has the same value for both formulations in the equation below:
By integrating the gradient formulation above, we can get the balanced L1 loss as:
$$ L\_{b}\left(x\right) = \frac{\alpha}{b}\left(b|x| + 1\right)ln\left(b|x| + 1\right) - \alpha|x| \text{ if } |x| < 1$$ $$ L\_{b}\left(x\right) = \gamma|x| + C \text{ otherwise } $$
The parameters $\gamma$, $\alpha$, and $b$ are constrained by $\alpha\text{ln}\left(b + 1\right) = \gamma$. The default parameters are set as $\alpha=0.5$ and $\gamma=1.5$.
Advantages of Balanced L1 Loss
The following are some of the advantages of using the balanced L1 loss function:
- It balances the localization and classification tasks.
- It promotes the crucial regression gradients of accurate samples.
- It better handles hard outliers during training.
- It preserves the computational efficiency.
Object detection is a crucial task in the field of machine learning, and to solve this task, the designers use a loss function, Balanced L1 Loss. This loss function is used for the simultaneous recognition and localization tasks. The balanced L1 loss function brings in the promotion magnification controlled by $\gamma$, and a factor denoted as $\alpha$ to promote the gradients of accurate targets while maintaining the efficiency of the model. Balanced L1 Loss helps in balancing the involved tasks, handling the outliers during training, and improving the performance of the model.
