Method Details

Details for method 'Multitask Learning'


Method overview

name Multitask Learning
challenge pixel-level semantic labeling
details Numerous deep learning applications benefit from multi-task learning with multiple regression and classification objectives. In this paper we make the observation that the performance of such systems is strongly dependent on the relative weighting between each task's loss. Tuning these weights by hand is a difficult and expensive process, making multi-task learning prohibitive in practice. We propose a principled approach to multi-task deep learning which weighs multiple loss functions by considering the homoscedastic uncertainty of each task. This allows us to simultaneously learn various quantities with different units or scales in both classification and regression settings. We demonstrate our model learning per-pixel depth regression, semantic and instance segmentation from a monocular input image. Perhaps surprisingly, we show our model can learn multi-task weightings and outperform separate models trained individually on each task.
publication Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Alex Kendall, Yarin Gal and Roberto Cipolla
project page / code
used Cityscapes data fine annotations
used external data ImageNet
runtime n/a
subsampling no
submission date November, 2017
previous submissions


Average results

Metric Value
IoU Classes 78.5439
iIoU Classes 57.4403
IoU Categories 89.9465
iIoU Categories 77.7065


Class results

Class IoU iIoU
road 98.4288 -
sidewalk 85.2084 -
building 92.7857 -
wall 54.1607 -
fence 60.7812 -
pole 62.4002 -
traffic light 73.3596 -
traffic sign 77.5348 -
vegetation 93.3098 -
terrain 71.4644 -
sky 95.0648 -
person 84.9143 66.9346
rider 69.5413 49.3073
car 95.3118 89.3979
truck 68.4814 39.9683
bus 86.1879 50.8117
train 80.0181 54.066
motorcycle 67.7836 47.2341
bicycle 75.5978 61.8025


Category results

Category IoU iIoU
flat 98.505 -
nature 92.9828 -
object 69.8465 -
sky 95.0648 -
construction 93.1928 -
human 85.2916 68.0424
vehicle 94.742 87.3706



Download results as .csv file

Benchmark page