Method Details

Details for method 'EfficientPS [Mapillary Vistas]'


Method overview

name EfficientPS [Mapillary Vistas]
challenge pixel-level semantic labeling
details Understanding the scene in which an autonomous robot operates is critical for its competent functioning. Such scene comprehension necessitates recognizing instances of traffic participants along with general scene semantics which can be effectively addressed by the panoptic segmentation task. In this paper, we introduce the Efficient Panoptic Segmentation (EfficientPS) architecture that consists of a shared backbone which efficiently encodes and fuses semantically rich multi-scale features. We incorporate a new semantic head that aggregates fine and contextual features coherently and a new variant of Mask R-CNN as the instance head. We also propose a novel panoptic fusion module that congruously integrates the output logits from both the heads of our EfficientPS architecture to yield the final panoptic segmentation output. Additionally, we introduce the KITTI panoptic segmentation dataset that contains panoptic annotations for the popularly challenging KITTI benchmark. Extensive evaluations on Cityscapes, KITTI, Mapillary Vistas and Indian Driving Dataset demonstrate that our proposed architecture consistently sets the new state-of-the-art on all these four benchmarks while being the most efficient and fast panoptic segmentation architecture to date.
publication EfficientPS: Efficient Panoptic Segmentation
Rohit Mohan, Abhinav Valada
project page / code
used Cityscapes data fine annotations
used external data ImageNet, Mapillary Vistas Research Edition
runtime n/a
subsampling no
submission date May, 2020
previous submissions


Average results

Metric Value
IoU Classes 84.2432
iIoU Classes 65.2168
IoU Categories 92.4617
iIoU Categories 83.5092


Class results

Class IoU iIoU
road 98.8372 -
sidewalk 88.1875 -
building 94.3341 -
wall 67.6418 -
fence 67.7056 -
pole 73.4296 -
traffic light 80.1669 -
traffic sign 83.2956 -
vegetation 94.2844 -
terrain 74.3772 -
sky 96.008 -
person 88.6634 75.6988
rider 75.3179 56.6791
car 96.6263 92.0156
truck 83.4653 50.037
bus 94.0368 64.3635
train 91.0837 61.4976
motorcycle 73.504 53.8876
bicycle 79.6556 67.5555


Category results

Category IoU iIoU
flat 98.7961 -
nature 93.944 -
object 78.8101 -
sky 96.008 -
construction 94.5531 -
human 88.8387 76.7631
vehicle 96.2819 90.2552



Download results as .csv file

Benchmark page