Method Details

Details for method 'EfficientPS [Mapillary Vistas]'


Method overview

name EfficientPS [Mapillary Vistas]
challenge instance-level semantic labeling
details Understanding the scene in which an autonomous robot operates is critical for its competent functioning. Such scene comprehension necessitates recognizing instances of traffic participants along with general scene semantics which can be effectively addressed by the panoptic segmentation task. In this paper, we introduce the Efficient Panoptic Segmentation (EfficientPS) architecture that consists of a shared backbone which efficiently encodes and fuses semantically rich multi-scale features. We incorporate a new semantic head that aggregates fine and contextual features coherently and a new variant of Mask R-CNN as the instance head. We also propose a novel panoptic fusion module that congruously integrates the output logits from both the heads of our EfficientPS architecture to yield the final panoptic segmentation output. Additionally, we introduce the KITTI panoptic segmentation dataset that contains panoptic annotations for the popularly challenging KITTI benchmark. Extensive evaluations on Cityscapes, KITTI, Mapillary Vistas and Indian Driving Dataset demonstrate that our proposed architecture consistently sets the new state-of-the-art on all these four benchmarks while being the most efficient and fast panoptic segmentation architecture to date.
publication EfficientPS: Efficient Panoptic Segmentation
Rohit Mohan, Abhinav Valada
project page / code
used Cityscapes data fine annotations
used external data ImageNet, Mapillary Vistas Research Edition
runtime n/a
subsampling no
submission date May, 2020
previous submissions


Average results

Metric Value
AP 39.1378
AP50% 63.0663
AP100m 52.4625
AP50m 55.3713


Class results

Class AP AP50% AP100m AP50m
person 42.3416 72.4521 58.0205 57.6497
rider 35.125 69.9796 48.7935 49.3166
car 59.0894 79.6709 77.4073 78.9209
truck 38.3288 48.2539 49.048 55.4332
bus 45.5071 58.16 64.7661 73.7616
train 37.5246 58.8052 48.8088 55.5707
motorcycle 29.5145 59.2139 36.2891 36.1795
bicycle 25.671 57.995 36.5669 36.1382



Download results as .csv file

Benchmark page