Method Details

Details for method 'EfficientPS [Mapillary Vistas]'


Method overview

name EfficientPS [Mapillary Vistas]
challenge instance-level semantic labeling
details Understanding the scene in which an autonomous robot operates is critical for its competent functioning. Such scene comprehension necessitates recognizing instances of traffic participants along with general scene semantics which can be effectively addressed by the panoptic segmentation task. In this paper, we introduce the Efficient Panoptic Segmentation (EfficientPS) architecture that consists of a shared backbone which efficiently encodes and fuses semantically rich multi-scale features. We incorporate a new semantic head that aggregates fine and contextual features coherently and a new variant of Mask R-CNN as the instance head. We also propose a novel panoptic fusion module that congruously integrates the output logits from both the heads of our EfficientPS architecture to yield the final panoptic segmentation output. Additionally, we introduce the KITTI panoptic segmentation dataset that contains panoptic annotations for the popularly challenging KITTI benchmark. Extensive evaluations on Cityscapes, KITTI, Mapillary Vistas and Indian Driving Dataset demonstrate that our proposed architecture consistently sets the new state-of-the-art on all these four benchmarks while being the most efficient and fast panoptic segmentation architecture to date.
publication EfficientPS: Efficient Panoptic Segmentation
Rohit Mohan, Abhinav Valada
project page / code
used Cityscapes data fine annotations
used external data ImageNet, Mapillary Vistas Research Edition
runtime n/a
subsampling no
submission date July, 2020
previous submissions 1


Average results

Metric Value
AP 39.7769
AP50% 64.8987
AP100m 52.8581
AP50m 55.7869


Class results

Class AP AP50% AP100m AP50m
person 43.0509 74.4185 58.4981 58.0978
rider 34.8495 69.7314 48.5332 48.9906
car 58.9859 79.9747 77.0966 78.594
truck 38.0948 48.8148 48.4854 55.5971
bus 49.6099 64.8736 69.3301 78.9983
train 38.9409 64.3333 48.3361 54.054
motorcycle 29.0003 57.9879 36.2814 36.0653
bicycle 25.6833 59.055 36.3042 35.8984



Download results as .csv file

Benchmark page