Method Details
Details for method 'Seamless Scene Segmentation'
Method overview
| name | Seamless Scene Segmentation |
| challenge | panoptic semantic labeling |
| details | Seamless Scene Segmentation is a CNN-based architecture that can be trained end-to-end to predict a complete class- and instance-specific labeling for each pixel in an image. To tackle this task, also known as "Panoptic Segmentation", we take advantage of a novel segmentation head that seamlessly integrates multi-scale features generated by a Feature Pyramid Network with contextual information conveyed by a light-weight DeepLab-like module. In this submission we use a single model, with a ResNet50 backbone, pre-trained on ImageNet and Mapillary Vistas Research Edition, and fine-tuned on Cityscapes' fine training set. Inference is single-shot, without any form of test-time augmentation. Validation scores of the submitted model are 64.97 PQ, 68.04 PQ stuff, 60.75 PQ thing, 80.73 IoU. |
| publication | Seamless Scene Segmentation Lorenzo Porzi, Samuel Rota Bulò, Aleksander Colovic and Peter Kontschieder The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019 https://research.mapillary.com/publication/cvpr19a |
| project page / code | https://github.com/mapillary/seamseg |
| used Cityscapes data | fine annotations |
| used external data | ImageNet, Mapillary Vistas Research Edition |
| runtime | n/a |
| subsampling | no |
| submission date | August, 2019 |
| previous submissions |
Average results
| Metric | All | Things | Stuff |
|---|---|---|---|
| PQ | 62.6441 | 55.9671 | 67.5 |
| SQ | 82.1406 | 80.305 | 83.4756 |
| RQ | 75.2881 | 69.6455 | 79.3918 |
Class results
| Class | PQ | SQ | RQ |
|---|---|---|---|
| road | 98.3025 | 98.4323 | 99.8682 |
| sidewalk | 76.8192 | 85.1156 | 90.2527 |
| building | 88.7939 | 90.9923 | 97.5839 |
| wall | 36.6138 | 76.3839 | 47.9339 |
| fence | 39.7462 | 75.0098 | 52.988 |
| pole | 59.0483 | 70.4056 | 83.8687 |
| traffic light | 51.5952 | 77.3387 | 66.7133 |
| traffic sign | 65.7409 | 80.5773 | 81.5873 |
| vegetation | 90.4431 | 91.8938 | 98.4214 |
| terrain | 45.8168 | 78.5129 | 58.3558 |
| sky | 89.5807 | 93.5698 | 95.7367 |
| person | 57.7226 | 78.5191 | 73.5141 |
| rider | 53.4809 | 74.8192 | 71.4801 |
| car | 68.8706 | 84.9135 | 81.1068 |
| truck | 52.6344 | 87.3801 | 60.2362 |
| bus | 62.1598 | 86.2938 | 72.0327 |
| train | 54.6632 | 81.001 | 67.4847 |
| motorcycle | 51.1558 | 76.3016 | 67.0442 |
| bicycle | 47.0497 | 73.2119 | 64.265 |