Method Details

Details for method 'Naive-Student (iterative semi-supervised learning with Panoptic-DeepLab)'


Method overview

name Naive-Student (iterative semi-supervised learning with Panoptic-DeepLab)
challenge panoptic semantic labeling
details Supervised learning in large discriminative models is a mainstay for modern computer vision. Such an approach necessitates investing in large-scale human-annotated datasets for achieving state-of-the-art results. In turn, the efficacy of supervised learning may be limited by the size of the human annotated dataset. This limitation is particularly notable for image segmentation tasks, where the expense of human annotation is especially large, yet large amounts of unlabeled data may exist. In this work, we ask if we may leverage semi-supervised learning in unlabeled video sequences to improve the performance on urban scene segmentation, simultaneously tackling semantic, instance, and panoptic segmentation. The goal of this work is to avoid the construction of sophisticated, learned architectures specific to label propagation (e.g., patch matching and optical flow). Instead, we simply predict pseudo-labels for the unlabeled data and train subsequent models with both human-annotated and pseudo-labeled data. The procedure is iterated for several times. As a result, our Naive-Student model, trained with such simple yet effective iterative semi-supervised learning, attains state-of-the-art results at all three Cityscapes benchmarks, reaching the performance of 67.8% PQ, 42.6% AP, and 85.2% mIOU on the test set. We view this work as a notable step towards building a simple procedure to harness unlabeled video sequences to surpass state-of-the-art performance on core computer vision tasks.
publication Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
Liang-Chieh Chen, Raphael Gontijo Lopes, Bowen Cheng, Maxwell D. Collins, Ekin D. Cubuk, Barret Zoph, Hartwig Adam, Jonathon Shlens
project page / code
used Cityscapes data fine annotations, video
used external data ImageNet, Mapillary Vistas Research Edition. Cityscapes train-extra set (coarse labels are not used but only images).
runtime n/a
subsampling no
submission date March, 2020
previous submissions


Average results

Metric AllThingsStuff
PQ 67.8056 61.5178 72.3785
SQ 83.7948 81.6131 85.3814
RQ 80.1976 75.3033 83.757


Class results

Class PQ SQ RQ
road 98.6487 98.779 99.8682
sidewalk 80.5 86.6708 92.8801
building 90.3251 92.0823 98.0917
wall 45.2387 79.1241 57.1744
fence 47.741 78.4639 60.8445
pole 68.7575 74.6538 92.1017
traffic light 59.5648 80.1426 74.3234
traffic sign 74.0997 83.4704 88.7736
vegetation 91.9029 92.471 99.3857
terrain 48.529 79.571 60.9883
sky 90.8562 93.7669 96.8958
person 60.2392 78.5947 76.6454
rider 58.0262 75.1227 77.2419
car 72.2248 85.4989 84.4745
truck 59.9812 87.9724 68.1818
bus 69.2143 88.3393 78.3505
train 64.5398 85.8707 75.1592
motorcycle 56.6085 77.6532 72.8992
bicycle 51.3086 73.8532 69.4737



Download results as .csv file

Benchmark page