Method Details

Details for method 'Adaptive Affinity Field on PSPNet'


Method overview

name Adaptive Affinity Field on PSPNet
challenge pixel-level semantic labeling
details Existing semantic segmentation methods mostly rely on per-pixel supervision, unable to capture structural regularity present in natural images. Instead of learning to enforce semantic labels on individual pixels, we propose to enforce affinity field patterns in individual pixel neighbourhoods, i.e., the semantic label patterns of whether neighbouring pixels are in the same segment should match between the prediction and the ground-truth. The affinity fields characterize geometric relationships within the image, such as "motorcycles have round wheels". We further develop a novel method for learning the optimal neighbourhood size for each semantic category, with an adversarial loss that optimizes over worst-case scenarios. Unlike the common Conditional Random Field (CRF) approaches, our adaptive affinity field (AAF) method has no extra parameters during inference, and is less sensitive to appearance changes in the image.
publication Adaptive Affinity Field for Semantic Segmentation
Tsung-Wei Ke*, Jyh-Jing Hwang*, Ziwei Liu, Stella X. Yu
ECCV 2018
project page / code
used Cityscapes data fine annotations
used external data ImageNet
runtime n/a
subsampling no
submission date May, 2018
previous submissions


Average results

Metric Value
IoU Classes 79.069
iIoU Classes 56.1287
IoU Categories 90.8236
iIoU Categories 78.477


Class results

Class IoU iIoU
road 98.5274 -
sidewalk 85.5567 -
building 93.0388 -
wall 53.8061 -
fence 58.9574 -
pole 65.9285 -
traffic light 75.015 -
traffic sign 78.415 -
vegetation 93.681 -
terrain 72.4427 -
sky 95.5842 -
person 86.4314 68.0611
rider 70.5064 48.4759
car 95.8813 89.9488
truck 73.9118 38.779
bus 82.6801 49.2535
train 76.8555 49.3955
motorcycle 68.6938 42.3452
bicycle 76.3982 62.7705


Category results

Category IoU iIoU
flat 98.7018 -
nature 93.3758 -
object 72.4589 -
sky 95.5842 -
construction 93.5006 -
human 86.8871 69.1361
vehicle 95.257 87.8178



Download results as .csv file

Benchmark page