Method Details

Details for method 'Mapillary Research: In-Place Activated BatchNorm'

name	Mapillary Research: In-Place Activated BatchNorm
challenge	pixel-level semantic labeling
details	In-Place Activated Batch Normalization (InPlace-ABN) is a novel approach to drastically reduce the training memory footprint of modern deep neural networks in a computationally efficient way. Our solution substitutes the conventionally used succession of BatchNorm + Activation layers with a single plugin layer, hence avoiding invasive framework surgery while providing straightforward applicability for existing deep learning frameworks. We obtain memory savings of up to 50% by dropping intermediate results and by recovering required information during the backward pass through the inversion of stored forward results, with only minor increase (0.8-2%) in computation time. Test results are obtained using a single model.
publication	In-Place Activated BatchNorm for Memory-Optimized Training of DNNs Samuel Rota Bulò, Lorenzo Porzi, Peter Kontschieder arXiv http://research.mapillary.com/publications/arXive17.html
project page / code	https://github.com/mapillary/inplace_abn
used Cityscapes data	fine annotations, coarse annotations
used external data	ImageNet, Mapillary Vistas Research
runtime	n/a
subsampling	no
submission date	January, 2018
previous submissions	1