Quantifying the Cost of Reliable Photo Authentication via High-Performance Learned Lossy Representations

Paweł Korus, Nasir Memon

International Conference on Learning Representations (ICLR), 2020




Detection of photo manipulation relies on subtle statistical traces, notoriously removed by aggressive lossy compression employed online. We demonstrate that end-to-end modeling of complex photo dissemination channels allows for codec optimization with explicit provenance objectives. We design a lightweight trainable lossy image codec, that delivers competitive rate-distortion performance, on par with best hand-engineered alternatives, but has lower computational footprint on modern GPU-enabled platforms. Our results show that significant improvements in manipulation detection accuracy are possible at fractional costs in bandwidth/storage. Our codec improved the accuracy from 37% to 86% even at very low bit-rates, well below the practicality of JPEG (QF 20).

Source Code and Deployed Models

This paper comes with two separate git repositories:

  • neural imaging toolbox contains the source code and scripts needed to construct, train and test the lossy image codec.
  • l3ic codec contains a deployed version of the codec.