Neural Imaging - Modeling & Optimization of Photo Acquisition and Distribution

Due to complex post-processing used by photo sharing platforms, passive authentication of photographs shared online is notoriously unreliable. This project explores modeling the entire workflow from the sensor to the browser, and end-to-end optimization of photo acquisition and distribution channels to facilitate reliable authentication at the receiver. We replace a digital camera with a neural network model, and optimize photo manipulation detection capabilities. The obtained results demonstrate that significant improvements in authentication performance can be obtained with only minor cost in image rendering fidelity. Optimization of lossy compression also shows that fractional increase in bit-rates are enough to obtain good manipulation detection, even at extremely low bitrates.

Downloads

• A neural imaging toolbox for modeling and optimization of complex photo acquisition and dissemination channels.
• A lightweight learned lossy image codec (l3ic) with good rate-distortion performance (comparable to BPG) and fast runtime on modern GPU-enabled platforms.