Research

Due to complex post-processing used by photo sharing platforms, passive authentication of photographs shared online is notoriously unreliable. This project explores modeling the entire workflow from the sensor to the browser, and end-to-end optimization of photo acquisition and distribution channels to facilitate reliable authentication at the receiver. We replace a digital camera with a neural network model, and optimize photo manipulation detection capabilities. The obtained results demonstrate that significant improvements in authentication performance can be obtained with only minor cost in image rendering fidelity. Optimization of lossy compression also shows that fractional increase in bit-rates are enough to obtain good manipulation detection, even at extremely low bitrates.

Acquisition of a photograph with a digital camera introduces many characteristic fingerprints that might be used as forensic features to address questions regarding the photo's origin, integrity, or processing history. By analyzing these features for various regions of the photograph, it becomes possible to detect anomalies and indicate which fragments of the image were affected by malicious tampering. However, reliable extraction of forensic features requires to analyze large image regions which is clearly contradictory with the desired high resolution of the resulting tampering map. My postdoctoral research focuses on the techniques for improving tampering localization capabilities by adopting multi-scale analysis.

In contrast to passive forensic methods that analyze intrinsic fingerprints, active protection methods involve preparation of smart images that are aware of their content. The information about the original appearance is encoded, along with its corresponding signatures, and embedded into the image itself as a digital watermark. Once the photograph is tampered with, a dedicated decoder can analyze the watermark and: a) precisely indicate the regions affected by malicious tampering; b) approximately recovery the original content of the tampered regions. My research focused on the analysis of this problem as an erasure communication channel. I show how digital fountain codes can be conveniently used to construct efficient self-recovery schemes with the efficiency reaching the theoretical capacity limits.

I've also collaborated on other research topics, including:

• error correction coding based on correction trees
• generation of image resembling 2-D bar-codes
• audio forensics
• advanced photo carving
• adversarial attacks on speaker verification systems