Research

My main line of research addresses problems at the intersection of multimedia and security, with particular focus on verification and assurance of media integrity. This interdisciplinary field combines concepts from computer vision, image processing, image quality assessment, machine learning, statistical signal processing, and communication & information theory. If you’d like to know more about the field, feel free to check out my recent survey.

I have worked both on active protection techniques (e.g., authentication watermarks for controlled-capture workflows) and on passive forensic analysis (e.g., using camera fingerprints or compression traces). Specific research projects are discussed below.

Due to complex post-processing used by photo sharing platforms, passive authentication of photographs shared online is notoriously unreliable. This project explores modeling the entire workflow from the sensor to the browser, and end-to-end optimization of photo acquisition and distribution channels to facilitate reliable authentication at the receiver. We replace a digital camera with a neural network model, and optimize photo manipulation detection capabilities. The obtained results demonstrate that significant improvements in authentication performance can be obtained with only minor cost in image rendering fidelity. Optimization of lossy compression also shows that fractional increase in bit-rates are enough to obtain good manipulation detection, even at extremely low bitrates.

Read more

Acquisition of a photograph with a digital camera introduces many characteristic fingerprints that might be used as forensic features to address questions regarding the photo's origin, integrity, or processing history. By analyzing these features for various regions of the photograph, it becomes possible to detect anomalies and indicate which fragments of the image were affected by malicious tampering. However, reliable extraction of forensic features requires to analyze large image regions which is clearly contradictory with the desired high resolution of the resulting tampering map. My postdoctoral research focuses on the techniques for improving tampering localization capabilities by adopting multi-scale analysis.

Read more

In contrast to passive forensic methods that analyze intrinsic fingerprints, active protection methods involve preparation of smart images that are aware of their content. The information about the original appearance is encoded, along with its corresponding signatures, and embedded into the image itself as a digital watermark. Once the photograph is tampered with, a dedicated decoder can analyze the watermark and: a) precisely indicate the regions affected by malicious tampering; b) approximately recovery the original content of the tampered regions. My research focused on the analysis of this problem as an erasure communication channel. I show how digital fountain codes can be conveniently used to construct efficient self-recovery schemes with the efficiency reaching the theoretical capacity limits.

Read more