In contrast to passive forensic methods that analyze intrinsic fingerprints, active protection methods involve preparation of smart images that are aware of their content. The information about the original appearance is encoded, along with its corresponding signatures, and embedded into the image itself as a digital watermark. Once the photograph is tampered with, a dedicated decoder can analyze the watermark and: a) precisely indicate the regions affected by malicious tampering; b) approximately recovery the original content of the tampered regions. My research focused on the analysis of this problem as an erasure communication channel. I show how digital fountain codes can be conveniently used to construct efficient self-recovery schemes with the efficiency reaching the theoretical capacity limits.
The content reconstruction capability, referred to as self-recovery, is an extension of digital image authentication schemes. In its essence, self-recovery schemes rely on the ability to communicate the appearance of the original image to a decoder through an unreliable channel - a potentially tampered digital image. Given that the original content authentication component already provides clues about the authenticity of individual image regions, the problem resembles erasure communication. However, there are important differences with respect to the traditional problem of sending arbitrary messages. A detailed analysis of this issue can be found in:
In certain applications, it might be beneficial to prioritize image content and guarantee high-quality reconstruction of important image fragments at the cost of unimportant background content. This adaptive self-recovery capability can be conveniently modeled in the proposed framework:
The proposed model of the self-recovery problem results in superior content reconstruction performance. A reference implementation of the proposed paradigm, based on digital fountain codes, achieves better results than various state-of-the-art alternatives described in the literature. For the sake of fair comparison in the same conditions, I have implemented 5 different algorithms, described in the following papers:
Watermarking With Flexible Self-Recovery Quality Based on Compressive Sensing and Compositive Reconstruction
-BReference Sharing Mechanism for Watermark Self-Embedding
Fragile Watermarking Scheme with Extensive Content Restoration Capability
Self-embedding watermark with flexible restoration quality
An example of content reconstruction result is shown below. The tampered region is located in the middle of the image, as shown in the top left sub-plot. More examples can be found in the supplementary materials for the original paper and my doctoral dissertation. For the sake of research reproducibility, the package also contains a Matlab implementation of the reference scheme (together with an iteractive demonstration with a graphical user interface).
In practical settings it becomes necessary to deal with high-resolution color images, often compressed with the lossy JPEG codec. The proposed framework can be easily extended to deal with such a scenario. The additional chrominance channels can be used to either provide content reconstruction capabilities for the color components, or to strengthen the protection (and thus the tolerable tampering rates) of the image's luminance. A detailed analysis of these considerations can be found in:
We also discuss immediate protection of the acquired images directly on the acquisition device. We developed a prototype application for the Android platform. By parallelizing the protection algorithm, it becomes possible to handle even large image efficiently. Contemporary smart phones can apply the protection to high-resolution color images within a few seconds. More information, including benchmarks for specific phones, can be found in the paper.
The developed self-recovery method was adopted in MITIS - a malicious image tampering identification system. The system includes a smart-phone app which protects photographs at the time of acquisition, and a desktop app with content authentication capabilities. Below you can see a sample screenshot of the authentication interface. A video demonstrating the operation of the system is available in supplementary materials.
The developed technology was presented on international innovation fares and exhibitions, and was awarded by:
The described self-recovery algorithms can also be used in other applications, including:
Similar communication models, based on erasure communication, can also be used for other watermarking problems, e.g., annotation watermarking, where they allow to recover from cropping, and multiplex many messages in a single medium.
We are also experimenting with other approaches to digital image self-recovery. We've recently developed a highly robust scheme capable of approximate reconstruction of the original image content even when the tampered image is heavily compressed and the tampering is extensive. The scheme is based on pseudo-randomly chosen profiles of pixel intensities. The greater the number of valid profiles can be extracted from the image, the better. The scheme is described in detail in:
The general principle of content reconstruction is illustrated below.
Below, we show practical recovery examples using our two different approaches. The top row shows high-quality reconstruction and precise tampering localization in a compressed JPEG image. The bottom row, shows a coarse recovery after very significant tampering (local changes by inpainting the wire, adding a building + global changes by brightness adjustment and medium JPEG compression). Despite low resolution, the original content of the photo is visible, and changes in brightness have also been recovered.