Evaluation of Random Field Models in Multi-modal Unsupervised Tampering Localization

Paweł Korus and Jiwu Huang

IEEE International Workshop on Information Forensics and Security, 2016

2016-wifs-mrf.pdf (5 MB)

10.1109/WIFS.2016.7823898

Abstract

While it is commonly known that successful forensic detectors should combine the clues from various forensic features, unsupervised multi-modal tampering localization is still an open problem. State-of-the-art fusion methods perform simple pixel-wise combination of the input tampering maps. In this study, we show that pixel-wise combination is sub-optimal and successful fusion needs to model dependencies between neighboring pixels and exploit the content of the tampered image. We evaluate two methods based on conditional random fields and demonstrate that they can exploit image content and precisely delineate the shape of the forgery. In contrast to existing methods based on explicit image segmentation, such an approach does not suffer from subtle object removal forgeries where meaningful segments do not exist. We also demonstrate that existing performance measures are insufficient to accurately assess tampering localization performance. Further work in this direction is needed.

Copyright

Copyright © 2016 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org.

Supplementary Materials

Supplementary materials for this paper include:

  • Additional details about the implemented schemes;
  • Comparison with a different formulation of the random field;
  • Evaluation of the presented model as a post-processing method for individual forensic detectors;
  • Extended discussion about the limitations and prospects for future work.
  • The document is available for download here: wifs-2016-supplement.pdf (~300 kB).

Slides from conference presentation are available here: wifs-2016-presentation.pdf (5.2 MB).

Dataset

The realistic image tampering dataset used in this study is available for download here.

Source code

Our implementation of selected methods used in this work can be found at github.com.