Over the years, researchers have proposed various approaches to JPEG forgery detection and localization. In most cases, experimental evaluation was limited to JPEG quality levels that are multiples of 5 or 10. Each study used a different dataset, making it difficult to directly compare the reported results. The goal of this work is to perform a unified, large-scale and fine-grained evaluation of the most popular state-of-the-art detectors. The obtained results allow to compare the detectors with respect to various criteria, and shed more light on the compression configurations where reliable tampering localization can be expected.