Potential advantages and limitations of using information fusion in media forensics—a discussion on the example of detecting face morphing attacks
Tóm tắt
Information fusion, i.e., the combination of expert systems, has a huge potential to improve the accuracy of pattern recognition systems. During the last decades, various application fields started to use different fusion concepts extensively. The forensic sciences are still hesitant if it comes to blindly applying information fusion. Here, a potentially negative impact on the classification accuracy, if wrongly used or parameterized, as well as the increased complexity (and the inherently higher costs for plausibility validation) of fusion is in conflict with the fundamental requirements for forensics. The goals of this paper are to explain the reasons for this reluctance to accept such a potentially very beneficial technique and to illustrate the practical issues arising when applying fusion. For those practical discussions the exemplary application scenario of morphing attack detection (MAD) is selected with the goal to facilitate the understanding between the media forensics community and forensic practitioners. As general contributions, it is illustrated why the naive assumption that fusion would make the detection more reliable can fail in practice, i.e., why fusion behaves in a field application sometimes differently than in the lab. As a result, the constraints and limitations of the application of fusion are discussed and its impact to (media) forensics is reflected upon. As technical contributions, the current state of the art of MAD is expanded by:
Tài liệu tham khảo
H. Drucker, C. Cortes, L.D. Jackel, Y. LeCun, V. Vapnik, Boosting and other ensemble methods. Neural Comput. 6(6), 1289–1301 (1994)
L. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms (Wiley-Interscience, 2004)
M. Tan, Multi-agent reinforcement learning: independent vs. cooperative agents. Readings in agents (Morgan Kaufmann Publishers Inc., San Francisco, 1997), pp. 487–494
T.K. Ho, in Hybrid Methods in Pattern Recognition, ed. by A. Kandel, H. Bunke. Multiple classifier combination: lessons and the next steps (World Scientific Publishing, 2002), pp. 171–198
A. Ross, K. Nandakumar, A.K. Jain, Handbook of Multibiometrics (Springer, 2006)
R.P. Srivastava, Alternative Form of Dempster’s Rule for Binary Variables. Int. J. Intell. Syst. 20(8), 789–797 (2005)
A. Makrushin, C. Kraetzer, J. Dittmann, C. Seibold, A. Hilsmann, P. Eisert, in Proc. 27th European Signal Processing Conference (EUSIPCO). Dempster-Shafer Theory for Fusing Face Morphing Detectors (A Coruna, 2019), pp. 1–5
A. Makrushin, A. Wolf, in Proc. 26th European Signal Processing Conference (EUSIPCO). An Overview of Recent Advances in Assessing and Mitigating the Face Morphing Attack (2018)
U. Scherhag, C. Rathgeb, J. Merkle, R. Breithaupt, C. Busch, Face Recognition Systems Under Morphing Attacks: A Survey. IEEE Access 7(2019), 23012–23026 (2019)
M. Ferrara, A. Franco, D. Maltoni, in Face Recognition Across the Electromagnetic Spectrum, ed. by T. Bourlai. On the effects of image alterations on face recognition accuracy (Springer, Cham, 2016), pp. 195–222
R.S.S. Kramer, M.O. Mireku, T.R. Flack, K.L. Ritchie, Face morphing attacks: investigating detection with humans and computers. Cogn. Res. Princ. Implications 4, 28 (2019)
A. Makrushin, T. Neubert, J. Dittmann, in Proc. 12th Int. Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6. Automatic generation and detection of visually faultless facial morphs (VISAPP, 2017), pp. 39–50
U. Scherhag, R. Raghavendra, K.B. Raja, M. Gomez-Barrero, C. Rathgeb, C. Busch, in Proc. 5th International Workshop on Biometrics and Forensics (IWBF). On the Vulnerability of Face Recognition Systems: Towards Morphed Face Attacks (2017)
National Institute of Standards and Technology (NIST) FRVT MORPH, https://pages.nist.gov/frvt/html/frvt_morph.html
C. Champod, J. Vuille, in International Commentary on Evidence. Vol. 9, Issue 1. Scientific evidence in Europe – admissibility, evaluation and equality of arms (2011) Available at: https://core.ac.uk/reader/85212846 (Last accessed: 26 Aug 2020)
M. Ferrara, A. Franco, D. Maltoni, in Proc. Int. Joint Conf. on Biometrics (IJCB). The magic passport (2014), pp. 1–7
C. Seibold, W. Samek, A. Hilsmann, P. Eisert, in Proc. Int. Workshop Digital Watermarking (IWDW2017). Detection of Face Morphing Attacks by Deep Learning (Springer, Berlin, 2017)
U. Scherhag, C. Rathgeb, C. Busch, in Proc. 13th IAPR Workshop on Document Analysis Systems (DAS’18). Towards detection of morphed face images in electronic travel documents (2018)
C. Kraetzer, A. Makrushin, T. Neubert, M. Hildebrandt, J. Dittmann, in Proc. 5th ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec’17). Modeling attacks on photo-ID documents and applying media forensics for the detection of facial morphing (ACM, New York, 2017), pp. 21–32
Utrecht ECVP as part of Psychological Image Collection at Stirling (PICS), http://pics.stir.ac.uk/2D_face_sets.htm, last accessed: 31 Aug 2020.
R. Raghavendra, S. Venkatesh, K. Raja, C. Busch, in 2019 IEEE 5th International Conference on Identity, Security, and Behavior Analysis (ISBA). Towards making morphing attack detection robust using hybrid scale-space colour texture features (2019), pp. 1–8
M. Ferrara, A. Franco, D. Maltoni, Face demorphing. Trans. Inf. Forensics Secur. 13(4), 1008–1017 (2018)
D.O. del Campo, C. Conde, D. Palacios-Alonso, E. Cabello, Border control morphing attack detection with a convolutional neural network de-morphing approach. IEEE Access 8, 92301–92313 (2020)
F. Peng, L. Zhang, M. Long, FD-GAN: Face De-Morphing Generative Adversarial Network for Restoring Accomplice’s Facial Image. IEEE Access 7, 75122–75131 (2019)
U. Scherhag, D. Budhrani, M. Gomez-Barrero, C. Busch, in International Conference on Image and Signal Processing (ICISP 2018). Detecting Morphed Face Images Using Facial Landmarks (2018), pp. 444–452
C. Seibold, W. Samek, A. Hilsmann, P. Eisert, Accurate and robust neural networks for security related applications exampled by face morphing attacks. Arxiv/CoRR abs/1806.04265 (2018)
U. Scherhag, C. Rathgeb, J. Merkle, C. Busch, in IEEE Transactions on Information Forensics and Security (TIFS). Deep Face Representations for Differential Morphing Attack Detection (2020)
L. Wandzik, G. Kaeding, R.V. Garcia, in Proc. 26th Eur. Signal Process. Conf. (EUSIPCO), Sep. 2018. Morphing detection using a general- purpose face recognition system (2018), pp. 1012–1016
R. Raghavendra, K. Raja, S. Venkatesh, C. Busch, in Proc. 30th Int. Conf. on Computer Vision and Pattern Recognition Workshop. Transferable Deep-CNN features for detecting digital and print-scanned morphed face images (2017)
T. Neubert, A. Makrushin, M. Hildebrandt, C. Kraetzer, J. Dittmann, Extended StirTrace Benchmarking of Biometric and Forensic Qualities of Morphed Face Images. IET Biometrics 7(4), 325–332 (2018)
T. Karras, S. Laine, T. Aila, in IEEE Conference on Computer Vision and Pattern Recognition. A style-based generator architecture for generative adversarial networks (2019), pp. 4401–4410
T. Karras, T. Aila, S. Laine, J. Lehtinen, in International Conference on Learning Representations. Progressive growing of GANs for improved quality, stability, and variation (2018)
N. Damer, A.M. Saladie, A. Braun, A. Kuijper, in Proc. IEEE 9th Int. Conf. Biometrics Theory, Appl. Syst. (BTAS), Oct. 2018. MorGAN: recognition vulnerability and attack detectability of face morphing attacks created by generative adversarial network (2018), pp. 1–10
S. Venkatesh, H. Zhang, R. Raghavendra, K. Raja, N. Damer, C. Busch, Can GAN generated morphs threaten face recognition systems equally as landmark based morphs? -vulnerability and detection (International Workshop on Biometrics and Forensics (IWBF), 2020)
G. Shafer, A Mathematical Theory of Evidence (Princeton University Press, 1976)
P. Smets, in Proc. 15th Conf. On Uncertainty in Artificial Intelligence. Practical uses of belief functions, vol 99 (1999), pp. 612–621
J. Kittler, M. Hatef, R.P.W. Duin, J. Matas, On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)
L. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms, 2nd edn. (Wiley, New York, 2014)
M. Fontani, A. Bonchi, A. Piva, M. Barni, in Proc. Media Watermarking, Security, and Forensics 2014, San Francisco, CA, USA, February 2, 2014, ed. by A. M. Alattar, N. D. Memon, C. Heitzenrater. Countering anti-forensics by means of data fusion, vol 9028 (SPIE Proceedings, 2014), p. 90280Z SPIE
M. Fontani, T. Bianchi, A. De Rosa, A. Piva, M. Barni, A framework for decision fusion in image forensics based on Dempster-Shafer theory of evidence. IEEE Trans. Inf. Forensics Secur. 8(4), 593–607 (2013)
Royal Courts of Justice, “R v T”, [2010] EWCA Crim 2439, Redacted Judgment, 2011, Available at: http://www.bailii.org/ew/cases/EWCA/Crim/2010/2439.pdf (last accessed: 10 Mar 2021)
Y. Peng, L.J. Spreeuwers, R.N.J. Veldhuis, in Proceedings of the 3rd International Workshop on Biometrics and Forensics, IWBF 2015. Likelihood Ratio Based Mixed Resolution Facial Comparison (IEEE Computer Society, USA, 2015), pp. 1–5
T. Kerkvliet, R. Meester, Assessing forensic evidence by computing belief functions. Law Probability Risk 15(2), 127–153 (2016)
T.G. Dietterich, in Multiple classifier systems. Ensemble methods in machine learning (Springer LNCS 1857, 2000), pp. 1–15
B. Quost, M.-H. Masson, T. Denœux, Classifier fusion in the Dempster–Shafer framework using optimized t-norm based combination rules. Int. J. Approx. Reason. 52(3), 353–374 (2011)
K. Tumer, J. Ghosh, Error correlation and error reduction in ensemble classifiers. Connect. Sci. 3–4(8), 385–404 (1996)
S.P. Lund, H. Iyer, Likelihood ratio as weight of forensic evidence: a closer look. J. Res. Nat. Instit. Stand. Technol. 122, 122.027 (2017) 2017
A. Nordgaard, R. Ansell, W. Drotz, L. Jaeger, Scale of conclusions for the value of evidence. Law Probability Risk 11(1), 1–24 (2012)
K. Nandakumar, Y. Chen, S.C. Dass, A. Jain, Likelihood Ratio-Based Biometric Score Fusion. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 342–347 (2008)
ISO/IEC JTC1 SC37 Biometrics, ISO/IEC 30107-3:2017 Information technology- biometric presentation attack detection - Part3: Testing & reporting. ISO, 2017.
G. Mahfoudi, B. Tajini, F. Retraint, F. Morain-Nicolier, J. L. Dugelay, M. Pic: DEFACTO: Image and Face Manipulation Dataset. 27th European Signal Processing Conference (EUSIPCO), A Coruna, Spain, 2019, pp. 1-5, (dataset:https://defactodataset.github.io/), 2019.
L. DeBruine, B. Jones: Face Research Lab London Set. May 30th, 2017. Available at: https://figshare.com/articles/dataset/Face_Research_Lab_London_Set/5047666 (last accessed 10 Sept 2020).
Alabama News Network Mugshot database, online: https://www.alabamanews.net/mugshots/ (last accessed: 9 Sept 2020).
D. Maltoni, D. Maio, A.K. Jain, S. Prabhakar, in Synthetic Fingerprint Generation. Handbook of Fingerprint Recognition (Springer, London, 2009), pp. 271–302