Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field.
Author(s): Husrev Taha Sencar, Luisa Verdoliva, Nasir Memon
Series: Advances In Computer Vision And Pattern Recognition
Edition: 1
Publisher: Springer
Year: 2022
Language: English
Commentary: TruePDF
Pages: 494
Tags: Systems And Data Security; Image Processing And Computer Vision; Computer Imaging, Vision, Pattern Recognition And Graphics; Signal, Image And Speech Processing; Machine Learning; Security Services
Preface
Contents
Symbols
Notation
Part I Present and Challenges
1 What's in This Book and Why?
1.1 Introduction
1.2 Overviews
2 Media Forensics in the Age of Disinformation
2.1 Media and the Human Experience
2.2 The Threat to Democracy
2.3 New Technologies, New Threats
2.3.1 End-to-End Trainable Speech Synthesis
2.3.2 GAN-Based Codecs for Still and Moving Pictures
2.3.3 Improvements in Image Manipulation
2.3.4 Trillion-Param Models
2.3.5 Lottery Tickets and Compression in Generative Models
2.4 New Developments in the Private Sector
2.4.1 Image and Video
2.4.2 Language Models
2.5 Threats in the Wild
2.5.1 User-Generated Manipulations
2.5.2 Corporate Manipulation Services
2.5.3 Nation State Manipulation Examples
2.5.4 Use of AI Techniques for Deception 2019–2020
2.6 Threat Models
2.6.1 Carnegie Mellon BEND Framework
2.6.2 The ABC Framework
2.6.3 The AMITT Framework
2.6.4 The SCOTCH Framework
2.6.5 Deception Model Effects
2.6.6 4Ds
2.6.7 Advanced Persistent Manipulators
2.6.8 Scenarios for Financial Harm
2.7 Investments in Countering False Media
2.7.1 DARPA SEMAFOR
2.7.2 The Partnership on AI Steering Committee on Media Integrity Working Group
2.7.3 JPEG Committee
2.7.4 Content Authenticity Initiative (CAI)
2.7.5 Media Review
2.8 Excerpts on Susceptibility and Resilience to Media Manipulation
2.8.1 Susceptibility and Resilience
2.8.2 Case Studies: Threats and Actors
2.8.3 Dynamics of Exploitative Activities
2.8.4 Meta-Review
2.9 Conclusion
References
3 Computational Imaging
3.1 Introduction to Computational Imaging
3.2 Automation of Geometrically Correct Synthetic Blur
3.2.1 Primary Cue: Image Noise
3.2.2 Additional Photo Forensic Cues
3.2.3 Focus Manipulation Detection
3.2.4 Portrait Mode Detection Experiments
3.2.5 Conclusions on Detecting Geometrically Correct Synthetic Blur
3.3 Differences Between Optical and Digital Blur
3.3.1 Authentically Blurred Edges
3.3.2 Authentic Sharp Edge
3.3.3 Forged Blurred Edge
3.3.4 Forged Sharp Edge
3.3.5 Distinguishing IGHs of the Edge Types
3.3.6 Classifying IGHs
3.3.7 Splicing Logo Dataset
3.3.8 Experiments Differentiating Optical and Digital Blur
3.3.9 Conclusions: Differentiating Optical and Digital Blur
3.4 Additional Forensic Challenges from Computational Cameras
References
Part II Attribution
4 Sensor Fingerprints: Camera Identification and Beyond
4.1 Introduction
4.2 Sensor Noise Fingerprints
4.3 Camera Identification
4.4 Sensor Misalignment
4.5 Image Manipulation Localization
4.6 Counter-Forensics
4.7 Camera Fingerprints and Deep Learning
4.8 Public Datasets
4.9 Concluding Remarks
References
5 Source Camera Attribution from Videos
5.1 Introduction
5.2 Challenges in Attributing Videos
5.3 Attribution of Downsized Media
5.3.1 The Effect of In-Camera Downsizing on PRNU
5.3.2 Media with Mismatching Resolutions
5.4 Mitigation of Video Coding Artifacts
5.4.1 Video Coding from Attribution Perspective
5.4.2 Compensation of Loop Filtering
5.4.3 Coping with Quantization-Related Weakening of PRNU
5.5 Tackling Digital Stabilization
5.5.1 Inverting Frame Level Stabilization Transformations
5.5.2 Inverting Spatially Variant Stabilization Transformations
5.6 Datasets
5.7 Conclusions and Outlook
References
6 Camera Identification at Large Scale
6.1 Introduction
6.2 Naive Methods
6.2.1 Linear Search
6.2.2 Sequential Trimming
6.3 Efficient Pairwise Correlation
6.3.1 Search over Fingerprint Digests
6.3.2 Pixel Quantization
6.3.3 Downsizing
6.3.4 Dimension Reduction Using PCA and LDA
6.3.5 PRNU Compression via Random Projection
6.3.6 Preprocessing, Quantization, Coding
6.4 Decreasing the Number of Comparisons
6.4.1 Clustering by Cameras
6.4.2 Composite Fingerprints
6.5 Hybrid Methods
6.5.1 Search over Composite-Digest Search Tree
6.5.2 Search over Full Digest Search Tree
6.6 Conclusion
References
7 Source Camera Model Identification
7.1 Introduction
7.1.1 Image Acquisition Pipeline
7.1.2 Problem Formulation
7.2 Model-Based Approaches
7.2.1 Color Filter Array (CFA)
7.2.2 Lens Effects
7.2.3 Other Processing and Defects
7.3 Data-Driven Approaches
7.3.1 Hand-Crafted Features
7.3.2 Learned Features
7.4 Datasets and Benchmarks
7.4.1 Template Dataset
7.4.2 State-of-the-art Datasets
7.4.3 Benchmark Protocol
7.5 Case Studies
7.5.1 Experimental Setup
7.5.2 Comparison of Closed-Set Methods
7.5.3 Comparison of Open-Set Methods
7.6 Conclusions and Outlook
References
8 GAN Fingerprints in Face Image Synthesis
8.1 Introduction
8.2 Related Work
8.2.1 Generative Adversarial Networks
8.2.2 GAN Detection Techniques
8.3 GAN Fingerprint Removal: GANprintR
8.4 Databases
8.4.1 Real Face Images
8.4.2 Synthetic Face Images
8.5 Experimental Setup
8.5.1 Pre-processing
8.5.2 Facial Manipulation Detection Systems
8.5.3 Protocol
8.6 Experimental Results
8.6.1 Controlled Scenarios
8.6.2 In-the-Wild Scenarios
8.6.3 GAN-Fingerprint Removal
8.6.4 Impact of GANprintR on Other Fake Detectors
8.7 Conclusions and Outlook
References
Part III Integrity and Authenticity
9 Physical Integrity
9.1 Introduction
9.1.1 Journalistic Fact Checking
9.1.2 Physics-Based Methods in Multimedia Forensics
9.1.3 Outline of This Chapter
9.2 Physics-Based Models for Forensic Analysis
9.2.1 Geometry and Optics
9.2.2 Photometry and Reflectance
9.3 Algorithms for Physics-Based Forensic Analysis
9.3.1 Principal Points and Homographies
9.3.2 Photometric Methods
9.3.3 Point Light Sources and Line Constraints in the Projective Space
9.4 Discussion and Outlook
9.5 Picture Credits
References
10 Power Signature for Multimedia Forensics
10.1 Electric Network Frequency (ENF): An Environmental Signature for Multimedia Recordings
10.2 Technical Foundations of ENF-Based Forensics
10.2.1 Reference Signal Acquisition
10.2.2 ENF Signal Estimation
10.2.3 Higher Order Harmonics for ENF Estimation
10.3 ENF Characteristics and Embedding Conditions
10.3.1 Establishing Presence of ENF Traces
10.3.2 Modeling ENF Behavior
10.4 ENF Traces in the Visual Track
10.4.1 Mechanism of ENF Embedding in Videos and Images
10.4.2 ENF Extraction from the Visual Track
10.4.3 ENF Extraction from a Single Image
10.5 Key Applications in Forensics and Security
10.5.1 Joint Time–Location Authentication
10.5.2 Integrity Authentication
10.5.3 ENF-Based Localization
10.5.4 ENF-Based Camera Forensics
10.6 Anti-Forensics and Countermeasures
10.6.1 Anti-Forensics and Detection of Anti-Forensics
10.6.2 Game-Theoretic Analysis on ENF-Based Forensics
10.7 Applications Beyond Forensics and Security
10.7.1 Multimedia Synchronization
10.7.2 Time-Stamping Historical Recordings
10.7.3 Audio Restoration
10.8 Conclusions and Outlook
References
11 Data-Driven Digital Integrity Verification
11.1 Introduction
11.2 Forensics Clues
11.2.1 Camera-Based Artifacts
11.2.2 JPEG Artifacts
11.2.3 Editing Artifacts
11.3 Localization Versus Detection
11.3.1 Patch-Based Localization
11.3.2 Image-Based Localization
11.3.3 Detection
11.4 Architectural Solutions
11.4.1 Constrained Networks
11.4.2 Two-Branch Networks
11.4.3 Fully Convolutional Networks
11.4.4 Siamese Networks
11.5 Datasets
11.6 Major Challenges
11.7 Conclusions and Future Directions
References
12 DeepFake Detection
12.1 Introduction
12.2 DeepFake Video Generation
12.3 Current DeepFake Detection Methods
12.3.1 General Principles
12.3.2 Categorization Based on Methodology
12.3.3 Categorization Based on Input Types
12.3.4 Categorization Based on Output Types
12.3.5 The DeepFake-o-Meter Platform
12.3.6 Datasets
12.3.7 Challenges
12.4 Future Directions
12.5 Conclusion and Outlook
References
13 Video Frame Deletion and Duplication
13.1 Introduction
13.2 Related Work
13.2.1 Frame Deletion Detection
13.2.2 Frame Duplication Detection
13.3 Frame Deletion Detection
13.3.1 Baseline Approaches
13.3.2 C3D Network for Frame Deletion Detection
13.3.3 Experimental Result
13.4 Frame Duplication Detection
13.4.1 Coarse-Level Search for Duplicated Frame Sequences
13.4.2 Fine-Level Search for Duplicated Frames
13.4.3 Inconsistency Detector for Duplication Localization
13.4.4 Experimental Results
13.5 Conclusions and Discussion
References
14 Integrity Verification Through File Container Analysis
14.1 Introduction
14.1.1 Main Image File Format Specifications
14.1.2 Main Video File Format Specifications
14.2 Analysis of Image File Formats
14.2.1 Analysis of JPEG Tables and Image Resolution
14.2.2 Analysis of Exif Metadata Parameters
14.2.3 Analysis of the JPEG File Format
14.2.4 Automatic Analysis of JPEG Header Information
14.2.5 Methods for the Identification of Social Networks
14.3 Analysis of Video File Formats
14.3.1 Analysis of the Video File Structure
14.3.2 Automated Analysis of mp4-like Videos
14.3.3 Efficient Video Analysis
14.4 Concluding Remarks
References
15 Image Provenance Analysis
15.1 The Problem
15.1.1 The Provenance Framework
15.1.2 Previous Work
15.2 Content Retrieval
15.2.1 Approaches
15.2.2 Datasets and Evaluation
15.2.3 Results
15.3 Graph Construction
15.3.1 Approaches
15.3.2 Datasets and Evaluation
15.3.3 Results
15.4 Content Clustering
15.4.1 Approach
15.4.2 Datasets and Evaluation
15.4.3 Results
15.5 Open Issues and Research Directions
15.6 Summary
References
Part IV Counter-Forensics
16 Adversarial Examples in Image Forensics
16.1 Introduction
16.2 Adversarial Examples in a Nutshell
16.2.1 Problem Definition and Review of the Most Popular Attacks
16.2.2 Adversarial Examples in the Physical Domain
16.2.3 White Versus Black-Box Attacks
16.3 Adversarial Examples in Multimedia Forensics
16.3.1 Transferability of Adversarial Examples in Multimedia Forensics
16.3.2 Increased-Confidence Adversarial Examples with Improved Transferability
16.4 Defenses
16.4.1 Detect Then Defend
16.4.2 Adversarial Training
16.4.3 Detector Randomization
16.4.4 Multiple-Classifier Architectures
16.5 Final Remarks
References
17 Anti-Forensic Attacks Using Generative Adversarial Networks
17.1 Introduction
17.2 Background on GANs
17.2.1 GANs for Image Synthesis
17.3 Brief Overview of Relevant Anti-Forensic Attacks
17.3.1 What Are Anti-Forensic Attacks
17.3.2 Anti-Forensic Attack Objectives and Requirements
17.3.3 Traditional Anti-Forensic Attack Design Procedure and Shortcomings
17.3.4 Anti-Forensic Attacks on Parametric Forensic Models
17.3.5 Anti-Forensic Attacks on Deep Neural Networks
17.4 Using GANs to Make Anti-Forensic Attacks
17.4.1 How GANs Are Used to Construct Anti-Forensic Attacks
17.4.2 Overview of Existing GAN-Based Attacks
17.4.3 Differences Between GAN-Based Anti-Forensic Attacks and Adversarial Examples
17.4.4 Advantages of GAN-Based Anti-Forensic Attacks
17.5 Training Anti-Forensic GANs
17.5.1 Overview of Adversarial Training
17.5.2 Knowledge Levels of the Victim Classifier
17.5.3 White Box Attacks
17.5.4 Black Box Scenario
17.5.5 Zero Knowledge
17.6 Known Problems with GAN-Based Attacks & Future Directions
References
