Latent Pattern Sensing: Deepfake Video Detection via Predictive Representation Learning

Ge，Shiming1; Lin，Fanzhao1; Li，Chenyu1; Zhang，Daichi1; Tan，Jiyong2; Wang，Weiping1; Zeng，Dan3

10.1145/3469877.3490586

2021-12-01

ACM International Conference Proceeding Series

Increasingly advanced deepfake approaches have made the detection of deepfake videos very challenging. We observe that the general deepfake videos often exhibit appearance-level temporal inconsistencies in some facial components between frames, resulting in discriminable spatiotemporal latent patterns among semantic-level feature maps. Inspired by this finding, we propose a predictive representative learning approach termed Latent Pattern Sensing to capture these semantic change characteristics for deepfake video detection. The approach cascades a CNN-based encoder, a ConvGRU-based aggregator and a single-layer binary classifier. The encoder and aggregator are pre-trained in a self-supervised manner to form the representative spatiotemporal context features. Finally, the classifier is trained to classify the context features, distinguishing fake videos from real ones. In this manner, the extracted features can simultaneously describe the latent patterns of videos across frames spatially and temporally in a unified way, leading to an effective deepfake video detector. Extensive experiments prove our approach's effectiveness, e.g., surpassing 10 state-of-the-arts at least 7.92%@AUC on challenging Celeb-DF(v2) benchmark.

Deepfake video detection predictive representation learning self-supervised learning

其他

英语

EI

20220411498009

Classification (of information) ; Computer vision ; Deep learning ; Fake detection ; Semantics

Ergonomics and Human Factors Engineering:461.4 ; Information Theory and Signal Processing:716.1 ; Computer Applications:723.5 ; Vision:741.2 ; Information Sources and Analysis:903.1

2-s2.0-85123054345

Scopus

1.Institute of Information Engineering,Chinese Academy of Sciences,China
2.Southern University of Science and Technology,China
3.Shanghai University,China

Ge，Shiming,Lin，Fanzhao,Li，Chenyu,et al. Latent Pattern Sensing: Deepfake Video Detection via Predictive Representation Learning[C],2021.