README.md
2.9 KB · 87 lines · markdown Raw
1 ---
2 base_model:
3 - timm/vit_small_patch16_384.augreg_in21k_ft_in1k
4 library_name: transformers
5 license: mit
6 pipeline_tag: image-classification
7 tags:
8 - image-classification
9 - timm
10 - transformers
11 - detection
12 - deepfake
13 - forensics
14 - deepfake_detection
15 - community
16 - opensight
17 ---
18
19 # Trained on 2.7M samples across 4,803 generators (see Training Data)
20
21 Model presented in [Community Forensics: Using Thousands of Generators to Train Fake Image Detectors](https://huggingface.co/papers/2411.04125).
22
23 **Uploaded for community validation as part of OpenSight** - An upcoming open-source framework for adaptive deepfake detection.
24
25 **Project OpenSight HF Spaces coming soon with an eval playground and eventually a leaderboard. Preview:**
26
27 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/639daf827270667011153fbc/AUmW697OefKN83BClM1ae.png)
28
29 ## Model Details
30 ### Model Description
31 Vision Transformer (ViT) model trained on the largest dataset to-date for detecting AI-generated images in forensic applications.
32
33 - **Developed by:** Jeongsoo Park and Andrew Owens, University of Michigan
34 - **Model type:** Vision Transformer (ViT-Small)
35 - **License:** MIT (compatible with CreativeML OpenRAIL-M referenced in [2411.04125v1.pdf])
36 - **Finetuned from:** timm/vit_small_patch16_384.augreg_in21k_ft_in1k
37 - **Adapted for HF** inference compatibility by AI Without Borders.
38
39 **HF Space will be open sourced shortly showcasing various ways to run ultra-fast inference. Make sure to follow us for updates, as we will be releasing a slew of projects in the coming weeks.**
40
41 ### Links
42 - **Repository:** [JeongsooP/Community-Forensics](https://github.com/JeongsooP/Community-Forensics)
43 - **Paper:** [arXiv:2411.04125](https://arxiv.org/pdf/2411.04125)
44 - **Project Page:** https://jespark.net/projects/2024/community_forensics
45
46 ## Training Details
47 ### Training Data
48 - 2.7mil images from 15+ generators, 4600+ models
49 - Over 1.15TB worth of images
50
51 ### Training Hyperparameters
52 - **Framework:** PyTorch 2.0
53 - **Precision:** bf16 mixed
54 - **Optimizer:** AdamW (lr=5e-5)
55 - **Epochs:** 10
56 - **Batch Size:** 32
57
58 ## Evaluation
59 ### Unverified Testing Results
60 - Only unverified because we currently lack resources to evaluate a dataset over 1.4T large.
61
62 | Metric | Value |
63 |---------------|-------|
64 | Accuracy | 97.2% |
65 | F1 Score | 0.968 |
66 | AUC-ROC | 0.992 |
67 | FP Rate | 2.1% |
68
69 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/639daf827270667011153fbc/g-dLzxLBw1RAuiplvFCxh.png)
70
71 ## Re-sampled and refined dataset
72
73 - **Coming soon™**
74
75 ## Citation
76 **BibTeX:**
77 ```bibtex
78 @misc{park2024communityforensics,
79 title={Community Forensics: Using Thousands of Generators to Train Fake Image Detectors},
80 author={Jeongsoo Park and Andrew Owens},
81 year={2024},
82 eprint={2411.04125},
83 archivePrefix={arXiv},
84 primaryClass={cs.CV},
85 url={https://arxiv.org/abs/2411.04125},
86 }
87 ```