You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
FaceForensics++ is a forensics dataset consisting of 1000 original video sequences that have been manipulated with three automated face manipulation methods: Deepfakes, Face2Face and FaceSwap. The data has been sourced from 977 youtube videos and all videos contain a trackable mostly frontal face without occlusions which enables automated tampering methods to generate realistic forgeries. As we provide binary masks the data can be used for image and video classification as well as segmentation. In addition, we provide 1000 Deepfakes models to generate and augment new data.
6
+
FaceForensics++ is a forensics dataset consisting of 1000 original video sequences that have been manipulated with four automated face manipulation methods: Deepfakes, Face2Face, FaceSwap and NeuralTextures. The data has been sourced from 977 youtube videos and all videos contain a trackable mostly frontal face without occlusions which enables automated tampering methods to generate realistic forgeries. As we provide binary masks the data can be used for image and video classification as well as segmentation. In addition, we provide 1000 Deepfakes models to generate and augment new data.
7
+
8
+
7
9
8
10
For more information, please consult [our updated paper](https://arxiv.org/abs/1901.08971).
We are offering an [automated benchmark](http://kaldir.vc.in.tum.de/faceforensics_benchmark/) for facial manipulation detection on the presence of compression based on our manipulation methods that contains 1000 images. If you are interested to test your approach on unseen data, check it out! For more information, please consult [our paper](https://arxiv.org/abs/1901.08971).
12
+
## What is new:
12
13
13
-
## What is new
14
+
-__DeepFakes Detection Dataset:__ We are hosting the DeepFakes Detection Dataset provided by Google & JigSaw. The dataset contains over 3000 manipulated videos from 28 actors in various scenes. The dataset has a similar file structure and is downloaded by default together with the regular dataset.
14
15
15
-
We included a fourth manipulation method that does face manipulation using GANs and [Neural Textures](https://arxiv.org/pdf/1904.12356.pdf). All results have been updated to incorporate the new manipulation method and we have updated the benchmark as well. We refer to the paper for more information.
16
+
-__Neural Textures:__We included a fourth manipulation method that does face manipulation using GANs and [Neural Textures](https://arxiv.org/pdf/1904.12356.pdf). All results have been updated to incorporate the new manipulation method and we have updated the benchmark as well. We refer to the paper for more information.
16
17
Unfortunately, we won't continue support on the old benchmark after this update, though you can still submit your models to the new benchmark by creating a new submission.
17
18
18
19
## Access
@@ -22,6 +23,10 @@ If you have not received a response within a week, it is likely that your email
22
23
23
24
Once, you obtain the download link, please head to the [download section](dataset/README.md). You can also find details about the generation of the dataset there.
We are offering an [automated benchmark](http://kaldir.vc.in.tum.de/faceforensics_benchmark/) for facial manipulation detection on the presence of compression based on our manipulation methods that contains 1000 images. If you are interested to test your approach on unseen data, check it out! For more information, please consult [our paper](https://arxiv.org/abs/1901.08971). You can download the benchmark images here.
28
+
29
+
25
30
## Original FaceForensics
26
31
You can view the original FaceForensics github [here](https://github.com/ondyari/FaceForensics/tree/original). Any request will also contain the download link to the original version of our dataset.
27
32
@@ -47,6 +52,8 @@ Please view our youtube video [here](https://www.youtube.com/watch?v=x2g48Q2I2ZQ
We use the [faceswap implementation from the deepfakes github](https://github.com/deepfakes/faceswap) for our generated DeepFakes videos. We made some changes to their implementation to make it fully automatic for our extracted videos. If you are interested in their current status, please head to the corresponding github.
4
4
We provide the source code that was used for our experiments as well as the scripts to produce new videos as well as to recreate our manipulated videos using our provided models.
NeuralTextures videos are manipulated by using the face model by Face2Face to track and render corresponding UV masks. Those masks are then fed into an encoder decoder architecture which is optimized using Neural Textures. See [the appendix](https://arxiv.org/pdf/1901.08971.pdf) as well as [the original paper](https://arxiv.org/pdf/1904.12356.pdf) for more implementation details.
4
+
5
+
## Example Video
6
+
7
+
8
+
## Disclaimer about Reenactment Quality
9
+
10
+
We manipulated every video in our dataset multiple times and chose the best visual result for our dataset. This was needed, as we trained our network using a generative adversarial loss which sometimes corrupted the training und the output process. By heart, NeuralTextures will reenact face motions of an input video to a target video, though by using this adversarial approach it can happen that the reenactment will not be precise. As we only evaluated for visual quality, those instances can happen but visual quality is the more important factor for our main detection task.
11
+
12
+
## Masks
13
+
In comparison to FaceSwap or Face2Face, it is not straightforward what to select as the NeuralTextures manipulated area. Mainly, we manipulate the region below the mouth though we feed a region that is a 1.7 scaled quadratic region around the Face2Face mask. The network can introduce noise in that area but that necessarily. We provide the used Face2Face masks for now, but will include the uv masks as well in the near future.
If you would like to download the FaceForensics++ data, please fill out [this google form](https://docs.google.com/forms/d/e/1FAIpQLSdRRR3L5zAv6tQ_CKxmK4W96tAab_pfBu2EKAgQbeDVhmXagg/viewform) and, once accepted, we will send you the link to our download script.. You will get a link to the download script which will be used throughout this text to obtain the full dataset. This includes 977 downloaded videos from youtube, 1000 original extracted sequences that contain a unobstructed face that can be easily tracked, as well as their manipulated versions using our three methods: Deepfakes, Face2Face and FaceSwap. We also provide all Deepfakes models.
6
6
7
+
8
+
(Example of a Face2Face manipulated video, videos of other methods can be found in their respective folders)
9
+
7
10
There are two ways to get the dataset: you can use the script to download all images or videos or generate most of the data on your own using the scripts provided in this folder which saves quite a bit of bandwidth if you are interested in the raw image material. However, you will have to download the Face2Face manipulated videos/images as there is no publicly available implementation to generate them from scratch.
8
11
9
12
The dataset has the following folder structure which will either be produced by the download or generation scripts.
@@ -14,42 +17,64 @@ FaceForensics++ dataset
14
17
< contains all original downloaded videos, video information files and their extracted sequences
15
18
which can be used to extract the original sequences used in the dataset >
16
19
|-- original_sequences
17
-
< c0/raw original sequence images/videos >
18
-
< c23/hq original sequence images/videos >
19
-
< c40/lq original sequence images/videos >
20
-
|-- manipulated_sequenecs
20
+
|-- youtube
21
+
< c0/raw original sequence images/videos of the FaceForensics++ dataset >
22
+
< c23/hq original sequence images/videos >
23
+
< c40/lq original sequence images/videos >
24
+
|-- actors
25
+
< images/videos from the DeepFakesDetection dataset >
26
+
|-- manipulated_sequences
21
27
|-- Deepfakes
22
-
< Deepfake sequence images/videos of all three compression degrees as well as models and masks after poisson image editing>
28
+
< images/videos of all three compression degrees as well as models and masks after poisson image editing>
29
+
|-- DeepFakesDetection
30
+
< images/videos ... as well as masks >
23
31
|-- Face2Face
24
-
< Face2Face sequence images/videos of all three compression degrees as well as masks >
32
+
<images/videos ... as well as masks >
25
33
|-- FaceSwap
26
-
< FaceSwap sequence images/videos of all three compression degrees as well as masks >
34
+
<images/videos ... as well as masks >
27
35
|-- NeuralTextures
28
-
< NeuralTextures sequence images/videos of all three compression degrees as well as masks >
36
+
<images/videos ... well as masks >
29
37
```
30
38
31
-
We renamed all original sequences to integers between `0` and `999`. The original youtube id's can be recovered using `conversion_dict.json`.
39
+
### Original sequence filenames
40
+
- FaceForensics++: We renamed all original sequences saved in the `youtube` folder to integers between `0` and `999`. The original youtube id's can be recovered using `conversion_dict.json`.
41
+
- DeepFakesDetection: The original DeepFakesDetection sequences are stored in the `actors` folder. The sequence filenames are of the form `<actor number>__<scene name>`.
42
+
43
+
### Manipulated sequence filenames
44
+
- FaceForensics++: All filenames are of the form `<target sequence>_<source sequence>` so you can easily identify the sources.
45
+
- DeepFakesDetection: We employ a similar naming scheme here, however it is a little bit more tricky. The naming scheme is `<target actor>_<source actor>__<sequence name>__<8 charactor long experiment id>`. The experiment id is necessary as some actor pairings have been recorded multiple times.
32
46
47
+
### Space requirement
33
48
Here is a overview of the space required to save/download the dataset:
34
49
35
-
- The original downladed videos from youtube: 38.5GB
36
-
- All h264 compressed videos with compression rate factor
37
-
- 0: ~400GB
38
-
- 23: ~8GB
39
-
- 40: ~1GB
40
-
- All raw extracted images as pngs: ~2TB
50
+
- FaceForensics++
51
+
- The original downladed source videos from youtube: 38.5GB
52
+
- All h264 compressed videos with compression rate factor
53
+
- raw/0: ~500GB
54
+
- 23: ~10GB
55
+
- 40: ~2GB
56
+
- All raw extracted images as pngs: ~2TB
57
+
- DeepFakesDetection:
58
+
- The 363 original source actor videos:
59
+
- raw/0: ~200GB
60
+
- 23: ~3GB
61
+
- 40: ~400MB
62
+
- The 3068 manipulated videos:
63
+
- raw/0: ~1.6TB
64
+
- c23: ~22GB
65
+
- c40: ~3GB
41
66
42
67
## 1. Download script
43
68
44
69
### General usage
45
70
Please consult
46
71
47
-
`python download-FaceForensics_v3.py -h`
72
+
`python download-FaceForensics_v4.py -h`
48
73
49
74
for a detailed overview of the download scrips parameter choices and their respective defaults. The general usage is as follows:
50
75
51
76
```shell
52
-
python download-FaceForensics_v3.py
77
+
python download-FaceForensics_v4.py
53
78
<output path>
54
79
-d <dataset type, e.g., Face2Face, original or all>
55
80
-c <compression quality, e.g., c23 or raw>
@@ -63,25 +88,33 @@ We advise you to download the compressed videos and extract the frames on your o
63
88
### Examples
64
89
In order to download all light compressed (i.e., a visually lossless compression rate factor of 23 using the h264 codec) original as well as altered videos of all three manipulation methods use
65
90
66
-
`python download-Faceforensics_v3.py <output path> -d all -c c23 -t videos`
91
+
`python download-Faceforensics_v4.py <output path> -d all -c c23 -t videos`
92
+
93
+
If you are only interested in a few samples of the dataset, say 10, append `--num_videos 10`.
67
94
68
-
For all lossless compressed (i.e., a compression rate factor of 0) extracted original videos run
95
+
For all raw/lossless compressed (i.e., a compression rate factor of 0) extracted original videos run
69
96
70
-
`python download-FaceForensics_v3.py <output path> -d original -c c0 -t videos`
97
+
`python download-FaceForensics_v4.py <output path> -d original -c raw -t videos`
98
+
99
+
The DeepFakesDetection dataset videos can be obtained by running
100
+
101
+
`python download-FaceForensics_v4.py <output path> -d <DeepFakesDetection or DeepFakesDetection_original>-c raw -t videos`
The zipped file contains all downloaded videos in their original length as well as a json file containing the frames that were extracted for our dataset. If you are only interested in the frame locations and video information because you want to download them on your own, use:
83
116
84
-
The zipped file contains all downloaded videos in their original length as well as a json file containing the frames that were extracted for our dataset.
@@ -95,6 +128,7 @@ We only downloaded the source video without audio. However, you can re-download
95
128
We provide binary masks for all our manipulation methods. For FaceSwap and Face2Face those masks are pretty self-explanatory. However, it is more difficult for DeepFakes and NeuralTextures.
96
129
- Deepfakes: after we feed in our face through the auto-encoder and warp it back to the image, we apply Poisson image editing. This process is done on a rectangular box around the face. Please consult the [DeepFakes readme](datasets/DeepFakes).
97
130
- NeuralTextures: NeuralTextures takes a 1.7 scaled part around the face bounding box of the Face2Face tracker as input and manipulates the whole region. However, the method has skip connections which allow it to directly copy pixel values from non-face areas of this crop. The NeuralTexture masks report the tracking results for those regions, though we will upload the manipulated regions as well and add more details to this process soon.
131
+
- DeepFakesDetection: masks are provided direcly after DeepFake output and thus are not rectangular shaped as the Deepfakes masks provided in FaceForensics++. We will provide those in the near future.
0 commit comments