DeepFaceLab
[Image: You are not allowed to view links. Register or Login to view.]
NOTE: For better results an NVIDIA or AMD GPU with 2GB+ vRAM is recommended. The more memory available, the better quality the results.
DeepFaceLab now compatible with AMD, NVIDIA, IntelHD graphics and all OpenCL 1.2 compatibile video cards with at least 256M video memory.
I have moved from Faceswap to DeepFaceLab due to ease of use, better outcomes, and a lot of time saved. This tutorial will be a mix of the creator's instructions, and how I use and understand the program. The GitHub can be found here: You are not allowed to view links. Register or Login to view.
You are not allowed to view links. Register or Login to view.
(Choose the latest version based on date)
DeepFaceLabCUDA9.2SSE - for NVIDIA video cards up to GTX 1080 Ti
DeepFaceLabCUDA10.1AVX - for RTX NVIDIA video cards with a CPU that supports AVX
DeepFaceLabOpenCLSSE - for AMD/IntelHD cards plus any 64-bit CPU
Features:- Available as standalone with zero dependencies, ready to be used with prebult binary (CUDA, OpenCL, ffmpeg, etc.) for all windows versions.
- New models (H64, H128, DF, LIAEF128, SAE, Villain) expanding from the original faceswap model.
- New architecture, easy to experiment with models.
- Works on 2GB old cards, such as GT730. Example of a deepfake trained on a 2GB gtx850m notebook in 18 hours: You are not allowed to view links. Register or Login to view.
- Face data embedded in png files (no more aligned file required).
- Automatically manage GPU by choosing the best GPU(s).
- New preview window
- Extractor and Converter in parallel.
- Added debug option for all stages.
- Multiple face extraction modes including S3FD, MTCNN, dlib, or manual extraction.
- Train in any resolution by increments of 16. Easily train on 256 with NVIDIA cards due to optimization settings.
Extraction Modes:
S3FD Extraction: Best extractor to date. More accurate with less false positives compared to MTCNN extraction. Possibly slightly smoother.
[Image: You are not allowed to view links. Register or Login to view.][Image: You are not allowed to view links. Register or Login to view.]
Left = S3FD
Right = MTNCC
MTCNN Extraction: this mode predicts faces more uniformly compared to dlib which creates a less jittered aligned output. The disadvantage of MTCNN extraction is that it will produce a much greater number of false positives, which will mean you have to spend more time cleaning up the facesets generated.
[Image: You are not allowed to view links. Register or Login to view.]
Left = dlib
Right = MTCNN
Manual Extractor: This uses a preview GUI that will allow users to properly align detected faces by changing the landmarks on the image itself. This is often very useful when faces are obstructed and can significantly improve the quality of your faceset to improve overall training and deepfakes.
[Image: You are not allowed to view links. Register or Login to view.]
[Image: You are not allowed to view links. Register or Login to view.]
Advanced Mask Editor:
[Image: You are not allowed to view links. Register or Login to view.]
Results of edited mask training + merging:
[Image: You are not allowed to view links. Register or Login to view.]
FANseg conversion - Obstructions no longer an issue!
[Image: You are not allowed to view links. Register or Login to view.]
Models Types:
H64 (2GB+): 64x64 face resolution, which is used in the original FakeApp or FaceSwap app, but this model in DeepFaceLab uses TensorFlow 1.8 DSSIM Loss function and separated mask decoder and better ConverterMasked. For 2GB and 3GB VRAM model works in reduced mode. This is also a good option for straight face-on scenes.
H64 example: Robert Downey Jr
[Image: You are not allowed to view links. Register or Login to view.]
[Image: You are not allowed to view links. Register or Login to view.]
H128 (3GB+): Same as above; however the resolution is improved 128x128 which conserves better face details, and will perform better with higher resolution videos and close-up shots. For 3GB and 4GB VRAM model works in reduced mode. Also great for direct face-on scenes and gives the highest resolution and details. Best option for Asian faces because of their relatively flat faces and even lighting on clear skin.
H128 example: Nicholas Cage
[Image: You are not allowed to view links. Register or Login to view.]
H128 example: Asian face on blurry target
[Image: You are not allowed to view links. Register or Login to view.]
[Image: You are not allowed to view links. Register or Login to view.]
DF (5+GB): dfaker model. Has 128x128 resolution with a full face model. When using this model, it is recommended not to mix src faces with different lighting conditions. Great for side faces but provides lower resolution and detail. This model covers a more "full" face which often expands and covers more areas of the cheeks. It keeps the face unmorphed giving a convincing face swap; however the dst face will have to have a similar shape.
DF example: Nicholas Cage
[Image: You are not allowed to view links. Register or Login to view.]
LIAEF (5GB+): New model which combines DF, IAE and experiments. The model tries to morph the src face into dst while keeping the facial features of src face but less aggressive morphine. This model has problems with closed eyes recognition. This model can partially fix dissimilar face shapes, but will result in a less recognizable face.
LIAEF128 example: Nicholas Cage
[Image: You are not allowed to view links. Register or Login to view.]
[Image: You are not allowed to view links. Register or Login to view.]
LIAEF128: Trump to Nicholas Cage example video
[Video: You are not allowed to view links. Register or Login to view.]
LIAEF128YAW (5GB+): Currently testing, but useful for when your src has too many side faces against dst faces. It feeds NN by sorted samples by yaw.
MIAEF128 (5GB+): Same as the model above, but it also tries to match brightness and color features.
This model has been discontinued by the developer
AVATAR (4GB+): non GAN, 256x256 face controlling model.
This model has been discontinued by the developer
AVATAR video example:
[Video: You are not allowed to view links. Register or Login to view.]
SAE (2GB+): Styled AutoEncoder that is similar to LIAEF but with a new face style loss. The SAE model is like a face morpher/stylizer instead to a direct swapper. Since this model is a face morpher, the results often produce unrecognizable results. The model can collapse on some scenes.
SAE example: Nicholas Cage on Trump
[Image: You are not allowed to view links. Register or Login to view.]
[Image: You are not allowed to view links. Register or Login to view.]
SAE example: Asian kpop star
[Image: You are not allowed to view links. Register or Login to view.]
SAE example: Alexei Navalny
[Image: You are not allowed to view links. Register or Login to view.]
SAE example: Nicholas Cage in obstructed magneto helmet
[Image: You are not allowed to view links. Register or Login to view.]
[Image: You are not allowed to view links. Register or Login to view.]
SAE model example of Cage-Trump:
[Video: You are not allowed to view links. Register or Login to view.]
General Overview of how DeepFaceLab Works:
Main Concept:
Taking the original dst face, aligning the predicted src face, and creating a masked area to swap or overlay the src face.
[Image: You are not allowed to view links. Register or Login to view.]
Convert Modes:
[Image: You are not allowed to view links. Register or Login to view.]
[Image: You are not allowed to view links. Register or Login to view.]
Convert Options:
Use predicted mask? (yes/no): (default = yes)
[Image: You are not allowed to view links. Register or Login to view.]
Erosion (-100 to +100): (default = 0)
A negative erosion number will essentially increase the area of the src face when converting onto the dst face. A positive number is "eroding" the src face, which reduces the are of the src face when converting onto the dst face.
[Image: You are not allowed to view links. Register or Login to view.]
Seamless Erosion (0 to 40): (default = 0)
Similar to the description above for erosion, but in seamless mode.
[Image: You are not allowed to view links. Register or Login to view.]
Blur (-200 to +200): (default = 0)
A negative blue will make the boarder of the cropped faceset more defined (a sharper line). This will make it look like you literally cut and pasted your src onto your dst face. Adding a positive blur will essentially blur or smooth the transition of the src face onto the dst face, making the boarder less noticeable.
[Image: You are not allowed to view links. Register or Login to view.]
Hist-match threshold (0 to 255): (default = 255)
This option will only be available if you select the hist-match mode. The default threshold is 255 which can cause some highlights to be blown out. Modifying the histogram is essentially adjusting the darks and the lights. A higher threshold here will allow a wider dynamic range, often causing highlights to be blown (bright white). A lower threshold will crush the whites, dulling brightness.
[Image: You are not allowed to view links. Register or Login to view.]
Face Scale (-50 to +50): (default = 0)
A negative face scale number will shrink your src face proportionally towards the center of the dst face. Adding a positive face scale will enlarge your src face.
[Image: You are not allowed to view links. Register or Login to view.]
Transfer Color from predicted face? (LCT/RCT/no): (default = no)
Selecting no will keep the original color of your src faceset. Depending on where you got your src videos and images from to create your faceset, it may have different skin tones compared to the rest of the dst face color. Choosing LCT or RCT method to transfer color may make skin tones more similar and realistic.
[Image: You are not allowed to view links. Register or Login to view.]
Degrade Color Power of Final Image: (default = 0)
Adding a positive number will cause the colors of the final converted image to be less "intense" usually making the video quality look more vintage and not as vibrant in colors.
[Image: You are not allowed to view links. Register or Login to view.]
Tips to creating a good DeepFake- A narrow src face is better for deepfakes compared to a wide face.
- Choose the correct model for your scene - Each model has it's advantages and disadvantages depending on the particular scene you're trying to make a deepfake with. Some models may work better with less powerful GPUs. See each model description above for recommended scenes, and vRAM suggestions. See descriptions above in the model section for a quick explanation.
- Quality over quantity - Using a src faceset with quality images will give you better results. In general, try to keep only clear images with no obstructions of the src face unless it is a very odd angle that you do not have a lot of images of. Try to delete duplicates and blurry images.
- Use less images if possible - The smaller amount of images you use for both src and dst while training, the faster the process is. Be sure to have enough images and facial expressions to cover the entire scene. Typically a src faceset of 1000-5000 is enough. Any more and you're likely adding training time without benefits. You can also work will <1000 photos as well, but results may vary.
- Generally, the longer the trained model, the better the results.
DeepFaceLab Versions Changelog:
Code: == 10.05.2019 ==
fixed work of model pretraining
== 08.05.2019 ==
SAE: added new option
Apply random color transfer to src faceset? (y/n, ?:help skip:%s) :
Increase variativity of src samples by apply LCT color transfer from random dst samples.
It is like 'face_style' learning, but more precise color transfer and without risk of model collapse,
also it does not require additional GPU resources, but the training time may be longer, due to the src faceset is becoming more diverse.
== 05.05.2019 ==
OpenCL: SAE model now works properly
== 05.03.2019 ==
fixes
SAE: additional info in help for options:
Use pixel loss - Enabling this option too early increases the chance of model collapse.
Face style power - Enabling this option increases the chance of model collapse.
Background style power - Enabling this option increases the chance of model collapse.
== 05.01.2019 ==
SAE: added option 'Pretrain the model?'
Pretrain the model with large amount of various faces.
This technique may help to train the fake with overly different face shapes and light conditions of src/dst data.
Face will be look more like a morphed. To reduce the morph effect,
some model files will be initialized but not be updated after pretrain: LIAE: inter_AB.h5 DF: encoder.h5.
The longer you pretrain the model the more morphed face will look. After that, save and run the training again.
== 04.28.2019 ==
fix 3rd pass extractor hang on AMD 8+ core processors
Converter: fixed error with degrade color after applying 'lct' color transfer
added option at first run for all models: Choose image for the preview history? (y/n skip:n)
Controls: [p] - next, [enter] - confirm.
fixed error with option sort by yaw. Remember, do not use sort by yaw if the dst face has hair that covers the jaw.
== 04.24.2019 ==
SAE: finally the collapses were fixed
added option 'Use CA weights? (y/n, ?:help skip: %s ) :
Initialize network with 'Convolution Aware' weights from paper https://arxiv.org/abs/1702.06295.
This may help to achieve a higher accuracy model, but consumes a time at first run.
== 04.23.2019 ==
SAE: training should be restarted
remove option 'Remove gray border' because it makes the model very resource intensive.
== 04.21.2019 ==
SAE:
fix multiscale decoder.
training with liae archi should be restarted
changed help for 'sort by yaw' option:
NN will not learn src face directions that don't match dst face directions. Do not use if the dst face has hair that covers the jaw.
== 04.20.2019 ==
fixed work with NVIDIA cards in TCC mode
Converter: improved FAN-x masking mode.
Now it excludes face obstructions such as hair, fingers, glasses, microphones, etc.
example https://i.imgur.com/x4qroPp.gifv
It works only for full face models, because there were glitches in half face version.
Fanseg is trained by using manually refined by MaskEditor >3000 various faces with obstructions.
Accuracy of fanseg to handle complex obstructions can be improved by adding more samples to dataset, but I have no time for that :(
Dataset is located in the official mega.nz folder.
If your fake has some complex obstructions that incorrectly recognized by fanseg,
you can add manually masked samples from your fake to the dataset
and retrain it by using --model DEV_FANSEG argument in bat file. Read more info in dataset archive.
Minimum recommended VRAM is 6GB and batch size 24 to train fanseg.
Result model\FANSeg_256_full_face.h5 should be placed to DeepFacelab\facelib\ folder
Google Colab now works on Tesla T4 16GB.
With Google Colaboratory you can freely train your model for 12 hours per session, then reset session and continue with last save.
more info how to work with Colab: https://github.com/chervonij/DFL-Colab
== 04.07.2019 ==
Extractor: added warning if aligned folder contains files that will be deleted.
Converter subprocesses limited to maximum 6
== 04.06.2019 ==
added experimental mask editor.
It is created to improve FANSeg model, but you can try to use it in fakes.
But remember: it does not guarantee quality improvement.
usage:
run 5.4) data_dst mask editor.bat
edit the mask of dst faces with obstructions
train SAE either with 'learn mask' or with 'style values'
Screenshot of mask editor: https://i.imgur.com/SaVpxVn.jpg
result of training and merging using edited mask: https://i.imgur.com/QJi9Myd.jpg
Complex masks are harder to train.
SAE:
previous SAE model will not work with this update.
Greatly decreased chance of model collapse.
Increased model accuracy.
Residual blocks now default and this option has been removed.
Improved 'learn mask'.
Added masked preview (switch by space key)
Converter:
fixed rct/lct in seamless mode
added mask mode (6) learned*FAN-prd*FAN-dst
changed help message for pixel loss:
Pixel loss may help to enhance fine details and stabilize face color. Use it only if quality does not improve over time.
fixed ctrl-c exit in no-preview mode
== 03.31.2019 ==
Converter: fix blur region of seamless.
== 03.30.2019 ==
fixed seamless face jitter
removed options Suppress seamless jitter, seamless erode mask modifier.
seamlessed face now properly uses blur modifier
added option 'FAN-prd&dst' - using multiplied FAN prd and dst mask,
== 03.29.2019 ==
Converter: refactorings and optimizations
added new option
Apply super resolution? (y/n skip:n) : Enhance details by applying DCSCN network.
before/after gif - https://i.imgur.com/jJA71Vy.gif
== 03.26.2019 ==
SAE: removed lightweight encoder.
optimizer mode now can be overriden each run
Trainer: the loss line now shows the average loss values after saving
Converter: fixed bug with copying files without faces.
XNViewMP : updated version
fixed cut video.bat for paths with spaces
== 03.24.2019 ==
old SAE model will not work with this update.
Fixed bug when SAE can be collapsed during a time.
SAE: removed CA weights and encoder/decoder dims.
added new options:
Encoder dims per channel (21-85 ?:help skip:%d)
More encoder dims help to recognize more facial features, but require more VRAM. You can fine-tune model size to fit your GPU.
Decoder dims per channel (11-85 ?:help skip:%d)
More decoder dims help to get better details, but require more VRAM. You can fine-tune model size to fit your GPU.
Add residual blocks to decoder? (y/n, ?:help skip:n) :
These blocks help to get better details, but require more computing time.
Remove gray border? (y/n, ?:help skip:n) :
Removes gray border of predicted face, but requires more computing resources.
Extract images from video: added option
Output image format? ( jpg png ?:help skip:png ) :
PNG is lossless, but produces images with size x10 larger than JPG.
JPG extraction is faster, especially on HDD instead of SSD.
== 03.21.2019 ==
OpenCL build: fixed, now works on most video cards again.
old SAE model will not work with this update.
Fixed bug when SAE can be collapsed during a time
Added option
Use CA weights? (y/n, ?:help skip: n ) :
Initialize network with 'Convolution Aware' weights.
This may help to achieve a higher accuracy model, but consumes time at first run.
Extractor:
removed DLIB extractor
greatly increased accuracy of landmarks extraction, especially with S3FD detector, but speed of 2nd pass now slower.
From this point on, it is recommended to use only the S3FD detector.
before https://i.imgur.com/SPGeJCm.gif
after https://i.imgur.com/VmmAm8p.gif
Converter: added new option to choose type of mask for full-face models.
Mask mode: (1) learned, (2) dst, (3) FAN-prd, (4) FAN-dst (?) help. Default - 1 :
Learned – Learned mask, if you choose option 'Learn mask' in model. The contours are fairly smooth, but can be wobbly.
Dst – raw mask from dst face, wobbly contours.
FAN-prd – mask from pretrained FAN model from predicted face. Very smooth not shaky countours.
FAN-dst – mask from pretrained FAN model from dst face. Very smooth not shaky countours.
Advantages of FAN mask: you can get a not wobbly shaky without learning it by model.
Disadvantage of FAN mask: may produce artifacts on the contours if the face is obstructed.
== 03.13.2019 ==
SAE: added new option
Optimizer mode? ( 1,2,3 ?:help skip:1) :
this option only for NVIDIA cards. Optimizer mode of neural network.
1 - default.
2 - allows you to train x2 bigger network, uses a lot of RAM.
3 - allows you to train x3 bigger network, uses huge amount of RAM and 30% slower.
Epoch term renamed to iteration term.
added showing timestamp in string of training in console
== 03.11.2019 ==
CUDA10.1AVX users - update your video drivers from geforce.com site
face extractor:
added new extractor S3FD - more precise, produces less false-positive faces, accelerated by AMD/IntelHD GPU (while MT is not)
speed of 1st pass with DLIB significantly increased
decreased amount of false-positive faces for all extractors
manual extractor: added 'h' button to hide the help information
fix DFL conflict with system python installation
removed unwanted tensorflow info from console log
updated manual_ru
== 03.07.2019 ==
fixes
upgrade to python 3.6.8
Reorganized structure of DFL folder. Removed unnecessary files and other trash.
Current available builds now:
DeepFaceLabCUDA9.2SSE - for NVIDIA cards up to GTX10x0 series and any 64-bit CPU
DeepFaceLabCUDA10.1AVX - for NVIDIA cards up to RTX and CPU with AVX instructions support
DeepFaceLabOpenCLSSE - for AMD/IntelHD cards and any 64-bit CPU
== 03.04.2019 ==
added
4.2.other) data_src util recover original filename.bat
5.3.other) data_dst util recover original filename.bat
== 03.03.2019 ==
Convertor: fix seamless
== for older changelog see github page ==
|
