MrDeepFakes Forums

Some content may not be available to Guests. Consider registering an account to enjoy unrestricted access to guides, support and tools

  • We are looking for community members who are intested in helping out. See our HELP WANTED post.

Emma Watson - On Couch

HaDenG

DF Admirer
Verified Video Creator
Hey everybody,
I have been testing the new SAE model from DFL. It is awesome! Finally, no more bloody eyebrows problem and the face now blends in perfectly. Set face style and background style 10.0, after having artifacts on previews then set 0.01 and 0.3-0.1 also use pixel loss after 50k epochs.
If anyone interested I can share options I used and some other tips.
Enjoy

https://mrdeepfakes.com/video/2585/emma-watson-on-couch
 

dpfks

DF Enthusiast
Staff member
Administrator
Verified Video Creator
Thanks for the tips, I will be testing it for future models now.
 

za123za

DF Vagrant
Thank you very much HaDenG,I fucking love it,will you create Daisy Ridley with this scene? I think she is perfect.
 

kenjathyme

DF Vagrant
HaDenG said:
Hey everybody,
I have been testing the new SAE model from DFL. It is awesome! Finally, no more bloody eyebrows problem and the face now blends in perfectly. Set face style and background style 10.0, after having artifacts on previews then set 0.01 and 0.3-0.1 also use pixel loss after 50k epochs.
If anyone interested I can share options I used and some other tips.
Enjoy

https://mrdeepfakes.com/video/2585/emma-watson-on-couch

Wow that's great. Since which build has the SAE model been available from DFL? It's been a whlie since I've downloaded from the repo. I see that its there's a separate download file "SAE_encoder df-512-42.h5" but how do you start training this type of model and use "overlay" conversion?
 

HaDenG

DF Admirer
Verified Video Creator
I see some people who are interested so I'm gonna share the full procedure and tips. This is going to be for DFL prebuild binaries version.

1. Extract frames as usual. For detecting landmarks use MT not dlib. MT produces less jittery results. Then sort by hist and delete false positive frames.

2. Even though extracted face looks okay its landmarks may still be wrong. Look for the folder named aligned_debug in data_dst. Check images and delete falsely detected landmarks then manually extract those. Check the video:

3. Start training for SAE but we are going to change some options. Set random_flip false if you have quality src face-set. What random_flip does is flipping faces on x-axis so if data set lacks some positions this option helps you but result face will look unnatural because human face is not symmetrical. If dst faces are less then src set yaw_feed true. But I don't recommend it.

4. Start training and check preview window if you spot artifacts on the faces (some weird colors) stop training and set face_style 0.01 and background style 0.3-0.1. Enable pixel_loss after 40k-50k epochs.

5. If you are happy with the faces shown on preview stop training and convert. On repo it's recommended using overlay option but in my case colors didn't matched so I used default seamless option. Erode,blur 0, seamless_erode 10, others default.
That's it.

If you have more questions I can answer them
 

Pocketspeed

DF Admirer
Verified Video Creator
@HaDenG  This is good information!  I don't use DFL, but this is very useful stuff.  Thank you for sharing!
 

HaDenG

DF Admirer
Verified Video Creator
halo2k said:
Why do you recommend SAE over H128?

H128 is a half-face model that only replaces middle of the face while DF and DF based SAE are full-face models that replace cheeks and jaw which result more similar faces to src. Different from DF, SAE handles obstructions as well. Check the image below (from repo) . Actually SAE contains all models. You will be asked some options when training starts for the first time. If you want to continue using H128, you can still use SAE.

https://github.com/iperov/DeepFaceLab/blob/master/doc/SAE_Cage_2.jpg
 

joelatoundra

DF Vagrant
when I try SAE training It doesn't work.

I have this message :
2019-02-03 03:24:57.650232: E tensorflow/stream_executor/cuda/cuda_blas.cc:464] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
2019-02-03 03:25:02.845292: E tensorflow/stream_executor/cuda/cuda_dnn.cc:353] Could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED

what is the problem ?
 

halo2k

DF Pleb
Verified Video Creator
HaDenG said:
halo2k said:
Why do you recommend SAE over H128?

H128 is a half-face model that only replaces middle of the face while DF and DF based SAE are full-face models that replace cheeks and jaw which result more similar faces to src. Different from DF, SAE handles obstructions as well. Check the image below (from repo) . Actually SAE contains all models. You will be asked some options when training starts for the first time. If you want to continue using H128, you can still use SAE.

https://github.com/iperov/DeepFaceLab/blob/master/doc/SAE_Cage_2.jpg

I tried SAE already and with obstructed faces, the result is not good, check my Reply here:
https://mrdeepfakes.com/forums/thread-deepfacelab-explained-and-usage-tutorial?pid=2741#pid2741

how do you solve this issue using SAE?
 

nephashi

DF Vagrant
Hi can you share your parameter setting? such as batch size, autoencoder dimension, encoder/decoder dimension?
 

HaDenG

DF Admirer
Verified Video Creator
nephashi said:
Hi can you share your parameter setting? such as batch size, autoencoder dimension, encoder/decoder dimension?

All default. I have gtx 1080 which has 8GB memory. In repo there are special settings for 11GB cards though. Also SAE architecture is DF, feed_yaw false, random_flip false.


halo2k said:
HaDenG said:
halo2k said:
Why do you recommend SAE over H128?

H128 is a half-face model that only replaces middle of the face while DF and DF based SAE are full-face models that replace cheeks and jaw which result more similar faces to src. Different from DF, SAE handles obstructions as well. Check the image below (from repo) . Actually SAE contains all models. You will be asked some options when training starts for the first time. If you want to continue using H128, you can still use SAE.

https://github.com/iperov/DeepFaceLab/blob/master/doc/SAE_Cage_2.jpg

I tried SAE already and with obstructed faces, the result is not good, check my Reply here:
https://mrdeepfakes.com/forums/thread-deepfacelab-explained-and-usage-tutorial?pid=2741#pid2741

how do you solve this issue using SAE?

I didn't work with hard obstructions but I don't think it works like it's advertised. If you keep feeding the model with same obstructions like in the image I shared it handles them but if you have various obstructions like hands in couple frames, hair in another and unobstructed frames in one dst set, SAE can't recreate those frames like originals, resulted frames look like H128 obstructed frames which is better than original DF though.
 

nephashi

DF Vagrant
Hi, I am using similar setting with you, but my src face is white while dst is darker, and the resulting color doesn't match. I tried turn up face power but it lead to weird color. Do you have same problem? or you prepare training data with similar color?
 

chortlemortle

DF Vagrant
HaDenG said:
Hey everybody,
I have been testing the new SAE model from DFL. It is awesome! Finally, no more bloody eyebrows problem and the face now blends in perfectly. Set face style and background style 10.0, after having artifacts on previews then set 0.01 and 0.3-0.1 also use pixel loss after 50k epochs.
If anyone interested I can share options I used and some other tips.
Enjoy

https://mrdeepfakes.com/video/2585/emma-watson-on-couch

H128 is working perfectly for me but I am having trouble implementing SAE. Can you walk through your settings step by step and including at what epochs you alter them? Hoping to get a 256 res SAE going, just got an rtx2080.

Thanks!
 
Top