MrDeepFakes Forums

Some content may not be available to Guests. Consider registering an account to enjoy unrestricted access to guides, support and tools

  • We are looking for community members who are intested in helping out. See our HELP WANTED post.

SAE settings for DFL on collab (colour mismatch)

mrspudtnik

DF Vagrant
Does anyone have a set of good SAE settings for use on collab? I tried using some settings iperov posted and also the defaults but i seem to get really bad colour mismatch on the merged scenes. The same scene and model works fine when using h128
 

dpfks

DF Enthusiast
Staff member
Administrator
Verified Video Creator
What colors are mis-matched? Train with random color transfer on
 

mrspudtnik

DF Vagrant
dpfks said:
What colors are mis-matched? Train with random color transfer on

The src face mask applied to the video doesnt match the colour of the rest of the head. Does applying rct during training really change this? i had it off but applied rct during the merge.
 

easy40

DF Admirer
Verified Video Creator
mrspudtnik said:
dpfks said:
What colors are mis-matched? Train with random color transfer on

The src face mask applied to the video doesnt match the colour of the rest of the head. Does applying rct during training really change this? i had it off but applied rct during the merge.

Show a screenshot to see what you're talking about.
 

mrspudtnik

DF Vagrant
ok so here is the colour mismatch im talking about. rct was off during training and on during the convert process

b9MVLWoh.png
 

dpfks

DF Enthusiast
Staff member
Administrator
Verified Video Creator
Turn it on while training for 30-60 min
 

mrspudtnik

DF Vagrant
finally managed to try this but it made pretty much no difference. Also tried a different model with the same result.

Also tried running seamless during conversion and this didnt work either
 

LCC

DF Pleb
Could you post some samples of your aligned source pictures, or maybe a screenshot of your aligned folder?
 

mrspudtnik

DF Vagrant
So same problem with both facesets below. Each have around 3000 pics. The colour issue does not happen when training with h128

jvCd7ZVh.png
8d2OwzAh.png
 

dpfks

DF Enthusiast
Staff member
Administrator
Verified Video Creator
Which one is your data_dst faceset

Training with random color transfer on makes more of a difference than RCT conversion
 

mrspudtnik

DF Vagrant
dpfks said:
Which one is your data_dst faceset

Training with random color transfer on makes more of a difference than RCT conversion

they are both dst facesets, and both have the same colour mismatch when using sae. Both are fine when training in h128. 

The example colour mismatch posted a few above is using the first faceset i.e. this one 

UCf7z8Hh.jpg
 

dpfks

DF Enthusiast
Staff member
Administrator
Verified Video Creator
Well without actually having the data_src or the data_dst facesets I can't comment/test.

But like I said I've done many deepfakes and training with RCT on works well.

Also convert options you used, or screen shot of training settings, and preview screens...
 

LCC

DF Pleb
The reason I asked to see your aligned folder, was to see the quality of your source photos. Especially to see if a lot were really dark.

The colour seems fine once brightened. The problem is the face is lacking a shitload of contrast.

l6ri3xRh.jpg


What does your source set look like? How many different videos? Are a lot of them low light?
 

mrspudtnik

DF Vagrant
dpfks said:
Well without actually having the data_src or the data_dst facesets I can't comment/test.

But like I said  I've done many deepfakes and training with RCT on works well.

Also convert options you used, or screen shot of training settings, and preview screens...
Do you generally keep rct on during the whole of training or just for an hour or so at the end? I'll try again running for longer with it on to see if that makes any difference. For training, all settings were default except rct for the last hour. Convert was all default settings, with the exception of rct, and i tried both learned and learned*FANB-prd*FAN-dst for the mask mode with the same results. Here is the preview for a differnet video attempt with the same results

7EeUVqGh.jpg



LCC said:
The reason I asked to see your aligned folder, was to see the quality of your source photos. Especially to see if a lot were really dark.

The colour seems fine once brightened. The problem is the face is lacking a shitload of contrast.

l6ri3xRh.jpg


What does your source set look like? How many different videos? Are a lot of them low light?

The source set is around 3200 images. Taken from probably 6 or 7 high res youtube vids. It's actually the opposite, only maybe 100 or so low light images. There are some images in from a scene which essentiallly looks like it would be black and white images, perhaps I should remove these
 
Train longer with rct on. Having some darker images shouldn't matter too much, but the black and white ones might cause issues. If you're only around 90000 iterations, and only turned rct on for the last little while it needs longer to fix color

Also, the predicted face in the preview doesn't line up with dst very well, and the top image looks really weird (probably because of dst hair) so something else is going on in that case. By 90000 src should be matching up with dst alignments a lot better than that
 

mrspudtnik

DF Vagrant
TheMadFaker said:
Train longer with rct on.  Having some darker images shouldn't matter too much, but the black and white ones might cause issues.  If you're only around 90000 iterations, and only turned rct on for the last little while it needs longer to fix color

Also, the predicted face in the preview doesn't line up with dst very well, and the top image looks really weird (probably because of dst hair) so something else is going on in that case.  By 90000 src should be matching up with dst alignments a lot better than that

ok will try this again with rct on for longer. that's interesting you mention how the pics line up. i didnt notice in the preview but yes when this particular set is converted the src face looks too big, but the mask size too small e.g. two sets of eyebrows. think it might be this particular dst video as there are a lot of close ups. not had this provlem in other videos
 

mrspudtnik

DF Vagrant
ok have trained for approx 20 hours on collab with rct on. still no luck.

kyQKpUUh.png
[/img]

added 60 blur setting which has helped the border (see below) but the colour is still off. these were done with rct on during convert. also tried no colour transfer and lct but both were worse than these

HmXucqEh.png
 

titan_rw

DF Pleb
mrspudtnik said:
added 60 blur setting which has helped the border (see below) but the colour is still off. these were done with rct on during convert. also tried no colour transfer and lct but both were worse than these

That looks pretty good.

Try some face styling.  It usually helps match colors.

Some erode would also help hide the edge color mismatch, however it will of course erode all around, including the eyebrows.

I do wish it was possible to erode differently for the top of the face (eyebrows) vs the rest of the face.  I find an erode setting I need for the sides / chin ends up giving me either double eyebrows, or keeping the original dst eyebrows.
 

mrspudtnik

DF Vagrant
titan_rw said:
mrspudtnik said:
added 60 blur setting which has helped the border (see below) but the colour is still off. these were done with rct on during convert. also tried no colour transfer and lct but both were worse than these

That looks pretty good.

Try some face styling.  It usually helps match colors.

Some erode would also help hide the edge color mismatch, however it will of course erode all around, including the eyebrows.

I do wish it was possible to erode differently for the top of the face (eyebrows) vs the rest of the face.  I find an erode setting I need for the sides / chin ends up giving me either double eyebrows, or keeping the original dst eyebrows.

i did try some erode but i think i was syill fixed on the colour mismatch. it has gotten better but I still can't get it to be as good as h128.

will try some face style next time i run - does it need to be set quite high or will say a value of 10-20 make a very noticeable difference?

also, if i was to run sae with the half face model would that give me results more in line with the h128 models?
 
Top