MrDeepFakes Forums

Some content may not be available to Guests. Consider registering an account to enjoy unrestricted access to guides, support and tools

  • We are looking for community members who are intested in helping out. See our HELP WANTED post.

Color Transfer

motherload1

DF Vagrant
Verified Video Creator
*See attached image*

Using suggestions about converting and color transfer I:
1) Made a backup of my SAE model after good results
2) Started training the same model with 'apply_random_ct' enabled for ~5hrs
3) Converting using Overlay and'apply_random_ct', I get horrible color mismatch (see image)
4) Try converting using Overlay and 'apply_random_ct', same deal
4) Okay fine, converting with Overlay and no color transfer, same as before
5) Take my backup model w/o the 'apply_random_ct' training
7) Run conversion on overlay and no color transfer, similar results
     Colors are less sharp/bright but still messed up
8) Convert using Seamless and no color transfer
     Pretty good results some 'mixed colors/green faces' on face side angles, but not too bad
     Problem with seamless is flickering, bright and dark faces, sometimes different parts of the face in the same frame

What did I do wrong? All but the seamless conversion made the girl look like clown, but I don't like the seamless because of reasons stated above.
On my last video using overlay (don't remember if I used RCT or not. but I didn't train with 'apply_random_ct' enabled) the colors were fine.
Is there anyway to fix this or is this just how it's going to be? 

Thank you for taking the time to read this and help if you can.

-motherload10

a2Fi8zHh.png
 

easy40

DF Admirer
Verified Video Creator
Not really sure what you mean when you say you did 5 hrs of RCT training, that is not even a thing. Do you mean that you used the "apply_random_ct" option?
 

motherload1

DF Vagrant
Verified Video Creator
easy40 said:
Not really sure what you mean when you say you did 5 hrs of RCT training, that is not even a thing. Do you mean that you used the  "apply_random_ct" option?

yes, sorry. I meant "apply_random_ct"
I now think that 5 hrs with it enabled is probably too long but it doesnt explain the model without "apply_random_ct" problems
 

easy40

DF Admirer
Verified Video Creator
motherload10 said:
easy40 said:
Not really sure what you mean when you say you did 5 hrs of RCT training, that is not even a thing. Do you mean that you used the  "apply_random_ct" option?

yes, sorry. I meant "apply_random_ct"
I now think that 5 hrs with it enabled is probably too long but it doesnt explain the model without "apply_random_ct" problems

You should be able to leave the apply_random_ct as long as you wish without having any problems. This looks like a problem with the conversion. You say that you are not using RCT or LCT to convert, but for some reason, the results look like they are applying the color transfer.

Your source dataset model has a ton of makeup?
 

motherload1

DF Vagrant
Verified Video Creator
easy40 said:
motherload10 said:
easy40 said:
Not really sure what you mean when you say you did 5 hrs of RCT training, that is not even a thing. Do you mean that you used the  "apply_random_ct" option?

yes, sorry. I meant "apply_random_ct"
I now think that 5 hrs with it enabled is probably too long but it doesnt explain the model without "apply_random_ct" problems

You should be able to leave the apply_random_ct as long as you wish without having any problems. This looks like a problem with the conversion. You say that you are not using RCT or LCT to convert, but for some reason, the results look like they are applying the color transfer.

Your source dataset model has a ton of makeup?

No, the src data set is the same that i used before. I only changed the dst data (same model though), only differences in terms of makeup is lipstick in the new dst faceset. Lighting is similar too, so thats not an issue.
 

easy40

DF Admirer
Verified Video Creator
motherload10 said:
easy40 said:
motherload10 said:
easy40 said:
Not really sure what you mean when you say you did 5 hrs of RCT training, that is not even a thing. Do you mean that you used the  "apply_random_ct" option?

yes, sorry. I meant "apply_random_ct"
I now think that 5 hrs with it enabled is probably too long but it doesnt explain the model without "apply_random_ct" problems

You should be able to leave the apply_random_ct as long as you wish without having any problems. This looks like a problem with the conversion. You say that you are not using RCT or LCT to convert, but for some reason, the results look like they are applying the color transfer.

Your source dataset model has a ton of makeup?

No, the src data set is the same that i used before. I only changed the dst data (same model though), only differences in terms of makeup is lipstick in the new dst faceset. Lighting is similar too, so thats not an issue.

It definitely looks like some color transfer is being done. Use the convert sae debug option and see exactly what is going on with the mask.
 

motherload1

DF Vagrant
Verified Video Creator
easy40 said:
motherload10 said:
easy40 said:
motherload10 said:
easy40 said:
Not really sure what you mean when you say you did 5 hrs of RCT training, that is not even a thing. Do you mean that you used the  "apply_random_ct" option?

yes, sorry. I meant "apply_random_ct"
I now think that 5 hrs with it enabled is probably too long but it doesnt explain the model without "apply_random_ct" problems

You should be able to leave the apply_random_ct as long as you wish without having any problems. This looks like a problem with the conversion. You say that you are not using RCT or LCT to convert, but for some reason, the results look like they are applying the color transfer.

Your source dataset model has a ton of makeup?

No, the src data set is the same that i used before. I only changed the dst data (same model though), only differences in terms of makeup is lipstick in the new dst faceset. Lighting is similar too, so thats not an issue.

It definitely looks like some color transfer is being done. Use the convert sae debug option and see exactly what is going on with the mask.
Okay, I'll get back to you tomorrow with what happens. Thanks for your help so far.
 

motherload1

DF Vagrant
Verified Video Creator
motherload10 said:
easy40 said:
motherload10 said:
easy40 said:
motherload10 said:
yes, sorry. I meant "apply_random_ct"
I now think that 5 hrs with it enabled is probably too long but it doesnt explain the model without "apply_random_ct" problems

You should be able to leave the apply_random_ct as long as you wish without having any problems. This looks like a problem with the conversion. You say that you are not using RCT or LCT to convert, but for some reason, the results look like they are applying the color transfer.

Your source dataset model has a ton of makeup?

No, the src data set is the same that i used before. I only changed the dst data (same model though), only differences in terms of makeup is lipstick in the new dst faceset. Lighting is similar too, so thats not an issue.

It definitely looks like some color transfer is being done. Use the convert sae debug option and see exactly what is going on with the mask.
Okay, I'll get back to you tomorrow with what happens. Thanks for your help so far.

Tried debug w/o color transfer and I didn't get clown colors this time. I got a brighter face but at least this is something I can work with. So I don't know what was going on before; I'll try with RCT again and see what happens. Thank you for your help. 
XAEk7Xwh.png
 

easy40

DF Admirer
Verified Video Creator
motherload10 said:
motherload10 said:
easy40 said:
motherload10 said:
easy40 said:
You should be able to leave the apply_random_ct as long as you wish without having any problems. This looks like a problem with the conversion. You say that you are not using RCT or LCT to convert, but for some reason, the results look like they are applying the color transfer.

Your source dataset model has a ton of makeup?

No, the src data set is the same that i used before. I only changed the dst data (same model though), only differences in terms of makeup is lipstick in the new dst faceset. Lighting is similar too, so thats not an issue.

It definitely looks like some color transfer is being done. Use the convert sae debug option and see exactly what is going on with the mask.
Okay, I'll get back to you tomorrow with what happens. Thanks for your help so far.

Tried debug w/o color transfer and I didn't get clown colors this time. I got a brighter face but at least this is something I can work with. So I don't know what was going on before; I'll try with RCT again and see what happens. Thank you for your help. 
XAEk7Xwh.png

Yes, it looks much much better. Try rct color transfer in debug mode. I suspect that you will get the right colors this time.
 

motherload1

DF Vagrant
Verified Video Creator
easy40 said:
motherload10 said:
motherload10 said:
easy40 said:
motherload10 said:
No, the src data set is the same that i used before. I only changed the dst data (same model though), only differences in terms of makeup is lipstick in the new dst faceset. Lighting is similar too, so thats not an issue.

It definitely looks like some color transfer is being done. Use the convert sae debug option and see exactly what is going on with the mask.
Okay, I'll get back to you tomorrow with what happens. Thanks for your help so far.

Tried debug w/o color transfer and I didn't get clown colors this time. I got a brighter face but at least this is something I can work with. So I don't know what was going on before; I'll try with RCT again and see what happens. Thank you for your help. 
XAEk7Xwh.png

Yes, it looks much much better. Try rct color transfer in debug mode. I suspect that you will get the right colors this time.

Much better.
4FLiVWsh.png
 

dpfks

DF Enthusiast
Staff member
Administrator
Verified Video Creator
Wait how did you fix this exactly? Did you revert back to no RCT while training and convert with rct in debug?

I notice I had this issue when greens are in background of faceset (or reds)
 

motherload1

DF Vagrant
Verified Video Creator
dpfks said:
Wait how did you fix this exactly? Did you revert back to no RCT while training and convert with rct in debug?

I notice I had this issue when greens are in background of faceset (or reds)

I discarded the model with color transfer training and used the backup model. Ran the debug w/ no RCT, the colors were fine. Then I ran the convert w/o no RCT, the colors were the same as in the debug. Turned on RCT in converting (didn't do the debug) to fix the skin tone issues. 
I have no idea why I was having the weird color issues before (with the model before color transfer training). I really didn't do anything other than run the debug and then convert; no training was done, only variable I changed was running the debug w/o RCT and the converting w/ it.
 
Top