MrDeepFakes Forums
  • New and improved dark forum theme!
  • Guests can now comment on videos on the tube.
   
godinshacklesEye detection issues. I found a very slow trick. Looking for a better way.
#1
Hi everyone,
so I've been struggling to get proper results. But recently, I've done some breakthroughs. You can skip further for the part about the eyes. It'll be underlined.

First and foremost, I've been training a SAE model at 176 with Google Colab. Quite happy with the results. Been changing my faceset as I observed that having many different types of lighting and makeup messes with my results. If you look up on Google, Wired has some great interviews with a white background and front/side angles.

Still struggling to find hard angles for the SRC (head looking at the ceiling) so I'm focusing on POV's that match my angles (man, this has got to be the part I hate the most. Fishing for a proper DST scene that I won't have to tamper too much with).

But I digress and all in all, pretty satisfied with my recent results.

EYE DETECTION PART

Now, one thing that really messes up my results is eye detection. I'm all about learning new ways and have recently seen that there are ways to work my trick with Premiere and After Effects. I'll look into that.

But anyway, in the meantime, my process has been the following. I've been doing a "test render" to see if my eyes are detected properly. When they are not, I used photoshop to turn the eyes of the DST extremely blue like this to force DFL to detect them:

[Image: xM9dTLIh.png]

This is a slow process but obviously, maybe 10 frames need to be done for about 150. Still looking for a faster way. Anyway, this boosted my eye detection from this (no blue eyes) to this (blue eyes):


[Image: yF6SaT1h.png]


I thought I'd share this. See if it can help anyone but also so that you guys could give me ideas to make it faster and easier?

My model is at 116K but I changed my whole SRC faceset. Doesn't really count. I'll probably still train it for a while but wanted to see if anything could be done about the eyes. Still tweaking, finding the weaknesses of the DST and working on making the results as good as possible so this is far from finished.


Thanks in advance.
#2
That's a neat trick, but also really time consuming. I think video editors can track eye movements, and maybe we can add a color mask while doing the tracking. But I think this is a solid solution for perfectionists who know what areas of the clips are problematic for eyes.

THREAD MOVED TO GUIDES SECTION
~ Fake it till you make it ~
You are not allowed to view links. Register or Login to view.
You are not allowed to view links. Register or Login to view.
#3
I just tried this by painting some pupils blue... When I went to train, none of the blue eyes were showing in the training grid. In the batch process area (command prompt dos box) it says "Corrupted JPG file: No Valid JPG info. 00812.jpg is not a dfl image file required for training." I saved it as a jpg with the exact same name as the original file. I then tried erasing the files in the model folder and starting over. Didn't see any images with blue eyes while training and I refreshed it so many times. It said the same thing as in the quotes above. It pretty much doesn't see the files anymore after saving them with the blue eyes. All I did was paint them, close the file and photoshop asked if I wanted to save each frame after and I said yes. I'm sure I'm missing something stupid. If you have an idea, let me know. Thanks!
#4
(08-24-2019, 07:41 AM)kilerb Wrote: You are not allowed to view links. Register or Login to view.I just tried this by painting some pupils blue...  When I went to train, none of the blue eyes were showing in the training grid.  In the batch process area (command prompt dos box) it says "Corrupted JPG file: No Valid JPG info.  00812.jpg is not a dfl image file required for training."  I saved it as a jpg with the exact same name as the original file.  I then tried erasing the files in the model folder and starting over.  Didn't see any images with blue eyes while training and I refreshed it so many times.  It said the same thing as in the quotes above.  It pretty much doesn't see the files anymore after saving them with the blue eyes.  All I did was paint them, close the file and photoshop asked if I wanted to save each frame after and I said yes.  I'm sure I'm missing something stupid.  If you have an idea, let me know.  Thanks!

You have to add the blue eyes, then re-extract the image. You cannot modify the image after extraction has bee ndone.
~ Fake it till you make it ~
You are not allowed to view links. Register or Login to view.
You are not allowed to view links. Register or Login to view.
#5
Tracing might be a good solution but im not sure if it could track the eyeball/iris movements. It's quite very small and not a lot of pixels. I'll might try this in the future, sometimes with wrong eye movements, its seems like the deepfake looks very unreal, and generally not alive. For anyone looking for a program vegas pro could do this and they have a free trial too.

Forum Jump:

Users browsing this thread: 1 Guest(s)