Mr DeepFakes Forums
  • New and improved dark forum theme!
  • Guests can now comment on videos on the tube.
   
godinshacklesEye detection issues. I found a very slow trick. Looking for a better way.
#1
Hi everyone,
so I've been struggling to get proper results. But recently, I've done some breakthroughs. You can skip further for the part about the eyes. It'll be underlined.

First and foremost, I've been training a SAE model at 176 with Google Colab. Quite happy with the results. Been changing my faceset as I observed that having many different types of lighting and makeup messes with my results. If you look up on Google, Wired has some great interviews with a white background and front/side angles.

Still struggling to find hard angles for the SRC (head looking at the ceiling) so I'm focusing on POV's that match my angles (man, this has got to be the part I hate the most. Fishing for a proper DST scene that I won't have to tamper too much with).

But I digress and all in all, pretty satisfied with my recent results.

EYE DETECTION PART

Now, one thing that really messes up my results is eye detection. I'm all about learning new ways and have recently seen that there are ways to work my trick with Premiere and After Effects. I'll look into that.

But anyway, in the meantime, my process has been the following. I've been doing a "test render" to see if my eyes are detected properly. When they are not, I used photoshop to turn the eyes of the DST extremely blue like this to force DFL to detect them:

[Image: xM9dTLIh.png]

This is a slow process but obviously, maybe 10 frames need to be done for about 150. Still looking for a faster way. Anyway, this boosted my eye detection from this (no blue eyes) to this (blue eyes):


[Image: yF6SaT1h.png]


I thought I'd share this. See if it can help anyone but also so that you guys could give me ideas to make it faster and easier?

My model is at 116K but I changed my whole SRC faceset. Doesn't really count. I'll probably still train it for a while but wanted to see if anything could be done about the eyes. Still tweaking, finding the weaknesses of the DST and working on making the results as good as possible so this is far from finished.


Thanks in advance.
#2
That's a neat trick, but also really time consuming. I think video editors can track eye movements, and maybe we can add a color mask while doing the tracking. But I think this is a solid solution for perfectionists who know what areas of the clips are problematic for eyes.

THREAD MOVED TO GUIDES SECTION
~ Fake it till you make it ~
You are not allowed to view links. Register or Login to view.

Forum Jump:

Users browsing this thread: 1 Guest(s)