Is this guy using some kind of modified DFL or is he faking the DeepFake

[TheFakeGuy]

So I came across this instagram post. And yeah the first few seconds have nothing new for us. But the moment he shows the morphing and face points is where he really gets me

https://www.instagram.com/p/B7hIeOVnQ3N/

Is he just deepfaking a blank 3d model in blender or some other 3d software? I mean rendering a blank face where you can rig the face and then deepfaking that same render with DeNiro?

And later he morphs the light effects on his background. How would you get that? Same process as before? Changing the light of a 3dmodel and then deepfaking that model with DeNiro?

Let's see if you guys have any better ideas

 

[TMBDF]

I think it is like you say, someone pointed it out in comments too, he is just faking his face onto a rigged 3D face model, at the end the comparison isn't really between faked and original face but rather. Or it could be some kind of real time approach (we've seen some already that let you manipulate face in real time).
Here you have an example of real time face reenactment:
And here manipulation:

 

[TheFakeGuy]

Yeah I thought about it for a while and I think the guy either downloaded a video where someone is deforming a 3d rigged face, or he had one rigged himself, and then deepfaked that video of the 3d rigged face.

Because those videos you posted aren't where someone changes lips movement to a video aren't available to the general public, the software or source code they use I mean. And I don't think a single guy could develop all that by himself, unless he had a team of researchers.

 

[TMBDF]

Yeah I thought about it for a while and I think the guy either downloaded a video where someone is deforming a 3d rigged face, or he had one rigged himself, and then deepfaked that video of the 3d rigged face.

Because those videos you posted aren't where someone changes lips movement to a video aren't available to the general public, the software or source code they use I mean. And I don't think a single guy could develop all that by himself, unless he had a team of researchers.

Well, we can only speculate.

 

[Fattee]

I think that video is using AI learning in Autodesk Flame https://www.autodesk.com/products/flame/overview

 

[TMBDF]

I think that video is using AI learning in Autodesk Flame https://www.autodesk.com/products/flame/overview

Can you point to the exact feature or do you mean he is just using some plugin for Maya that is integrated with Flame (Maya is a 3D software and as pretty much all of them features some sort of way of controlling rigs for 3D models).

 

[Fattee]

Maya is a separate package to flame. flame has some AI learning built into it, I do't really have much info on it.

 

[TMBDF]

Maya is a separate package to flame. flame has some AI learning built into it,  I do't really have much info on it.

I see something about machine learned normal maps for relighting, but that has functionality nowhere near what that video shows, I was referring to possibility that this is just using some kind of rig addon or built in feature of Maya and Flame simply has integration with it and you may have saw some demo of Flame with that face rig but in that video someone is simply overlaying faked face over it.

Either way it's not available and probably doesn't exist or is a part of some research like many we see nowadays, with no public code or any clear plans on how it's gonna be used (and knowing the industry and it's capitalistic ways - probably some paid product).