MrDeepFakes Forums

Some content may not be available to Guests. Consider registering an account to enjoy unrestricted access to guides, support and tools

  • We are looking for community members who are intested in helping out. See our HELP WANTED post.

AVATAR model usage question

fredd_a_stare

DF Vagrant
I would like to experiment with Avatar model. Used SAE and DF a lot previously.

The only instructions I found are very appreciated but sparse. 

As I understand, there are 2 stages to the training. But there is no mention of any changes to be made to dst or src folders between the stages. 
So you just run 1 stage until happy, and then run the 2nd stage (with batch size that your gpu can handle).

In the instructions, it is stated that you need 10 - 20 minutes of footage -  frames for the src side. 
Here is where my question comes: If i am putting 10 minutes worth of frames into the src, and about 1 minute of footage into the dst (the "puppteer" face, if you will) then which of the src frames does get "merged" at the end? I know from using other models that normally the dst side is merged, and not the src side. But in the AVATAR model case, I assumed that the frames of the person you are trying to manipulate are the ones that would be "merged" for the final output. I am missing information as to how this works and therefore not able to compute:)) The instructions are hard to understand.

Any help appreciated!

Couldnt find anything on the forum so far.
 

fredd_a_stare

DF Vagrant
So far from trying the different types of avatar model, I learned that the merging process takes the frames of the person doing the "puppeteering", in the DST folder, and creates a corresponding sequence using the SRC face/head (the person u are trying to manipulate). The more frames with more facial expressions that you have in the SRC, the better chance that the result of the merge will be good. There is no relationship of chronology between the SRC and DST - to answer my own question above. SRC is used only as bank of data to provide the necessary facial expressions to build the model and resulting merge. 

I have not seen a built in feature which will then re-position the new, merged face, into an original video. All I am able to get upon merging is a stabilized, centered on the nose, square of the head/face/head and chest (respectively depending on type used). -So the only way to composite the new face back into the original video is to do manual tracking and attachment + manual animation in post... ?

It would be a great help if someone who knows the process can chime in here and let me know if this is correct or I missed a major step.. 

Many thanks
 
Top