Mr DeepFakes Forums
  • New and improved dark forum theme!
  • Guests can now comment on videos on the tube.
   
Total Likes Received: 5 (0.03 per day | 0.47 percent of total 1065)
(Find All Threads Liked ForFind All Posts Liked For) Total Likes Given: 2 (0.01 per day | 0.19 percent of total 1066)
(Find All Liked ThreadsFind All Liked Posts)

yvanderfel
(DeepFake Producer)
DeepFake Producer

Registration Date: 12-06-2018
Date of Birth: Not Specified
Local Time: 05-22-2019 at 02:46 PM
Status:

yvanderfel's Most Liked Post
Post Subject Numbers of Likes
VR Deepfakes; what's different? 3
Thread Subject Forum Name
VR Deepfakes; what's different? Guides
Post Message
In the last few days I've gotten a few messages from people who want to get started doing VR clips so I thought I'd do a guide and explain to the best of my ability, how the process of making VR differs from regular 2d clips.

keep in mind I only use Deepfacelab.


Making VR deepfakes is exactly the same process with two added steps. Frame splitting before face extraction, then frame merging back to 16:9 stereo picture format, before the final conversion to video. 

1st you'll want to cut the sequence of the VR video you want to work with, using the "cut video" batch file provided with DFL, just like you would do with a regular DF. I usually keep my VR clips under 5-7 minutes. Next you need to extract frames from said clip, as you would do with regular DFs. Then only you're ready to split the frames in your data_dst with this program:

Here's a link for SplitVR2.0, A small utility made by /uCoderMotor. 
You are not allowed to view links. Register or Login to view.
It uses ffmpeg so if you know you're way with it, you might not even need this. 
the program is stand alone and pretty self explanatory.

You want to edit the "Split" batch file so it knows your data_dst directory as well as the "Join" batch file so it knows where your data_dst/merged folder will be. That's pretty much all there is to it. The program will split every frame you have in a given folder and rename them properly with a "L" or "R" prefix

Once you've completed the splitting process you will end up with a backup folder in data_dst that you can either delete or keep if you want. At that point you're ready to exact/align faces.

Everything you know about 2d deepfakes also applies in VR. You want the face to be as stable as possible and make sure the angles will translate well when training. VR videos can have bad sequences where the faces are really warped. I usually skip those parts altogether. When the faces are right in the center of the frame, they wont be warped at all. I know there is a program called "VRripReplace" that can "unwarp" VR spherical projection but it seemed pretty tedious to make it work with DFL the last time I checked. I'm interested to hear about your experiences with it, if you have tried it.

Also, you might get better results when training one channel at a time, probably even more so when working with low VRAM(?) but I usually do L & R at the same time. 

Once you've completed training and converting just as you would do with 2d DFs, right before converting to mp4, you need to merge back all your converted frames with the "join" batch file (be sure it knows where your data_dst/merged folder is) 

then, convert to mp4 and you're ready to go! 

As you can see, it's not much more complicated, It's just more time consuming, but the results will be worth it!

I hope this comes handy to some of you!

yvanderfel's Forum Info
Joined:
12-06-2018
Last Visit:
Yesterday, 09:49 PM
Total Posts:
11 (0.07 posts per day | 0.18 percent of total posts) Find All Posts
Total Threads:
3 (0.02 threads per day | 0.22 percent of total threads) Find All Threads
Time Spent Online:
14 Hours, 17 Minutes, 29 Seconds
Thanks/Likes:
Given: 2 | Recieved: 5
Members Referred:
0
yvanderfel's Contact Details