MrDeepFakes Forums
  • New and improved dark forum theme!
  • Guests can now comment on videos on the tube.
   
Pocketspeed[GUIDE] FaceSwap - How To Get Started: Part Three - Conversion and FAQ
#1
[Image: unXUbcP.png]

If you have not read Parts One and Two of this guide, please go back and do so now.

This guide is based upon my personal understanding and workflow. There may be better or more efficient ways to accomplish the same tasks. This guide will cover only the essential basics of Faceswap usage. After you have used Faceswap a few times, you can experiment with more advanced settings and usage on your own. The purpose of this guide is simply to get you started.

Step 1: CONVERSION

Once we have our trained model and we have satisfactory results, it is time to convert. The video that you want your replacement face swapped into is your target video. We will need all the frames from our target video. So we will use the Effmpeg tab of the GUI again:

[Image: MYw1Hna.jpg]

You might have all the frames from your target video when you extracted frames in Part One of this guide. If you already have all the frames from your target video saved, you can use those. If not, then you will need to get the frames from your video.
  • Select action to extract.
  • Select input as your video.
  • Select output as a folder to contain your frames. You can create a new folder if necessary, such as Img_A_Convert. Or if your ImgA folder that you created earlier is empty, you can use it.
  • Select your reference video (that is, your target video) if necessary.
  • Leave FPS as -1.
  • Click the Effmpeg button towards the bottom left of the GUI.
  • Wait for the process to finish.
Now you should have every single frame from your target video.

Remember the folders you created before the extraction and training process? Well now, we will go back to our “Faceswap_project” folder and create another new folder. We will call this folder “Merged_frames”.

Now we have to extract faces from our Img_A_Convert folder. We need the faces from every frame (at least the ones that contain faces), and we need an alignments file so that Faceswap can use them as a reference.

A quick word about Alignments: alignments are essentially the faces that you extracted earlier in the process, but referenced in such a way that the application can find the data. Think of alignments as a file that contains facial landmarks that the application can recognize and use. Alignments are typically stored in a .json file.

In order to get our necessary alignments (json) file, we will need to use the extract tab in the GUI one more time:

[Image: ZlgCB3v.jpg]

This time, we will need to use our Img_A_Convert folder for our input, because we need an alignment for every frame for the video we wish to create.
  • Select input as Img_A_Convert (or whereever you have all your single frames from you target video stored).
  • Select output as Align_for_Conversion_A (that you created earlier in Step One).
  • Extract as you did in Step One.
  • Wait for the process to finish.
Now you will need to “clean” these extractions as you did in Step One, but we will be a little more forgiving in what we we will keep. Blurs, partial faces, and odd angles can be left in. The only faces we will discard are false positives, or faces that we do not want. Pro-tip: every frame that does not have an aligned face attached to it will not be converted later. This means that if you use faces from frames 1-100, delete or discard faces from frames 101-120, then resume using faces from 121 onward, then frames 101-120 will not have swapped faces in your end result video.

If you have to discard frames due to bad alignments, it is possible to manually get aligned faces from frames that you need. This will be covered in the Advanced Tips section later in this guide. Pro-tip: It is a good practice to edit your target video with a video editing program. You should do this before you extract your frames for conversion. Edit out sections of your target video that you don't need or want. Also edit out sections of your target video that may cause problems during the conversion process, such as kissing, extreme face angles, actors that you do not want or need, etc.

Once you have a face and an alignment for every frame that you wish to convert, we can begin our conversion by using the Convert tab of the GUI:

[Image: fGqGDy0.jpg]
  • Select input as the folder in which you have stored your frames for conversion. In our example, this is Img_A_Convert.
  • Select output as your newly created Merged_frames folder.
  • Select Model Dir as your model folder from training. This should be your “Model” folder from Part Two of this guide.
  • Select Input Aligned Dir as your “cleaned” Align_for_Conversion_A folder. This is the folder that contains your face alignments that you wish to convert. If any alignments are deleted or missing, they will be skipped in the conversion process.
  • Select Trainer as the training model you selected earlier. In our example, this will be “Original”.
  • Leave Converter as “masked”.
  • Select your Mask Type to the mask type you chose in the edit tab during Part Two of this guide. In our example, we chose mask type as “none”, and in this case, we should select Mask Type as “none”.
  • Blur and Erode settings can be used to blend the replacement face into your images that you are merging. If you get a strange box effect around the edges of the swapped faces, or things don't just look quite right, Blur and Erode can be useful to get the merged frames to look better. Simply delete the contents from your output folders, and convert again with different Blur and Erode settings. Keep experimenting until you achieve satisfactory results. Warning: Do not delete any contents from your input folders, or you will have to start the conversion process over.
  • Sharpen has options which may, or may not, improve your results. Generally, you should leave Sharpen to none. Feel free to experiment with the Sharpen setting later after you have more experience with using Faceswap.
  • There are also checkboxes toward the bottom of the GUI for things such as Match Histogram, Avg Color Adjust, Seamless, and Smooth Box. Seamless is known for delivering flickering results in the final video. In our example, we will check the boxes for Match Histogram, Avg Color Adjust, and Smooth Box. We will leave all other boxes unchecked. We will avoid Seamless in almost every circumstance. We can use Blur and Erode settings to compensate for seams if we need to, but usually, Smooth Box will take care of what we need.
  • Click the Convert button towards the bottom left of the GUI.
  • Wait for the process to finish.
You should now have a Merged_frames folder that contains all of your still-image frames for your video. Now you only need to make a video from these frames. There are multiple ways to do this. Various video editing programs can do it. Or you can use the Effmpeg tab in the GUI:

[Image: MYw1Hna.jpg]
  • Select action as Gen-Vid.
  • Select input as the folder containing your merged frames. In our example, this is the “Merged_frames” folder.
  • Select your Reference Video. This should be contained in the folder containing your target video. You will need to select video specifically. Use the “All files” option to find your video, if necessary.
  • Select Extract Filetype as the file type in your Merged_frames folder. Generally, this should be PNG.
  • Check the Mux Audio checkbox in the Options section if you wish to include the audio from your target video.
  • Press the Effmpeg tab.
  • Wait for the process to finish.
In your Faceswap_project folder, there should now be an “out.mkv” file. This is your converted video. Double-click to view it.

Your Faceswap project is now complete! Congratulations on your first Faceswap project!

Advanced Tips

Back up your data and folders before experimenting with any of these techniques.

You should use default settings and options when first learning how to use Faceswap. After a little practice, you may want to experiment with different settings in the GUI tabs. Remember that hovering over most selections will give you additional information and explanations about each setting or option.

The Alignments tab in the GUI can be used to modify or create more precision with your alignments, especially for Conversion. There is even a Manual Alignment option available. The Manual Alignment option can be used to specifically designate the facial landmarks that you want. This is especially useful for bad alignments, or frames for which Faceswap failed to get the alignments that you need or want. Manual Alignments are particularly useful for face images that involve kissing or extreme angles. Sometimes faces will not be detected in frames, or faces will be detected poorly (that is, the face is upside-down, etc.). Manual Alignments are useful, and necessary, for these frames. Otherwise, these frames will not be converted, or converted badly.

The Sort tab in the GUI has a multitude of options for sorting your extracted faces. You can even rename your files (that is, your extracted faces) if you need to for conversion. Warning: Sort by Blur often yields inconsistent results.

Feel free to experiment with the Global and Model settings under the Edit tab of the GUI. Each project is different, and some settings will work better under different conditions.

Frequently Asked Questions (FAQ)

I can't get Faceswap installed correctly. Every time I try to run it I get error messages. Please help?

Use the installer listed in Part One of this guide for the easiest installation process. It installs all the necessary requirements, creates your python virtual environment, and even handles your CUDA necessities and creates a desktop shortcut for you. The installer download can be found here: You are not allowed to view links. Register or Login to view.

I'm trying Faceswap for the first time, and it just times out when I try to extract the faces I want. What gives?

The very first time that you run the extraction process, the face alignment network may take a very long time to start working. Sometimes, it will even completely time out and fail. Close out Faceswap, then try again. The extraction process should work on the second and subsequent attempts. Note that major updates to Faceswap may cause this issue to arise in the future. Simply try again, and it should work.

I can't get faces from my images. Faceswap can't find my files?

Make sure that you have all of your folders named correctly, and that the selections in the GUI point to the correct locations and folder names. Also, sometimes switching any JPG images to PNG can help to solve this problem. If you followed this guide precisely, you should not have any problems. If the problem persists, please post a thread describing the issue. Other users are often excellent sources of support.

Every time I try to train my model, it keeps crashing. I get strange errors, and I don't know what they mean?

For specific fixes, you will need to check your crash log. But a vast majority of the time, a crash during training is an Out-Of-Memory error (OOM). This means that there is not enough VRAM to begin or complete the selected action. First, check the requirements for the model trainer you are using. Example: the Villain model must have an absolute minimum of 8 Gb of VRAM, and 10+ Gb of VRAM is recommended. The Unbalanced model needs 6+ Gb of VRAM. Second, if you meet all the requirements, then try lowering the batch size. Remember that batch sizes need to be in powers of 2 (16, 32, 64, 128, etc.). If the problem persists, please post a thread describing the issue.

How long will it take to train a 3 minute clip?

Unfortunately, the time to create a DeepFake is not dependent (much) on the length of the video. Your system specs, particularly your GPU VRAM, is the biggest factor in determining how fast you can create a DeepFake clip. Especially long clips can take more time to create, but this is not due to the length of the video, but because of the length of time needed for face and frame extraction, sorting through the face alignments, and conversion time. Long videos are often more complicated to create as well.

My audio is off in the DeepFake I created. The voices don't match the mouth movements, etc.?

Make sure that the FPS (framerate) you have chosen for your output video matches the FPS of your reference video. Also, different video formats can exacerbate this issue. So make sure that if you are using an MP4 file, that you use MP4 as the format for your creation. If the Effmpeg tab seems to be giving you trouble, you can use FFMpeg directly. Just do a Google search for FFMpeg tutorial. There are other methods for using still-images (or frames) to create a video. You can Google search these as well, and you may have better luck with a different solution. (Personally, I've found that open-source and free Blender works very well. I don't even use Effmpeg or FFMpeg for converting to video any more.)

In my final video, I get these weird box-like shapes that appear quickly over a face, then disappear quickly. Also, sometimes the original face shows up for a moment, then my replacement face is back. It doesn't look good. What do I do?

The box-like appearances are usually due to a lack of good training for a particular face or face angle. The software isn't perfect, and sometimes it just can't handle certain angles or faces well. If you suspect it is a training issue, then train your model for more time, then try again. If it is a bad angle, there is little you can do. Try to edit these out using a video editing program. If you are getting the original face appearing briefly, then you most likely do not have a face alignment in your JSON file for that particular frame or series of frames. This means that you have deleted or discarded this face during the Faceswap process. You can either extract frames again to try to recover these alignments, use the manual alignments option to get your own alignment for the frame, or simply edit these out using a video editing program. The best practice is to take care at the beginning of your project, and edit your target video first. This will help to keep these issues to a minimum.

What are occlusions/obstructions and how can I handle them?

Face occlusions and obstructions are simply things that partially or completely block or obscure the face, or portions of the face. They can be anything, and include things such as: a hand passing in front of a face, eyeglasses, sunglasses, hats and brims of hats, etc. There are no hard rules for handling occlusions. You can always leave occluded faces in your alignments that you will convert, but leave them out of alignments for training. Your results may vary. Some users have reported having good luck with trying to match similar occlusions for Data A and Data B. That is, if A has eyeglasses, then B should have similar eyeglasses. Then you would train those faces in a different model folder. You may have to use a video editor for your resulting video, splicing in your occluded face sections. This is experimental and not guaranteed. Best practice is to avoid face occlusions as much as possible.

Can I contact the Faceswap devs with questions?

You can. Be aware that the official Faceswap project is a completely Safe-For-Work (SFW) environment. There is a zero-tolerance policy for Not-Safe-For-Work (NSFW) content or questions. Any NSFW issues can be posted here at MrDeepFakes.com. You can reach out to the Faceswap devs through either Faceswap-playground portion of Github, or through the official Discord server. Be courteous, be respectful, and keep it SFW, please!


This concludes How To Get Started With FaceSwap. Thank you for taking the time to read this guide, and hopefully you will find it useful! Good luck!
#2
Getting an issue on the conversion to Gen-Vid, not sure what the issue is.

07/03/2019 20:01:44 ERROR Got Exception on main handler:
Traceback (most recent call last):
File "C:\Users\Kevin\MiniConda3\envs\faceswap\lib\site-packages\ffmpy.py", line 95, in run
stderr=stderr
File "C:\Users\Kevin\MiniConda3\envs\faceswap\lib\subprocess.py", line 729, in __init__
restore_signals, start_new_session)
File "C:\Users\Kevin\MiniConda3\envs\faceswap\lib\subprocess.py", line 1017, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\Kevin\faceswap\lib\cli.py", line 124, in execute_script
process.process()
File "C:\Users\Kevin\faceswap\tools\effmpeg.py", line 199, in process
self.ref_vid = DataItem(path=self.args.ref_vid)
File "C:\Users\Kevin\faceswap\tools\effmpeg.py", line 58, in __init__
self.set_fps()
File "C:\Users\Kevin\faceswap\tools\effmpeg.py", line 123, in set_fps
self.fps = Effmpeg.get_fps(self.path)
File "C:\Users\Kevin\faceswap\tools\effmpeg.py", line 368, in get_fps
_fps = ffp.run(stdout=subprocess.PIPE)[0].decode("utf-8")
File "C:\Users\Kevin\MiniConda3\envs\faceswap\lib\site-packages\ffmpy.py", line 99, in run
raise FFExecutableNotFoundError("Executable '{0}' not found".format(self.executable))
ffmpy.FFExecutableNotFoundError: Executable 'ffprobe' not found
07/03/2019 20:01:44 CRITICAL An unexpected crash has occurred. Crash report written to C:\Users\Kevin\faceswap\crash_report.2019.07.03.200141755667.log. Please verify you are running the latest version of faceswap before reporting
Process exited.
#3
hi thanks for all info . you mentioned video for input B for extraction .
however i have only photos of celeberity ..ofcource i have video for first celeberity .

will extraction from photos will work for one person?
#4
(08-03-2019, 01:35 PM)yoldashh Wrote: You are not allowed to view links. Register or Login to view.hi thanks for all info . you mentioned video for input B for extraction .
however i have only photos of celeberity ..ofcource i have video for first celeberity .

will extraction from photos will work for one person?

As long as they are in the correct folder and Faceswap is directed to them, yes.
#5
Great tutorial, thank you.

I can't seem to get the manual alignment to work.

I keep getting ERROR: alignments file not found at: 'D:\faceswap\"D:\faceswap\Projects\Img_Convert_A\alignments.json

Which alignment file am I supposed to be using?
#6
(08-29-2019, 03:49 AM)Strifexz Wrote: You are not allowed to view links. Register or Login to view.Great tutorial, thank you.

I can't seem to get the manual alignment to work.  

I keep getting ERROR: alignments file not found at: 'D:\faceswap\"D:\faceswap\Projects\Img_Convert_A\alignments.json

Which alignment file am I supposed to be using?

The alignments.json file is created when you extract faces from your video
~ Fake it till you make it ~
You are not allowed to view links. Register or Login to view.
You are not allowed to view links. Register or Login to view.
#7
Hi
Thanks for that awesome guide
i'm using the last version of faceswap , everything works fine until the step of merged frames and generating the video , here's my setting for generating the video
[Image: hxN1xRe.jpg]
btw , the faces when merging the frames looks blurry even the quality of the videos is HD , here's an example of the 10 first frames : You are not allowed to view links. Register or Login to view.
and the worst is the video  Huh i can only hear the audio and noimages appears on the screen ; the video
You are not allowed to view links. Register or Login to view.
i'm really confused , any help will be appreciated
Thank you

Forum Jump:

Users browsing this thread: 1 Guest(s)