MrDeepFakes Forums

Some content may not be available to Guests. Consider registering an account to enjoy unrestricted access to guides, support and tools

  • We are looking for community members who are intested in helping out. See our HELP WANTED post.

I can train, but not extract faces or convert?

WJSNG

DF Vagrant
So the title pretty much says it all. I can run training just fine if my friend sends me extracted faces. but I can't extract faces from my video extractions and I can't convert my trained models into videos I get an error saying no faces found? I'm so confused, how can I do one but not the other?
it just keeps saying copying without faces, but the model is fully trained to 75 thousand itr?
here is what happens when I try to convert and same thing when i extract it says no faces found?
[font=Whitney,]Converting:  99%|###################################################################5| 894/900 [00:18<00:00, 48.24it/s]no faces found for 00900.jpg, copying without faces
Converting: 100%|####################################################################| 900/900 [00:18<00:00, 47.43it/s]
Done.[/font]


[font=Whitney,]I am using AMD with the Plaidml and keras backend. [/font]
[font=Whitney,]I have a AMD r9 390 with 8 gigs of VRAM[/font]
 

tania01

DF Admirer
Verified Video Creator
the extracted faces for the dst are linked to the extracted frames of the dst. it will allow you to train faces as long as they are extracted with dfl. if you don't use the faces extracted from the dst video it will not convert. if your system is not powerful enough to handle extraction, i recommend using colab. i dunno if you're following the guide or not, you have to split the frames first then you should be able to extract faces.
 

WJSNG

DF Vagrant
Hey there, thank you so much for the reply. my computer is more the powerful enough, I'm able to train very quickly, I did exactly that I followed the build and extracted the frames using the corrupt bat, but when it comes time to extract the faces from those frames, it says no faces were found? it's very strange.

I have heard of using collab before but when I google it all I get is like a join project page? do you have a link to what it is? I have not been able to find anything about it.
 

tania01

DF Admirer
Verified Video Creator
corrupt bat? post a couple of extracted frames. it's hard to recommend a solution without seeing the problem.
 

WJSNG

DF Vagrant
a couple frames sure one second. just know it's a 3d face, but my friend is able to extract frames from it. let me run the process and post the full error it's very strange, even he is confused.

This is the error log
But I can train so I know my GPU and CPU can detect faces? I feel it has something to do with this line but i have no idea why it's doing this.
"Advanced Micro Devices, Inc. Hawaii (OpenCL) doesnt response, terminating it."
here is one of the frames too.

5) data_dst extract faces S3FD all GPU.bat
I apologize for the long post coming
Performing 1st pass...
Running on Advanced Micro Devices, Inc. Hawaii (OpenCL).
Using plaidml.keras.backend backend.
INFO:plaidml:Opening device "opencl_amd_hawaii.0"
  0%|                                                                                          | 0/900 [00:00<?, ?it/s]INFO:plaidml:Analyzing Ops: 38 of 254 operations complete
INFO:plaidml:Analyzing Ops: 55 of 254 operations complete
INFO:plaidml:Analyzing Ops: 106 of 254 operations complete
INFO:plaidml:Analyzing Ops: 126 of 254 operations complete
INFO:plaidml:Analyzing Ops: 206 of 254 operations complete
100%|################################################################################| 900/900 [06:12<00:00,  2.42it/s]
Performing 2nd pass...
Running on Advanced Micro Devices, Inc. Hawaii (OpenCL).
Using plaidml.keras.backend backend.
INFO:plaidml:Opening device "opencl_amd_hawaii.0"
  0%|                                                                                          | 0/900 [00:00<?, ?it/s]INFO:plaidml:Analyzing Ops: 228 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 520 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 833 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 1153 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 1457 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 1776 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 2092 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 2403 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 2725 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 3050 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 3352 of 3771 operations complete
INFO:plaidml:Analyzing Ops: 3690 of 3771 operations complete
Advanced Micro Devices, Inc. Hawaii (OpenCL) doesnt response, terminating it.

Performing 3rd pass...
Running on CPU0.
Running on CPU1.
Running on CPU2.
Running on CPU3.
Running on CPU4.
Running on CPU5.
Running on CPU6.
Running on CPU7.
0it [00:00, ?it/s]
-------------------------
Images found:        900
Faces detected:      0
-------------------------
Done.




706dd138ebd3d2480e6a0cfbd730b40b.jpg
 

tania01

DF Admirer
Verified Video Creator
extracters sometimes have trouble even with regular faces in videos. don't what the extractor would do with this kind of image. try manual extraction.
@"dpfks" you ever seen this error before?

same image as above, resized: 
Dp0cprDh.jpg
 

WJSNG

DF Vagrant
You know the weird thing? three of my friends who do DF's can extract the faces from this, they have no issues it's just me. very strange, and I thought it was maybe like something wrong with my CPU or GPU? but I can train a 256 model on my PC just fine.
 

tania01

DF Admirer
Verified Video Creator
do they also have amd cards? i'm not sure, but you might be missing some drivers or something.
 

WJSNG

DF Vagrant
I think something might be more going on. for training, I can only get toa bout 18k itr before this error pops up ever 100 itr on me. I can restart the model training but then this error will reappear.

Error: cannot convert float NaN to integer]
Traceback (most recent call last):
File "C:\DeepFaceLabOpenCLSSE\_internal\DeepFaceLab\mainscripts\Trainer.py", line 107, in trainerThread
iter, iter_time = model.train_one_iter()
File "C:\DeepFaceLabOpenCLSSE\_internal\DeepFaceLab\models\ModelBase.py", line 491, in train_one_iter
preview_lh = ModelBase.get_loss_history_preview(self.loss_history, self.iter, preview.shape[1], preview.shape[2])
File "C:\DeepFaceLabOpenCLSSE\_internal\DeepFaceLab\models\ModelBase.py", line 595, in get_loss_history_preview
ph_max = int ( (plist_max[col][p] / plist_abs_max) * (lh_height-1) )
ValueError: cannot convert float NaN to integer
Done.
 

dougonnabone

DF Pleb
I also have been getting similar face extraction failures on an AMD card. MT extract does succeed *rarely*, but it is too unreliable and just a waste of time to repeatedly end up with "faces detected = 0". I have the most current drivers. And yes - i know all the prerequisite steps, configurations, etc necessary to extract successfully

I've seen this problem posted often enough from people with good gear who know what they are doing, to suspect it is a problem in the code somewhere

I seem to remember getting reliable extractions from a 6/19 (May) build of DFL CLSSE - the problem only began for me with the June 2nd build. Unfortunately that May build is no longer on Mega and no one has answered my inquiries in shoutbox for that older version. Maybe I will make a post requesting it

Extracting on Colab has an issue - For a 5 minute video, if extracting as png, a data_dst folder of 23+ GB is created --- far too large of a workspace.zip to export to the G-Drive free 15 GB limit. So the option is buy more G-Drive space, or extract as lossy .jpg
 

WJSNG

DF Vagrant
Can someone please tell me where to find google colab? I looked it up but what I found is not what I needed, I only need to draw out faces from a 30 second video, so it will do more than enough for me. i just need my rotating face.
 

WJSNG

DF Vagrant
Yikes that looks more complicated than even DFL normally is. thank you, I can't believe I missed that tutorial.
 

dougonnabone

DF Pleb
WJSNG said:
Yikes that looks more complicated than even DFL normally is.  thank you, I can't believe I missed that tutorial.


It's not bad once you've been through it a couple times (and *if* you already have some experience and understanding of DFL from working on your local machine - it would be rough using Colab DFL without that prior experience.) 

It's pretty straightforward, and i often prefer working on it to working on my local PC

Just go through the guide, step by step. And if you only need it for face extractions, it should be easy. Extract what you need on Colab, then download them to your local machine and finish the project there. Just don't attempt to extract more than a 2 - 3 minute video as pngs - it will create a data-dst folder that is too large for the Google Drive free 15 GB limit. Use jpg instead - it is a "lossy" format, but gets the job done well enough
 

dpfks

DF Enthusiast
Staff member
Administrator
Verified Video Creator
WJSNG said:
I think something might be more going on. for training, I can only get toa bout 18k itr before this error pops up ever 100 itr on me. I can restart the model training but then this error will reappear.

Error: cannot convert float NaN to integer]
Traceback (most recent call last):
 File "C:\DeepFaceLabOpenCLSSE\_internal\DeepFaceLab\mainscripts\Trainer.py", line 107, in trainerThread
   iter, iter_time = model.train_one_iter()
 File "C:\DeepFaceLabOpenCLSSE\_internal\DeepFaceLab\models\ModelBase.py", line 491, in train_one_iter
   preview_lh = ModelBase.get_loss_history_preview(self.loss_history, self.iter, preview.shape[1], preview.shape[2])
 File "C:\DeepFaceLabOpenCLSSE\_internal\DeepFaceLab\models\ModelBase.py", line 595, in get_loss_history_preview
   ph_max = int ( (plist_max[col][p] / plist_abs_max) * (lh_height-1) )
ValueError: cannot convert float NaN to integer
Done.

Your model collapsed. You need need to start over. I always recommend using the backup feature.

I am not sure about your face extraction issue. Maybe AMD version doesn't do as good as a job as NVIDIA
 

WJSNG

DF Vagrant
Yikes, that is brutal, I wish I knew why it collapsed so I could avoid it in the future.
I know SAE runs really poorly on my machine but the LIAE and H run just fine even the experimental 256 H that was posted here.
but I really need the full face, is there someone here who might be able to turn the LIAE128 model.py to a 256 version? I tried editing it myself but it didn't work.
 

dpfks

DF Enthusiast
Staff member
Administrator
Verified Video Creator
WJSNG said:
Yikes, that is brutal, I wish I knew why it collapsed so I could avoid it in the future.
I know SAE runs really poorly on my machine but the LIAE and H run just fine even the experimental 256 H that was posted here.
but I really need the full face, is there someone here who might be able to turn the LIAE128 model.py to a 256 version? I tried editing it myself but it didn't work.

There will always be a chance your model can collapse. You can turn on gradient clipping to reduce this, or just use the autobackup feature.
 

coveredinnormal

DF Vagrant
This  script has been a life saver for me. The auto backups that are now in DFL can be overwritten if left too long so you may come back to all collapsed backups. I found this post months ago and I have mine set to save every 15-30 min depending. The only bad thing is, if you come back to check on your model and its fine , then delete every backup except the latest one. That saves HD space. https://mrdeepfakes.com/forums/thread-model-backup-batch-files?highlight=backup+script
 
Top