MrDeepFakes Forums
  • New and improved dark forum theme!
  • Guests can now comment on videos on the tube.
   
homerj975Google Colab Problem
#1
I have been trying to use Google Colab but am running into a problem that I hope someone can help me with.  When it stops the training after 12 hrs where do the files go? I know that it "backs up" after 11 hrs but where are those files when I come back to Colab after I have been kicked off? There are no longer in the Files area like they were before and my Google Drive Workspace is not updated.  This is the second time I have run into this problem and I don't understand what I am missing.  The first time, it kept all the data sets but the model was missing, this time, the model folder is there but the data sets are missing.  I saved the notebook a couple times before I got kicked off but when I import from google drive files are missing. I do not have any experience with this type of thing so forgive me if this question is incredibly stupid. 

Thanks
#2
(05-09-2019, 03:44 AM)homerj975 Wrote: You are not allowed to view links. Register or Login to view.I have been trying to use Google Colab but am running into a problem that I hope someone can help me with.  When it stops the training after 12 hrs where do the files go? I know that it "backs up" after 11 hrs but where are those files when I come back to Colab after I have been kicked off? There are no longer in the Files area like they were before and my Google Drive Workspace is not updated.  This is the second time I have run into this problem and I don't understand what I am missing.  The first time, it kept all the data sets but the model was missing, this time, the model folder is there but the data sets are missing.  I saved the notebook a couple times before I got kicked off but when I import from google drive files are missing. I do not have any experience with this type of thing so forgive me if this question is incredibly stupid. 

Thanks

Hello!
I am a DFL-Colab developer.
All data is copied to the Google Drive that you connected. The important point is that this may not be the account from which you run DFL-Colab. Files are copied to the account that you connect when you enter the secret code generated by Google.

Further. Previously, there were problems with the fact that the Colab machine was turned off and reset (along with the files) earlier than 11 hours and then the backup did not occur. To solve this problem, a backup option was introduced every hour during training. Try using it.
#3
thanks for the quick reply. I did use the back up every hour today but where do the files go? The model folder is still in my files and my drive but the data sets are now missing. I saved a copy of the notebook to my drive two hours before that I can save but it doesn't load for me. After being kicked off after 12 hours, is there a time limit to when I can use it again? is that why?
#4
(05-09-2019, 03:56 AM)homerj975 Wrote: You are not allowed to view links. Register or Login to view.thanks for the quick reply. I did use the back up every hour today but where do the files go? The model folder is still in my files and my drive but the data sets are now missing. I saved a copy of the notebook to my drive two hours before that I can save but it doesn't load for me.  After being kicked off after 12 hours, is there a time limit to when I can use it again? is that why?

As far as I understand, when you start a new session and import the workspace from your Drive, only the model is imported, without your datasets?

Compare the size of your archive on Colab and the archive that is uploaded to your Drive. The size of the archive on Colab can be found if you hover over it with the mouse.

[Image: ZjwDAzfh.jpg]

Regarding your last question. Officially, Google does not set any limits for the session from the same account. But from experience, after a while you will stop giving a free GPU backend if you constantly train your models. This is considered a heavy calculation, which Google does not approve of at Colab. Therefore, it is recommended to alternate accounts at each session.
#5
(05-09-2019, 04:04 AM)chervonij Wrote: You are not allowed to view links. Register or Login to view.
(05-09-2019, 03:56 AM)homerj975 Wrote: You are not allowed to view links. Register or Login to view.thanks for the quick reply. I did use the back up every hour today but where do the files go? The model folder is still in my files and my drive but the data sets are now missing. I saved a copy of the notebook to my drive two hours before that I can save but it doesn't load for me.  After being kicked off after 12 hours, is there a time limit to when I can use it again? is that why?

As far as I understand, when you start a new session and import the workspace from your Drive, only the model is imported, without your datasets?

ah ok so then if I wanted to continue the training, I just need to use a different account. Yes only the model was imported. Last time I had tried Colab, it was the opposite so I am not sure what happened.  I also have no experience with this, and am just following along with the guide that is on this site. (A big thank you to @dpfks for making the Colab guide! and to everyone who made the build for Colab!)

One last question, is there a way to remove images from aligned folder after extracting the faces?  I couldn't figure it out last night so to remove the false positives etc. I have been exporting it to my drive and then downloading to my computer and then sorting and then reuploading.  It's a lot of work to just remove some files but with it being free, it's not that big of a deal if that's what I need to do.
#6
@homerj975

In order to remove false positives, in any case, you need to download them to your computer. Colab does not provide such interactivity to do it quickly without downloading aligned faces. But in any case, you can export/import only aligned images. This is fast enough because they are small in size.
#7
(05-09-2019, 04:23 AM)chervonij Wrote: You are not allowed to view links. Register or Login to view.@homerj975

In order to remove false positives, in any case, you need to download them to your computer. Colab does not provide such interactivity to do it quickly without downloading aligned faces. But in any case, you can export/import only aligned images. This is fast enough because they are small in size.

Yes that isn't a problem but I do not have a good computer so the extracting process is very time consuming on my local machine. Thanks for your help, I appreciate it!

Forum Jump:

Users browsing this thread: 1 Guest(s)