-
Notifications
You must be signed in to change notification settings - Fork 53
DFL Train Data Tracker
The DFL Train Data Tracker is intended to help you track the changes in your DeepFaceLab trained model between various iterations.
You can export a snapshot of your current model, train some more, export a new snapshot, and then use this tool to compare and easily see the changes between different model iterations.
Please note:
This feature requires the use of the community DFL fork, and is only available in MVE 0.9.0 or later.
Before you can make use of the Train Data Tracker, you first need to export model state history
by using --gen-snapshot
flag.
For example, if using the .bat
file for SAEHD training, it could look something like this:
As you can see in the picture, only lines 10 and 11 are different compared to 6) train SAEHD with auto-gen-conf.bat
For more information, visit the article on MVE-DFL fork wiki:
Exporting training data snapshots
To access this feature go to Tools > DFL Train Data Tracker
from the main MVE menu (on top).
You will be presented with a split screen interface:
And you start using the feature by loading the model state history (which you prepared following the steps mentioned earlier in this article or via instructions from community DFL wiki article).
You load the model state history by following these 3 simple steps:
- be sure you're on Configuration panel
- show MVE where your Model State is on your disk
it is the directory with
*_state_history
in its name and that hasconfig.json
in it, by default it is located withinmodel
directory
- click Load (and wait)
known issue: progress bar will not update until you left click anywhere within MVE, but that is just a visual issue - loading happens normally with or without the click.
Once your model is loaded, you will be presented with an interface like this:
Immediately it becomes obvious that the interface is split:
- on the left side you can see the loaded set, and some of the options for selecting how you can see it (options 1-6)
- on the right side (if option for Set Graphs (marked with number 9 in the picture) is selected) you can see the option for choosing which set is loaded, and two Face Graphs - one for distribution of faces based on their distribution per angle (same as regular Heatmap), and one for distribution of average loss per angle
The marked options in the picture are:
- select size of loaded faces below
- option to define custom image scales
- you can use several of the sort options on the faces
- dropdown for selecting which snapshot is previewed below
- checkmark for disabling timeline
- which pictures you want loaded in Timeline (output, swap, both, or none) in addition to aligned face
- dropdown menu for selecting which set should be loaded: dst or src
- Unselect button - to reset your selection if you used any of the Set Graphs to limit set to only pictures that conform to your selected angles
- the option to display Set Graphs and Set selection in the first place
The Face Graphs function exactly the same like you'd expect if you ever used Face Graph in MVE.
For example, by selecting Yaw-60, Pitch 5 option on the graph, where it says 7 Faces, it will also show 7 faces (plus their model-generated images for current snapshot) on the left side of the screen
When you click on any of the faces in the bottom part of the interface, if Timeline is enabled (option 5 in the Interface Explained picture), you will see the Aligned Face + all the model-generated pictures that are exported in your snapshots. You can also customize which set samples will be shown (output and/or swap)
An example pic:
Please note that there is a horizontal scroll-bar at the bottom of Timeline, which you can use to see the pictures from all of your snapshots.