In this repository we have the code for experiments produced in the following paper:
A surprising linear relationship predicts test performance in deep networks (https://arxiv.org/pdf/1807.09659.pdf)
To see examples of how to reproduce the experimental results you can see the submission file to slurm written here:
but an example command is:
python flatness_expts.py -mdl GBoixNet -label_corrupt_prob 0.5 -train_alg SGD -epochs 300 -exptlabel MoreRLInits
note that the important file is flatness_expts.py. The experiments are simple and described in the paper. It should be simple to change the arguments of the script to change the experiment.
Note that the file:
is a slurm submission script and one can use the array command to send multiple runs of similar experiments.
Cool trick for emulating sbatch:
nohup python flatness_expts.py -mdl GBoixNet -label_corrupt_prob 0.5 -train_alg SGD -epochs 300 -exptlabel MoreRLInits
see my code example: https://github.com/brando90/Generalization-Puzzles-in-Deep-Networks/blob/286e5aa9a7fcf28ef0a7fecd135189b0494674d7/pytorch_experiments/my_sbatch.sh