How to tune hyper-paramter while using one-shot algorithms.

[songlei00]

• NNI version: 2.3
• Training service: local
• OS: ubuntu18.04
• I use PyTorch.

I found a possible solution in Can we do hyperparameter optimization while doing NAS?. However, when we use HPO in NNI, we should use function nni.report_final_result to record the metric during training, as follows:

for epoch in range(args.epochs):
  metric = run_model()
  nni.report_final_result(metric)

One-shot algorithms like Darts package almost all details. I don't know how to add nni.report_final_result. If I don't add it, the web UI is empty, even if the trial is succesed, like the following picture.

[ultmaster]

You should report the result for the final accuracy of retrained searched model. This means you should combine the search script with retrain script, report the final test accuracy from retrain script.