Inquiry about pixel-wise consistency

[jangbi1]

Hello, @usert5432. ``

Thanks for your quick and detailed response. #22

Additionally, as I read through your paper, I generated an issue because I had some questions.

I am interested in applying pixel-wise consistency loss for video translation, but I can’t find an explanation for this in the readme.

I'm wondering if you could check on this for me, and if it's possible to use it, I think it would greatly help my research.

Once again, thank you for sharing your astonishing research. I look forward to your reply.

Sincerely,

Jangbi.

[usert5432]

Hello @jangbi1,

I am interested in applying pixel-wise consistency loss for video translation, but I can’t find an explanation for this in the readme.
I'm wondering if you could check on this for me, and if it's possible to use it, I think it would greatly help my research.

Sure, the difference below shows the modifications need to be done to the Male-2-Female scripts/celeba_hq/train_m2f_translation.py to enable pixel-wise consistency (matching the paper's configuration). After applying this difference, the pixel-wise consistency can be enabled by adding --lambda-consist MAGNITUDE to the script invocation line.

Please, let me know if I should elaborate more on this.

We will update our documentation and provide the proper training scripts with the pixel-wise consistency, but it will take some time, since we are in the middle of another project.

diff --git a/scripts/celeba_hq/train_m2f_translation.py b/scripts/celeba_hq/train_m2f_translation.py
index 05f8fcc..1f7fba4 100644
--- a/scripts/celeba_hq/train_m2f_translation.py
+++ b/scripts/celeba_hq/train_m2f_translation.py
@@ -33,6 +33,11 @@ def parse_cmdargs():
         default = 1e-4, help = 'learning rate of the generator'
     )

+    parser.add_argument(
+        '--lambda-consist', dest = 'lambda_consist', type = float,
+        default = 0.0, help = 'magnitude of the forward-consisntecy loss'
+    )
+
     return parser.parse_args()

 def get_transfer_preset(cmdargs):
@@ -108,8 +113,10 @@ args_dict = {
     'model_args' : {
         'lambda_a'        : cmdargs.lambda_cyc,
         'lambda_b'        : cmdargs.lambda_cyc,
+        'lambda_consist'  : cmdargs.lambda_consist,
         'lambda_idt'      : 0.5,
         'avg_momentum'    : 0.9999,
+        'consistency'     : { 'name' : 'resize', 'size' : 32, },
         'head_queue_size' : 3,
         'head_config'     : {
             'name'            : BH_PRESETS[cmdargs.head],
@@ -131,7 +138,8 @@ args_dict = {
 # args
     'label'  : (
         f'{cmdargs.gen}-{cmdargs.head}_({cmdargs.no_pretrain}'
-        f':{cmdargs.lambda_cyc}:{cmdargs.lambda_gp}:{cmdargs.lr_gen})'
+        f':{cmdargs.lambda_cyc}:{cmdargs.lambda_gp}:{cmdargs.lr_gen}'
+        f':{cmdargs.lambda_consist})'
     ),
     'outdir' : os.path.join(ROOT_OUTDIR, 'celeba_hq_resized_lanczos', 'm2f'),
     'log_level'  : 'DEBUG',