test: move "doing A/B-Test" print statement below ignore-check

Move the print statement that states which A/B-Tests we're performing below the check that filters out ignored test cases. This way, we avoid claiming to do an A/B-Test for things that are later actually filtered out (meaning no A/B-Test actually happens). Signed-off-by: Patrick Roy <[email protected]>
firecracker-microvm · Oct 16, 2024 · a31c79a · a31c79a
1 parent 0f306e5
commit a31c79a
Showing 1 changed file with 2 additions and 3 deletions.
diff --git a/tools/ab_test.py b/tools/ab_test.py
@@ -211,9 +211,6 @@ def analyze_data(
         ), "A and B run produced incomparable data. This is a bug in the test!"
 
         for metric, (values_a, unit) in metrics_a.items():
-            print(
-                f"Doing A/B-test for dimensions {dimension_set} and property {metric}"
-            )
             result = check_regression(
                 values_a, metrics_b[metric][0], n_resamples=n_resamples
             )
@@ -269,6 +266,8 @@ def analyze_data(
         if is_ignored(dict(dimension_set)):
             continue
 
+        print(f"Doing A/B-test for dimensions {dimension_set} and property {metric}")
+
         values_a = processed_emf_a[dimension_set][metric][0]
         baseline_mean = statistics.mean(values_a)