Fixed READMEs for several examples.

foss-for-synopsys-dwc-arc-processors · Sep 16, 2021 · c53549e · c53549e
1 parent 3eb1abb
commit c53549e
Show file tree

Hide file tree

Showing 10 changed files with 114 additions and 65 deletions.
diff --git a/examples/example_cifar10_caffe/README.md b/examples/example_cifar10_caffe/README.md
@@ -187,25 +187,25 @@ Console output using provided small test set may look like:
        ACCURACY CALCULATION on Input IDX testset according to IDX labels set
        IDX test file shape: [20,32,32,3,]
        Model input shape: [32,32,3,]
-
-              2 of 20 test vectors are processed (2 are correct: 100.000 %)
-              4 of 20 test vectors are processed (4 are correct: 100.000 %)
-              6 of 20 test vectors are processed (6 are correct: 100.000 %)
-              8 of 20 test vectors are processed (8 are correct: 100.000 %)
-              10 of 20 test vectors are processed (10 are correct: 100.000 %)
-              12 of 20 test vectors are processed (12 are correct: 100.000 %)
-              14 of 20 test vectors are processed (14 are correct: 100.000 %)
-              16 of 20 test vectors are processed (16 are correct: 100.000 %)
-              18 of 20 test vectors are processed (18 are correct: 100.000 %)
-              20 of 20 test vectors are processed (20 are correct: 100.000 %)
-       Final Accuracy: 100.000 % (20 are correct of 20)
+       
+                2 of 20 test vectors are processed (2 are correct: 100.000 %)
+                4 of 20 test vectors are processed (4 are correct: 100.000 %)
+                6 of 20 test vectors are processed (5 are correct: 83.333 %)
+                8 of 20 test vectors are processed (7 are correct: 87.500 %)
+               10 of 20 test vectors are processed (9 are correct: 90.000 %)
+               12 of 20 test vectors are processed (11 are correct: 91.667 %)
+               14 of 20 test vectors are processed (11 are correct: 78.571 %)
+               16 of 20 test vectors are processed (13 are correct: 81.250 %)
+               18 of 20 test vectors are processed (14 are correct: 77.778 %)
+               20 of 20 test vectors are processed (16 are correct: 80.000 %)
+       Final Accuracy: 80.000 % (16 are correct of 20)
        FINISHED
 
 where:
-* `Final Accuracy: 100.000 % (20 are correct of 20)` reflects how many samples from the testset were accurately predicted in comparison with reference label. The accuracy itself may vary depending on the target platform and `run_*` command. In particular :
+* `Final Accuracy: 80.000 % (16 are correct of 20)` reflects how many samples from the testset were accurately predicted in comparison with reference label. The accuracy itself may vary depending on the target platform and `run_*` command. In particular :
 
-  * `run_FX16` and `run_FX16_FX8_FX8`: Accuracy should be  `100.000 %` for provided small test dataset. 
-  * `run_SA8`:  Accuracy should be  `90.000 %` for provided small test dataset.
+  * `run_FX16` and `run_FX16_FX8_FX8`: Accuracy should be  `80.000 %` for provided small test dataset. 
+  * `run_SA8`:  Accuracy should be  `75.000 %` for provided small test dataset.
 
 ### 3 **External test-set processing.**
 Console output using provided small test set should looks mostly the same as for accuracy measurement mode, but without accuracy values.

diff --git a/examples/example_cifar10_caffe/small_test_base/labels.idx b/examples/example_cifar10_caffe/small_test_base/labels.idx
diff --git a/examples/example_cifar10_caffe/small_test_base/tests.idx b/examples/example_cifar10_caffe/small_test_base/tests.idx
diff --git a/examples/example_har_smartphone/README.md b/examples/example_har_smartphone/README.md
@@ -156,8 +156,8 @@ where:
 * `Result Quality: S/N=5346.3     (74.6 db)` reflects the signal-to-noise ration of the model output in comparison with reference float. The ratio itself (`S/N` and `x db`) may vary depending on the target platform and `run_*` command. In particular :
 
   * `run_FX16`: Result may slightly fluctuates around `S/N=5346.3     (74.6 db)` 
-  * `run_FX16_FX8_FX8`: Result may slightly fluctuates around `S/N=56.0       (35.0 db)` 
   * `run_SA8`: Result may slightly fluctuates around `S/N=4.6        (13.3 db)` 
+  * `run_FX16_FX8_FX8`: Result may slightly fluctuates around `S/N=56.0       (35.0 db)` 
 
 ### 2. **Accuracy measurement for testset.**
 

diff --git a/examples/tutorial_emnist_tflm/README.md b/examples/tutorial_emnist_tflm/README.md
@@ -10,7 +10,7 @@ The first part is disclosed in this readme. The details of the conversion tutori
 
  **Important notes:**
 
-* Example is supported only for VPX configurations with guard bits. For EM/HS, please use [MLI 1.1 release version](https://github.com/foss-for-synopsys-dwc-arc-processors/embarc_mli/tree/Release_1.1/examples/tutorial_emnist_tflm).
+* This example can't be built for x86 platform. The example is supported only for VPX configurations with guard bits. For EM/HS, please use [MLI 1.1 release version](https://github.com/foss-for-synopsys-dwc-arc-processors/embarc_mli/tree/Release_1.1/examples/tutorial_emnist_tflm).
 
 ## Generate Tensorflow Lite Micro Library
 
@@ -82,7 +82,7 @@ The first step is to open a command line and change working directory to the roo
 
 1. Build project using generated TCF and appropriate built-in runtime library for it. Use multithreaded build process (4 threads):
 
-       gmake TCF_FILE=../../hw/vpx5_integer_full.tcf BUILDLIB_DIR=vpx5_integer_full JOBS=4 build
+       gmake TCF_FILE=./hw/vpx5_integer_full.tcf BUILDLIB_DIR=vpx5_integer_full JOBS=4 build
 
 1. Change working directory and build the example:
 

diff --git a/user_tests/CMakeLists.txt b/user_tests/CMakeLists.txt
@@ -38,6 +38,10 @@ else()
     message(FATAL_ERROR "Rounding mode ${ROUND_MODE} is not supported")
 endif()
 
+if (FULL_ACCU STREQUAL ON)
+    list(APPEND TEST_COMPILE_DEFINITIONS FULL_ACCU_ON)
+endif()
+
 # Note: AVEPOOL_16BIT_MUL should be same as the one in mli_lib.cmake
 if (NOT DEFINED AVEPOOL_16BIT_MUL)
     set(AVEPOOL_16BIT_MUL OFF)

diff --git a/user_tests/tests/mli_krn_conv2d/tests_mli_krn_conv2d.cc b/user_tests/tests/mli_krn_conv2d/tests_mli_krn_conv2d.cc
@@ -55,23 +55,32 @@ struct conv2d_test_operands {
 #if defined(CRC_RM_CONVERGENT) || defined(CRC_RM_UP)
 
 // Shared CRC Results
-const crc32_calc test_1_chksum_fx16{ 0x3669E8DA }, test_1_chksum_fx16_fx8_fx8{ 0x627FD168 }, test_1_chksum_sa8{ 0x63A6B2EC },
+const crc32_calc test_1_chksum_fx16{ 0x3669E8DA }, test_1_chksum_fx16_fx8_fx8{ 0x627FD168 },
                  test_2_chksum_fx16{ 0x6075722F }, test_2_chksum_fx16_fx8_fx8{ 0xBFE5DC3D }, test_2_chksum_sa8{ 0x5D288208 },
                  test_3_chksum_fx16{ 0xE2100158 }, test_3_chksum_fx16_fx8_fx8{ 0x550F135E }, test_3_chksum_sa8{ 0x9740102D },
                  test_4_chksum_fx16{ 0x987AC0A8 }, test_4_chksum_fx16_fx8_fx8{ 0x21C772CE }, test_4_chksum_sa8{ 0x056EDB56 },
                  test_5_chksum_fx16{ 0xD8CA1273 }, test_5_chksum_fx16_fx8_fx8{ 0x186AA252 }, test_5_chksum_sa8{ 0x01D390FA },
                  test_6_chksum_fx16{ 0x150A5D20 }, 
                  test_7_chksum_fx16{ 0x05737544 }, test_7_chksum_fx16_fx8_fx8{ 0x7FFA25C2 }, test_7_chksum_sa8{ 0x5E7CF172 },
                  test_8_chksum_fx16{ 0x69862892 }, test_8_chksum_fx16_fx8_fx8{ 0xA124C817 }, test_8_chksum_sa8{ 0x99E3EE1D },
-                 test_9_chksum_fx16{ 0x3B2662E7 }, test_9_chksum_fx16_fx8_fx8{ 0x5C4D2278 }, test_9_chksum_sa8{ 0x3DB4B9EF },
+                 test_9_chksum_fx16{ 0x3B2662E7 }, test_9_chksum_fx16_fx8_fx8{ 0x5C4D2278 },
                  test_10_chksum_fx16{ 0x0AD3FF47 }, test_10_chksum_fx16_fx8_fx8{ 0x0CDE9B47 }, test_10_chksum_sa8{ 0xA4EB24F1 },
                  test_11_chksum_fx16{ 0xEE754246 }, test_11_chksum_fx16_fx8_fx8{ 0x77A6F1AD }, test_11_chksum_sa8{ 0x10AA2F03 };
 // Platform Specific CRC Results
-#if defined(CRC_RM_UP)
+#if defined(FULL_ACCU_ON)
+const crc32_calc test_1_chksum_sa8{ 0x63A6B2EC }, test_9_chksum_sa8{ 0x3DB4B9EF };
+#else
+const crc32_calc test_1_chksum_sa8{ 0xA3FFD976 }, test_9_chksum_sa8{ 0x7D8D9C29 };
+#endif
+
+#if defined(CRC_RM_UP) && defined(FULL_ACCU_ON)
+const crc32_calc test_6_chksum_fx16_fx8_fx8{ 0x8C24C65A }, test_6_chksum_sa8{ 0x2BA3EA5D };
+#elif defined(CRC_RM_UP) && !defined(FULL_ACCU_ON)
 const crc32_calc test_6_chksum_fx16_fx8_fx8{ 0x8C24C65A }, test_6_chksum_sa8{ 0x36699F43 };
-#else 
+#else
 const crc32_calc test_6_chksum_fx16_fx8_fx8{ 0x0BA52872 }, test_6_chksum_sa8{ 0x2BA3EA5D };
 #endif
+
 #else // Not defined CRC_*
 const crc32_calc  test_1_chksum_fx16, test_1_chksum_fx16_fx8_fx8, test_1_chksum_sa8,
                   test_2_chksum_fx16, test_2_chksum_fx16_fx8_fx8, test_2_chksum_sa8,
@@ -288,10 +297,10 @@ int main() {
         }
 #endif
 
-#if V2DSP_XY == V2DSP_XY && defined(CRC_RM_CONVERGENT)
+#if PLATFORM == V2DSP_XY && defined(CRC_RM_CONVERGENT)
         if (strstr(cur_test->descr, "Test 1 SA8_SA8_SA32") != nullptr ||
-            strstr(cur_test->descr, "Test 9-1 SA8_SA8_SA32 Dil+Pad") != nullptr ||
-            strstr(cur_test->descr, "Test 9-2 SA8_SA8_SA32 k3x3 Dil") != nullptr) {
+                strstr(cur_test->descr, "Test 9-1 SA8_SA8_SA32 Dil+Pad") != nullptr ||
+                strstr(cur_test->descr, "Test 9-2 SA8_SA8_SA32 k3x3 Dil") != nullptr) {
             // Em9d fails bitwise comparison with reference .
             reporter.report_message(cur_test->descr, "SKIPPED due to a known issue");
             continue;

diff --git a/user_tests/tests/mli_krn_depthwise_conv/tests_mli_krn_depthwise_conv.cc b/user_tests/tests/mli_krn_depthwise_conv/tests_mli_krn_depthwise_conv.cc
@@ -61,13 +61,21 @@ const crc32_calc test_1_chksum_fx16{ 0x968FB503 },
                  test_6_chksum_fx16{ 0xF03253BB }, test_6_chksum_fx16_fx8_fx8{ 0x122832FF }, test_6_chksum_sa8{ 0x42F80E2D },
                  test_7_chksum_fx16{ 0xDC4EBBE7 }, test_7_chksum_fx16_fx8_fx8{ 0x10447ABF }, test_7_chksum_sa8{ 0xBC318965 },
                  test_8_chksum_fx16{ 0x4D6EFB91 },                                           test_8_chksum_sa8{ 0xF888FAB3 },
-                 test_9_chksum_fx16{ 0xBF6D526A }, test_9_chksum_fx16_fx8_fx8{ 0xFB8CEA65 }, test_9_chksum_sa8{ 0xEA5733C6 },
+                 test_9_chksum_fx16{ 0xBF6D526A }, test_9_chksum_fx16_fx8_fx8{ 0xFB8CEA65 }, 
                  test_10_chksum_fx16{ 0xC02567E8 }, test_10_chksum_fx16_fx8_fx8{ 0x882F41DB }, test_10_chksum_sa8{ 0x390F7E80 };
 // Platform Specific CRC Results
-#if defined(CRC_RM_UP)
+#if defined(FULL_ACCU_ON)
+const crc32_calc test_9_chksum_sa8{ 0xEA5733C6 };
+#else
+const crc32_calc test_9_chksum_sa8{ 0xE0165E99 };
+#endif
+
+#if defined(CRC_RM_UP) && defined(FULL_ACCU_ON)
+const crc32_calc test_1_chksum_fx16_fx8_fx8{ 0x46A964A7 }, test_1_chksum_sa8{ 0x46D36646 },
+                 test_8_chksum_fx16_fx8_fx8{ 0xB73234B8 };
+#elif defined(CRC_RM_UP) && !defined(FULL_ACCU_ON)
 const crc32_calc test_1_chksum_fx16_fx8_fx8{ 0x46A964A7 }, test_1_chksum_sa8{ 0x366725E9 },
                  test_8_chksum_fx16_fx8_fx8{ 0xB73234B8 };
-
 #else 
 const crc32_calc test_1_chksum_fx16_fx8_fx8{ 0xA6CC6260 }, test_1_chksum_sa8{ 0x46D36646 },
                  test_8_chksum_fx16_fx8_fx8{ 0x55734EF0 };
@@ -273,7 +281,7 @@ int main() {
         }
 #endif
 
-#if V2DSP_XY == V2DSP_XY && defined(CRC_RM_CONVERGENT)
+#if PLATFORM == V2DSP_XY && defined(CRC_RM_CONVERGENT)
         if (strstr(cur_test->descr, "Test 9 SA8_SA8_SA32 k5x5 Dil") != nullptr) {
             // Em9d fails bitwise comparison with reference .
             reporter.report_message(cur_test->descr, "SKIPPED due to a known issue");

diff --git a/user_tests/tests/mli_krn_group_conv2d/tests_mli_krn_group_conv2d.cc b/user_tests/tests/mli_krn_group_conv2d/tests_mli_krn_group_conv2d.cc
@@ -56,21 +56,29 @@ struct group_conv2d_test_operands {
 #if defined(CRC_RM_CONVERGENT) || defined(CRC_RM_UP)
 
 // Shared CRC Results
-const crc32_calc test_1_chksum_fx16{ 0xB782413A }, test_1_chksum_fx16_fx8_fx8{ 0xEE4193A9 }, test_1_chksum_sa8{ 0x11D2D466 },
+const crc32_calc test_1_chksum_fx16{ 0xB782413A }, test_1_chksum_fx16_fx8_fx8{ 0xEE4193A9 },
                  test_2_chksum_fx16{ 0x65FD03D2 }, test_2_chksum_fx16_fx8_fx8{ 0xB34C4A01 }, test_2_chksum_sa8{ 0x33341D2B },
                  test_3_chksum_fx16{ 0x71B9E656 }, test_3_chksum_fx16_fx8_fx8{ 0x67578BCF }, test_3_chksum_sa8{ 0xAAD727F2 },
-                 test_4_chksum_fx16{ 0x2D7FD678 }, test_4_chksum_fx16_fx8_fx8{ 0x96994E5E }, test_4_chksum_sa8{ 0xCC24BECE },
+                 test_4_chksum_fx16{ 0x2D7FD678 }, test_4_chksum_fx16_fx8_fx8{ 0x96994E5E },
                  test_5_chksum_fx16{ 0xB719F4A0 }, test_5_chksum_fx16_fx8_fx8{ 0x985577E0 }, test_5_chksum_sa8{ 0xBD32CF0A },
                  test_6_chksum_fx16{ 0x3013D414 }, test_6_chksum_fx16_fx8_fx8{ 0xCB153206 }, test_6_chksum_sa8{ 0xC61C378D },
                  test_7_chksum_fx16{ 0xC979FA6E }, test_7_chksum_fx16_fx8_fx8{ 0x5D29B6B4 }, test_7_chksum_sa8{ 0x28AFE63F },
                  test_8_chksum_fx16{ 0xAC7AFCAE }, test_8_chksum_sa8{ 0xE97450E4 },
                  test_9_chksum_fx16{ 0x2DDBDF54 }, test_9_chksum_fx16_fx8_fx8{ 0x575372E7 },
-                 test_10_chksum_fx16{ 0x78039E74 }, test_10_chksum_fx16_fx8_fx8{ 0x1783EEEC }, test_10_chksum_sa8{ 0x3685F72F };
+                 test_10_chksum_fx16{ 0x78039E74 }, test_10_chksum_fx16_fx8_fx8{ 0x1783EEEC };
 // Platform Specific CRC Results
-#if defined(CRC_RM_UP)
-const crc32_calc test_8_chksum_fx16_fx8_fx8{ 0x629A3BD0 }, test_9_chksum_sa8{ 0x38F5EB92 };
+#if defined(FULL_ACCU_ON)
+const crc32_calc test_1_chksum_sa8{ 0x11D2D466 }, test_10_chksum_sa8{ 0x3685F72F };
+#else
+const crc32_calc test_1_chksum_sa8{ 0xB0EBD3DC }, test_10_chksum_sa8{ 0x079DC058 };
+#endif
+
+#if defined(CRC_RM_UP) && defined(FULL_ACCU_ON)
+const crc32_calc test_8_chksum_fx16_fx8_fx8{ 0x629A3BD0 }, test_9_chksum_sa8{ 0x38F5EB92 }, test_4_chksum_sa8{ 0xE5DBC862 };
+#elif defined(CRC_RM_UP) && !defined(FULL_ACCU_ON)
+const crc32_calc test_8_chksum_fx16_fx8_fx8{ 0x629A3BD0 }, test_9_chksum_sa8{ 0x38F5EB92 }, test_4_chksum_sa8{ 0xCC24BECE };
 #else 
-const crc32_calc test_8_chksum_fx16_fx8_fx8{ 0x0597C4DB }, test_9_chksum_sa8{ 0x396D8D7A };
+const crc32_calc test_8_chksum_fx16_fx8_fx8{ 0x0597C4DB }, test_9_chksum_sa8{ 0x396D8D7A }, test_4_chksum_sa8{ 0xCC24BECE };
 #endif
 #else // Not defined CRC_*
 const crc32_calc  test_1_chksum_fx16, test_1_chksum_fx16_fx8_fx8, test_1_chksum_sa8,
@@ -290,24 +298,24 @@ int main() {
 
 #if defined(__Xvec_guard_bit_option) && (__Xvec_guard_bit_option == 0)
         if (strstr(cur_test->descr, "Test 1 FX16") != nullptr ||
-            strstr(cur_test->descr, "Test 2 FX16 ReluGen") != nullptr ||
-            strstr(cur_test->descr, "Test 3 SA8_SA8_SA32 Mult FPG") != nullptr ||
-            strstr(cur_test->descr, "Test 4 FX16 DW call") != nullptr ||
-            strstr(cur_test->descr, "Test 5 FX16 Conv2D call") != nullptr ||
-            strstr(cur_test->descr, "Test 6 FX16 IO_Memstr") != nullptr ||
-            strstr(cur_test->descr, "Test 7 FX16 W_Memstr") != nullptr ||
-            strstr(cur_test->descr, "Test 9 FX16 k3x3 Mstr+Dil") != nullptr ||
-            strstr(cur_test->descr, "Test 10 FX16 k5x5 Mstr+Dil") != nullptr ||
-            strstr(cur_test->descr, "Test 10 SA8 k5x5 Mstr+Dil") != nullptr) {
+                strstr(cur_test->descr, "Test 2 FX16 ReluGen") != nullptr ||
+                strstr(cur_test->descr, "Test 3 SA8_SA8_SA32 Mult FPG") != nullptr ||
+                strstr(cur_test->descr, "Test 4 FX16 DW call") != nullptr ||
+                strstr(cur_test->descr, "Test 5 FX16 Conv2D call") != nullptr ||
+                strstr(cur_test->descr, "Test 6 FX16 IO_Memstr") != nullptr ||
+                strstr(cur_test->descr, "Test 7 FX16 W_Memstr") != nullptr ||
+                strstr(cur_test->descr, "Test 9 FX16 k3x3 Mstr+Dil") != nullptr ||
+                strstr(cur_test->descr, "Test 10 FX16 k5x5 Mstr+Dil") != nullptr ||
+                strstr(cur_test->descr, "Test 10 SA8 k5x5 Mstr+Dil") != nullptr) {
             // VPX fails bitwise comparison with reference .
             reporter.report_message(cur_test->descr, "SKIPPED due to a known issue");
             continue;
         }
 #endif
 
-#if V2DSP_XY == V2DSP_XY && defined(CRC_RM_CONVERGENT)
+#if PLATFORM == V2DSP_XY && defined(CRC_RM_CONVERGENT)
         if (strstr(cur_test->descr, "Test 1 SA8_SA8_SA32") != nullptr ||
-            strstr(cur_test->descr, "Test 10 SA8 k5x5 Mstr+Dil") != nullptr) {
+                strstr(cur_test->descr, "Test 10 SA8 k5x5 Mstr+Dil") != nullptr) {
             // Em9d fails bitwise comparison with reference .
             reporter.report_message(cur_test->descr, "SKIPPED due to a known issue");
             continue;