diff --git a/README.md b/README.md
index a050d1e..0811560 100644
--- a/README.md
+++ b/README.md
@@ -1,137 +1,64 @@
-# Table of contents
-- [Test definition](#test-definition)
- * [Unit test](#unit-test)
- * [Test proprieties](#test-proprieties)
- + [Critical test](#critical-test)
- + [Long run test](#long-run-test)
- + [Test template](#test-template)
- * [Test suit](#test-suit)
- * [Test setup](#test-setup)
- * [Setup test directory](#setup-test-directory)
+## 0. Setup test directory
-# Test definition
-## Unit test
-A unit test can be defined to evaluate a procedure (block of code).
+The provided directory tree below illustrates an example of how to use the
+framework:
-## Test proprieties
-For each test, the following parameters can be set:
-
-### Critical test
-- True - The test process should be stopped if the test fails
-- False - The test process can continue even if the test fails
-
-### Long run test
-- True - The test duration is considered long (long-time run)
-- False - The test is considered fast (short-time run
-
-### Test template
-A test definition must follow the outlined nomenclature.
-
-```C
-BAO_TEST(#suite_name, #test_name, #criticality, #long_run)
-{
- // test code here
-}
+```c
+ MUT
+ ├── ci
+ ├── src
+ ├── tests
+ │ ├── configs
+ │ │ ├── test_cfg1.dts
+ │ │ ├── test_cfg2.dts
+ │ ├── src
+ │ │ ├── test_src1.c
+ │ │ ├── test_src2.c
+ │ ├── bao-tests (git repository)
+ │ ├── bao-nix (git repository)
```
-## Test suit
-The term "test suite" refers to a group of tests. Grouping the tests enables a more organized selection of tests, making running numerous tests easier.
-
-
-
-
- | Suit name |
- Test name |
- Critical test |
- Long test |
-
-
-
-
- Suit_1
|
- Test_1 |
- T/F |
- T/F |
-
-
- | Test_2 |
- T/F |
- T/F |
-
-
- | Test_3 |
- T/F |
- T/F |
-
-
- Suit_2
|
- Test_1 |
- T/F |
- T/F |
-
-
- | Test_2 |
- T/F |
- T/F |
-
-
- | Test_3 |
- T/F |
- T/F |
-
-
-
-
-## Test setup
-In order to use Bao Test Framework you have go throught the following steps:
-
-1. Add the following lines to your linkerscript
- ```c
- .testframework : {
- __testframework_start = . ;
- *(.testframework.PRIO* .testframework.prio*)
- *(.testframework.*)
- __testframework_end = . ;
- }
- ```
-
-2. Add the following lines to your makefile (use the variable TEST_DIR to indicate the directory where all the file containing tests are. The variable REPO_DIR should point to the base directory of this repository. Additionally, the variables names C_SRC, INC_DIRS, and CFLAGS may have to be modified according to your makefile.)
-
- ```c
- TESTS_DIR:=$(SRC_DIR)/tests
- REPO_DIR:=$(SRC_DIR)/../bao-tests
- include bao-tests/src/bao-test.mk
- C_SRC += $(BAO_TEST_SRCS)
- INC_DIRS += $(BAO_TEST_INC_DIR)
- CFLAGS += $(BAO_TEST_FLAGS)
- ```
-(Note: If you are using the C library without the Python tool, make sure you add also delete the test build directory in your clean rule)
-
-3. Add to your source the entry point (bao_test_entry()) to our framework, where you feel it is correct.
-4. Create a .c file (or multiple) in the directory specified early, include bao_test.h and create tests according to the [test definition example](#Test template).
-5. Build your system normally adding the following variable to your make command:
+## 1. Get Bao-nix repository
+```sh
+git clone https://github.com/bao-project/bao-nix.git
+```
- ```c
- BAO_TEST=1 -> mandatory to use the framework, otherwise no test will be executed
- SUITES="suite1 suite2" -> to specify which suites need to be executed
- TESTS="test1 test2" -> to specify which tests are to bexecuted
- ```
+## 2. Get Bao-tests repository
+```sh
+git clone https://github.com/bao-project/bao-tests.git
+```
-## Setup test directory
-The test platform directory provides a template of a VMM, a VM, and a Guest. These templates are available in the BAO_Test/ directory. A test relating to a MUT should be placed in that module's directory. Tests must be grouped into suits (directory name equals to the suit name) inside the module directory.
+## 3. How to configure
+Configuration of the test framework is performed through a .dts file. This file allows you to define the test recipe, select the target platform, and specify
+the tests to be run in a given setup. The configuration file follows this
+template:
+```dts
+/dts-v1/;
+/ {
+ platform = "test-target-platform";
+
+ test_config {
+ recipe_test {
+ nix_file = "test-recipe.nix";
+ suites = "list-of-suites";
+ tests = "list-of-tests";
+ log_level = "verbose-level";
+ };
+ };
+};
+```
+## 4. How to use
-```c
-BAO_Test/
-├─ VMM_template
-├─ VM_template
-├─ VM_Test/
-│ ├─ Suit_1/
-│ │ ├─ Test_1
-│ │ ├─ Test_n
-│ ├─ Suit_n/
-│ │ ├─ Test_1
-│ │ ├─ Test_n
-├─ VMM_Test/
-├─ Guest_Test/
+After setting up the directory, run the following commands:
+```sh
+cd /path-to-MUT-dir/tests/bao-tests/framework/
+python3 test_framework.py
```
+Alternatively, you can launch the framework with the following arguments:
+```sh
+python3 test_framework.py \
+ -dts_path /path/to/config.dts \ # config.dts file
+ -bao_test_src_path /path/to/bao-tests/src \ # bao-tests repo (/src)
+ -tests_src_path /path/to/tests/src # tests to be executed
+```
diff --git a/codegen.py b/framework/codegen.py
similarity index 59%
rename from codegen.py
rename to framework/codegen.py
index 3ff1abb..e0be596 100644
--- a/codegen.py
+++ b/framework/codegen.py
@@ -1,9 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# Copyright (c) Bao Project and Contributors. All rights reserved
-import sys
-import argparse
-import shutil
-import os
"""
This script is used to generate Bao Project tests code.
@@ -11,8 +7,18 @@
corresponding test functions.
"""
+import sys
+import argparse
+import shutil
+import os
def parse_args():
+ """
+ Function to parse command-line arguments for generating tests code.
+
+ Returns:
+ args (argparse.Namespace): Parsed command-line arguments.
+ """
parser = argparse.ArgumentParser(description='Script to parse tests \
sourcesand generate tests code')
@@ -29,20 +35,38 @@ def parse_args():
def get_srcs_list(base_dir):
+ """
+ Function to retrieve a list of C source files from the specified directory.
+
+ Args:
+ base_dir (str): Base directory to search for source files.
+
+ Returns:
+ list: List of C source file paths.
+ """
c_srcs = []
- for root, dirs, files in os.walk(base_dir):
+ for root, _, files in os.walk(base_dir):
for file in files:
if file.endswith(".c"):
c_srcs.append(os.path.join(root, file))
return c_srcs
def generate_code(base_dir):
+ """
+ Function to generate code based on specified C source files.
+
+ Args:
+ base_dir (str): Base directory containing C source files.
+
+ Returns:
+ str: Generated code.
+ """
c_files = get_srcs_list(base_dir)
tests_list = {}
code = ""
for file in c_files:
- with open(file, "r") as f:
- file_code = f.readlines()
+ with open(file, "r", encoding="utf8") as c_file:
+ file_code = c_file.readlines()
for line in file_code:
if "BAO_TEST" in line:
@@ -60,11 +84,11 @@ def generate_code(base_dir):
else:
tests_list[suite_name] = [test_name]
- for suite in tests_list.keys():
- for test in tests_list[suite]:
- code += "\t#if defined " + test + " || " + suite + "\n"
- code += "\tentry_test_" + suite + "_" + test + "();" + "\n"
- code += "\t#endif" + "\n" + "\n"
+ for suite, tests in tests_list.items():
+ for test in tests:
+ code += f"\t#if defined {test} || {suite}\n"
+ code += f"\tentry_test_{suite}_{test}();\n"
+ code += "\t#endif\n\n"
return code[:-2]
@@ -75,29 +99,29 @@ def generate_code(base_dir):
tests_code = generate_code(tool_args.base_dir)
# Copy template to output directory
- template_file = "./template.c"
- if not os.path.isfile(template_file):
+ TEMPLATE_FILE = "../src/template.c"
+ if not os.path.isfile(TEMPLATE_FILE):
print("Template file missing!")
sys.exit()
- shutil.copy(template_file, tool_args.out_code)
+ shutil.copy(TEMPLATE_FILE, tool_args.out_code)
# Read template
- with open(tool_args.out_code, "r") as f:
- code = f.readlines()
+ with open(tool_args.out_code, "r", encoding="utf8") as code_file:
+ read_code = code_file.readlines()
# Get codegen.py writable sections
code_sec_begin, code_sec_end = -1, -1
- for index, line in enumerate(code):
- if "// codegen.py section begin" in line:
+ for index, code_line in enumerate(read_code):
+ if "// codegen.py section begin" in code_line:
code_sec_begin = index
- if "// codegen.py section end" in line:
+ if "// codegen.py section end" in code_line:
code_sec_end = index
# Write generated code to output file
- out_code = ''.join(code[0:code_sec_begin+1])
- out_code += tests_code
- out_code += ''.join(code[code_sec_end:])
- with open(tool_args.out_code, "w") as f:
- f.writelines(out_code)
+ OUT_CODE = ''.join(read_code[0:code_sec_begin+1])
+ OUT_CODE += tests_code
+ OUT_CODE += ''.join(read_code[code_sec_end:])
+ with open(tool_args.out_code, "w", encoding="utf8") as out_file:
+ out_file.writelines(OUT_CODE)
print("Successfully generated bao tests code")
diff --git a/framework/test_framework.py b/framework/test_framework.py
index 22a8576..d6d0476 100644
--- a/framework/test_framework.py
+++ b/framework/test_framework.py
@@ -5,10 +5,171 @@
Test framework main file
"""
import argparse
+import os
+import sys
+import subprocess
+import psutil
import constants as cons
+from pydevicetree import Devicetree
-PARSER = argparse.ArgumentParser(description="Bao Testing Framework")
+test_config = {
+ 'nix_file': '',
+ 'suites': '',
+ 'tests': '',
+ 'tests_configs': {
+ 'log_level': ''
+ }
+}
+
+def parse_args():
+ """
+ Parse python script arguments.
+ """
+ parser = argparse.ArgumentParser(description="Bao Testing Framework")
+
+ parser.add_argument("-dts_path", "--dts_path",
+ help="Path to .dts configuration file",
+ default="../../configs/config.dts")
+
+ parser.add_argument("-bao_test_src_path", "--bao_test_src_path",
+ help="Path to bao-test /src dir",
+ default="../src")
+
+ parser.add_argument("-tests_src_path", "--tests_src_path",
+ help="Path to bao-test /src dir",
+ default="../../src")
+
+ input_args = parser.parse_args()
+ return input_args
+
+def parse_dts_file(file_path):
+ """
+ Parse a DTS (Device Tree Source) file and extract relevant information.
+
+ Args:
+ file_path (str): The path to the DTS configuration file.
+ """
+ tree = Devicetree.parseFile(file_path)
+ test_config['platform'] = \
+ tree.children[0].properties[0].values[0]
+ test_config['nix_file'] = \
+ tree.children[0].children[0].children[0].properties[0].values[0]
+ test_config['suites'] = \
+ tree.children[0].children[0].children[0].properties[1].values[0]
+ test_config['tests'] = \
+ tree.children[0].children[0].children[0].properties[2].values[0]
+ test_config['tests_configs']['log_level'] = \
+ tree.children[0].children[0].children[0].properties[2].values[0]
+
+def run_command_in_terminal(command):
+ """
+ Run a command in a new Terminal window.
+
+ Args:
+ command (str): The command to execute.
+ """
+ # pylint: disable=R1732
+ terminal_process = subprocess.Popen(
+ ['/bin/bash', '-c', command],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE
+ )
+ # pylint: enable=R1732
+
+ return terminal_process
+
+def terminate_children_processes(parent_process):
+ """
+ Terminate all child processes of the given parent process.
+
+ Args:
+ parent_process: The parent process whose children will be terminated.
+ """
+ parent = psutil.Process(parent_process.pid)
+ children = parent.children(recursive=True)
+ for child in children:
+ child.terminate()
+ child.wait()
+
+def deploy_test(platform):
+ """
+ Deploy a test on a specific platform.
+
+ Args:
+ platform (str): The platform to deploy the test on.
+ """
+ if platform in ["qemu-aarch64-virt", "qemu-riscv64-virt"]:
+ arch = platform.split("-")[1]
+ run_cmd = "../platform/qemu/run.sh " + arch
+ process = run_command_in_terminal(run_cmd)
+ # Connection to platform will run here
+ # which will stall the platform untill the results
+ # are sent from the platform to the framework
+ terminate_children_processes(process)
if __name__ == '__main__':
print(cons.BLUE_TEXT + "Framework init..." + cons.RESET_COLOR)
+ print(cons.BLUE_TEXT +
+ "Framework init..." +
+ cons.RESET_COLOR)
+
+ args = parse_args()
+
+ print(cons.BLUE_TEXT +
+ "Reading config.dts..." +
+ cons.RESET_COLOR)
+
+ dts_path = args.dts_path
+ print("config.dts file: " + dts_path)
+
+ if args.dts_path is None:
+ print(cons.RED_TEXT +
+ "Error: Please provide the --dts_path argument." +
+ cons.RESET_COLOR)
+ else:
+ dts_path = args.dts_path
+
+ parse_dts_file(dts_path)
+ print(cons.GREEN_TEXT + "config.dts successfully read!" + cons.RESET_COLOR)
+
+ print(cons.BLUE_TEXT +
+ "Creating tests source file..." +
+ cons.RESET_COLOR)
+
+ bao_test_src = args.bao_test_src_path
+ tests_src = args.tests_src_path
+ RUN_CMD = "python3 codegen.py -dir " + tests_src + " "
+ RUN_CMD += "-o " + bao_test_src + "/testf_entry.c"
+ os.system(RUN_CMD)
+
+ print(cons.BLUE_TEXT + "Running nix build..." + cons.RESET_COLOR)
+ BUILD_CMD = 'nix-build ../../' + test_config['nix_file']
+ list_suites = test_config['suites'].split()
+ list_tests = test_config['tests'].split()
+ BUILD_CMD += " --argstr platform " + test_config['platform']
+ if len(list_suites):
+ BUILD_CMD += " --argstr list_suites \""
+ for suit in list_suites:
+ BUILD_CMD += suit + " "
+ BUILD_CMD = BUILD_CMD[:-1] + "\""
+ if len(list_tests):
+ BUILD_CMD += " --argstr list_tests \""
+ for suit in list_tests:
+ BUILD_CMD += suit + " "
+ BUILD_CMD = BUILD_CMD[:-1] + "\""
+
+ print(BUILD_CMD)
+ res = os.system(BUILD_CMD)
+ if res==0:
+ print(cons.GREEN_TEXT +
+ "nix build successfully completed..." +
+ cons.RESET_COLOR)
+
+ else:
+ print(cons.RED_TEXT +
+ "nix build failed..." +
+ cons.RESET_COLOR)
+ sys.exit(-1)
+
+ print(cons.BLUE_TEXT + "Launching QEMU..." + cons.RESET_COLOR)
+ deploy_test(test_config['platform'])
diff --git a/template.c b/src/template.c
similarity index 52%
rename from template.c
rename to src/template.c
index 6e0da79..046f631 100644
--- a/template.c
+++ b/src/template.c
@@ -1,8 +1,8 @@
/*
-* Copyright (c) Bao Project and Contributors. All rights reserved
-*
-* SPDX-License-Identifier: Apache-2.0
-*/
+ * Copyright (c) Bao Project and Contributors. All rights reserved
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
#include "testf.h"
#include
@@ -11,7 +11,6 @@
unsigned int testframework_tests;
unsigned int testframework_fails;
-
void testf_entry(void)
{
// codegen.py section begin
@@ -19,10 +18,10 @@ void testf_entry(void)
// codegen.py section end
if (testframework_tests > 0) {
- LOG_TESTS();
- } else {
- INFO_TAG();
- printf("No tests were executed!\n");
- }
- return;
+ LOG_TESTS();
+ } else {
+ INFO_TAG();
+ printf("No tests were executed!\n");
+ }
+ return;
}