-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merge artifacts (properties, status, tests) in single table #136
Comments
Used Pandas to merge the tables (Status, Properties and Results)
Markdown:
|
@kebi-be The contents are missing here? (I saw them in the mail) Could you make sure to drop the rendundant How do you handle instructions which do not have tests available? (an inner-join would drop those but we should have information about the non-existent tests in the table) I do not understand how you combine the status and test results in the same row? For each instruction there are multiple test and multiple status rows. |
I deleted the files as I noticed the redundant columns in the df. Instructions without tests will simple have an NaN value. I will update this. I will add NaN in the empty fields of the non-existing tests. I first merge the status and properties tables (inner join) and then merge the results to the status_prop_table. Since for the OpenASIP_base_MAX tests results there are 5 different tests and for this instruction there are 10 different passes from the status table. So combining these will results to the multiple rows (5x10). What are your expectations. Otherwise I donot see any other way of combining these. |
I think we have to aggregate the data to display it in a meaningful way. Of course we will loose some information (i.e. names of failing passes/tests) but should get some overall (per instruction, per-class, per-set, total) metrics on how good Seal5 peforms. In the following comments I will post some ideas on how the aggregations could be done. |
For the Status Data (last row aggregates over a instructions in the set)
|
Same idea for the test results data (added dummy test data for MAXU):
|
I also had the idea to give a metric for the test coverage of each instruction. Here is an example of what I mean: A Test Coverage of 100% means the following tests (
Here is a table with some manually entered coverage data:
|
@kebi-be As requested, I have added a All artifacts can be found here: Minimal Example: Passes
Test Results
|
An initial test coverage analysis is also available using the new Example Output:
|
No description provided.
The text was updated successfully, but these errors were encountered: