DMN Engine Rule #170

AntoCuc · 2018-10-12T08:44:10Z

AntoCuc
Oct 12, 2018

Implementation of a DMN Engine Rule in order to show coverage of evaluation of decision tables.

falko · 2018-10-12T13:08:02Z

falko
Oct 12, 2018
Collaborator

@AntoCuc would you like to propose an implementation in a pull request?

0 replies

AntoCuc · 2018-10-12T16:27:02Z

AntoCuc
Oct 12, 2018
Author

@falko sure thing.

I am trying to understand the code and would be well glad if somebody from the project can tell me whether I am going insane with the approach.

So far I understand (from the most linear use case):

PathCoverageExecutionListener is registered with the Process Engine via StandaloneInMemProcessEngineConfiguration
PathCoverageExecutionListener allows registration of the PathCoverageExecutionListener
PathCoverageExecutionListener, on notification, adds the element to the runState as covered

Is my understanding inline with the team's?

I created a JUnit Test (I normally work with TDD)

    @Test
    @Deployment(resources = {"Example.dmn"})
    public void testBronze() {
        final Map<String, Object> variables = new HashMap<String, Object>();
        variables.put("status", "bronze");
        variables.put("sum", 0);

        final DmnDecisionTableResult result =
                classRule.getDecisionService().evaluateDecisionTableByKey(DECISION_DEFINITION_KEY, variables);
        assertEquals(1, result.size());
        assertEquals("notok", result.getFirstResult().get("result"));
        assertEquals("work on your status first, as bronze you're not going to get anything", result.getFirstResult().get("reason"));
        classRule.addTestMethodCoverageAssertionMatcher("testBronze", new TypeSafeMatcher<Double>() {
            @Override
            protected boolean matchesSafely(Double value) {
                return !Double.isNaN(value);
            }

            @Override
            public void describeTo(Description description) {
                description.appendText("a number rather than NaN");
            }
        });
    }

which yields:

"C:\Program Files\Java\jdk1.8.0_65\bin\java.exe" -ea -Didea.test.cyclic.buffer.size=1048576 "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2018.1.5\lib\idea_rt.jar=53599:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2018.1.5\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2018.1.5\lib\idea_rt.jar;C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2018.1.5\plugins\junit\lib\junit-rt.jar;C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2018.1.5\plugins\junit\lib\junit5-rt.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_65\jre\lib\rt.jar;C:\source\camunda-bpm-process-test-coverage\test\target\test-classes;C:\source\camunda-bpm-process-test-coverage\core\target\classes;C:\Users\antonio\.m2\repository\org\camunda\bpm\camunda-engine\7.7.0\camunda-engine-7.7.0.jar;C:\Users\antonio\.m2\repository\org\camunda\bpm\model\camunda-bpmn-model\7.7.0\camunda-bpmn-model-7.7.0.jar;C:\Users\antonio\.m2\repository\org\camunda\bpm\model\camunda-xml-model\7.7.0\camunda-xml-model-7.7.0.jar;C:\Users\antonio\.m2\repository\org\camunda\bpm\model\camunda-cmmn-model\7.7.0\camunda-cmmn-model-7.7.0.jar;C:\Users\antonio\.m2\repository\org\camunda\bpm\dmn\camunda-engine-dmn\7.7.0\camunda-engine-dmn-7.7.0.jar;C:\Users\antonio\.m2\repository\org\camunda\commons\camunda-commons-utils\1.4.0\camunda-commons-utils-1.4.0.jar;C:\Users\antonio\.m2\repository\org\camunda\bpm\model\camunda-dmn-model\7.7.0\camunda-dmn-model-7.7.0.jar;C:\Users\antonio\.m2\repository\org\camunda\bpm\dmn\camunda-engine-feel-api\7.7.0\camunda-engine-feel-api-7.7.0.jar;C:\Users\antonio\.m2\repository\org\camunda\bpm\dmn\camunda-engine-feel-juel\7.7.0\camunda-engine-feel-juel-7.7.0.jar;C:\Users\antonio\.m2\repository\org\camunda\commons\camunda-commons-logging\1.4.0\camunda-commons-logging-1.4.0.jar;C:\Users\antonio\.m2\repository\org\slf4j\slf4j-api\1.7.7\slf4j-api-1.7.7.jar;C:\Users\antonio\.m2\repository\org\camunda\commons\camunda-commons-typed-values\1.4.0\camunda-commons-typed-values-1.4.0.jar;C:\Users\antonio\.m2\repository\org\apache\commons\commons-email\1.2\commons-email-1.2.jar;C:\Users\antonio\.m2\repository\javax\mail\mail\1.4.1\mail-1.4.1.jar;C:\Users\antonio\.m2\repository\javax\activation\activation\1.1\activation-1.1.jar;C:\Users\antonio\.m2\repository\org\mybatis\mybatis\3.2.8\mybatis-3.2.8.jar;C:\Users\antonio\.m2\repository\org\springframework\spring-beans\3.1.2.RELEASE\spring-beans-3.1.2.RELEASE.jar;C:\Users\antonio\.m2\repository\org\springframework\spring-core\3.1.2.RELEASE\spring-core-3.1.2.RELEASE.jar;C:\Users\antonio\.m2\repository\commons-logging\commons-logging\1.1.1\commons-logging-1.1.1.jar;C:\Users\antonio\.m2\repository\joda-time\joda-time\2.1\joda-time-2.1.jar;C:\Users\antonio\.m2\repository\org\camunda\bpm\camunda-engine-spring\7.7.0\camunda-engine-spring-7.7.0.jar;C:\Users\antonio\.m2\repository\org\springframework\spring-context\3.1.2.RELEASE\spring-context-3.1.2.RELEASE.jar;C:\Users\antonio\.m2\repository\org\springframework\spring-aop\3.1.2.RELEASE\spring-aop-3.1.2.RELEASE.jar;C:\Users\antonio\.m2\repository\org\springframework\spring-expression\3.1.2.RELEASE\spring-expression-3.1.2.RELEASE.jar;C:\Users\antonio\.m2\repository\org\springframework\spring-asm\3.1.2.RELEASE\spring-asm-3.1.2.RELEASE.jar;C:\Users\antonio\.m2\repository\org\springframework\spring-jdbc\3.1.2.RELEASE\spring-jdbc-3.1.2.RELEASE.jar;C:\Users\antonio\.m2\repository\org\springframework\spring-tx\3.1.2.RELEASE\spring-tx-3.1.2.RELEASE.jar;C:\Users\antonio\.m2\repository\aopalliance\aopalliance\1.0\aopalliance-1.0.jar;C:\Users\antonio\.m2\repository\commons-lang\commons-lang\2.4\commons-lang-2.4.jar;C:\Users\antonio\.m2\repository\org\springframework\spring-orm\3.1.2.RELEASE\spring-orm-3.1.2.RELEASE.jar;C:\Users\antonio\.m2\repository\commons-dbcp\commons-dbcp\1.4\commons-dbcp-1.4.jar;C:\Users\antonio\.m2\repository\commons-pool\commons-pool\1.5.4\commons-pool-1.5.4.jar;C:\Users\antonio\.m2\repository\commons-io\commons-io\2.5\commons-io-2.5.jar;C:\Users\antonio\.m2\repository\org\apache\commons\commons-lang3\3.4\commons-lang3-3.4.jar;C:\Users\antonio\.m2\repository\org\hamcrest\hamcrest-library\1.3\hamcrest-library-1.3.jar;C:\Users\antonio\.m2\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;C:\Users\antonio\.m2\repository\junit\junit\4.12\junit-4.12.jar;C:\Users\antonio\.m2\repository\com\h2database\h2\1.3.168\h2-1.3.168.jar;C:\Users\antonio\.m2\repository\org\springframework\spring-test\3.1.2.RELEASE\spring-test-3.1.2.RELEASE.jar;C:\Users\antonio\.m2\repository\cglib\cglib\2.2.2\cglib-2.2.2.jar;C:\Users\antonio\.m2\repository\asm\asm\3.3.1\asm-3.3.1.jar" com.intellij.rt.execution.junit.JUnitStarter -ideVersion5 -junit4 org.camunda.bpm.extension.process_test_coverage.dmn.DmnBusinessRuleTaskCoverageTest
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Oct 16, 2018 3:49:27 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [camunda.cfg.xml]
Oct 16, 2018 3:49:35 PM org.camunda.bpm.extension.process_test_coverage.junit.rules.TestCoverageProcessEngineRule succeeded
INFO: testBronze(org.camunda.bpm.extension.process_test_coverage.dmn.DmnBusinessRuleTaskCoverageTest) succeeded.
Oct 16, 2018 3:49:35 PM org.camunda.bpm.extension.process_test_coverage.junit.rules.TestCoverageProcessEngineRule handleTestMethodCoverage
INFO: testBronze test method coverage is NaN
Oct 16, 2018 3:49:36 PM org.camunda.bpm.extension.process_test_coverage.junit.rules.TestCoverageProcessEngineRule succeeded
INFO: org.camunda.bpm.extension.process_test_coverage.dmn.DmnBusinessRuleTaskCoverageTest succeeded.

java.lang.AssertionError: 
Expected: a number rather than NaN
     but: was <NaN>
Expected :a number rather than NaN
     
Actual   :<NaN>

My plan would be to practically create an equivalent workflow with the DMN. I can see from the API there is a DmnDecisionEvaluationListener, a DmnDecisionEvaluationEvent that could be leveraged.
Also the DmnDecisionTableImpl would represent the model that extended might contain coverage data.

Off the mark?

0 replies

AntoCuc · 2018-10-15T09:13:20Z

AntoCuc
Oct 15, 2018
Author

Mmm, I started putting together the code for the registration of the engine events and there appears to be a slight abstraction problem.

The current coverage plugin only works on processes due to the ProcessEngineConfigurationImpl exposing the hooks to register the listeners independently of the initialisation state of the engine. In fact it registers the listeners before initialisation. :(

On the other hand, the DefaultDmnEngineConfiguration does not allow for the registration of the listeners until engine initialisation.

The above can be easily verified by pasting the code below in the camunda.cfg.xml :

<property name="dmnEngineConfiguration.customPostDecisionTableEvaluationListeners">
	<list>
		<bean class="org.camunda.bpm.extension.process_test_coverage.listeners.MyCustomPostDecisionTableEvaluationListener" />
	</list>
</property>

which should, in theory, register the DMN MyCustomPostDecisionTableEvaluationListener but fails with:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'processEngineConfiguration' defined in class path resource [camunda.cfg.xml]: Error setting property values; nested exception is org.springframework.beans.NullValueInNestedPathException: Invalid property 'dmnEngineConfiguration' of bean class [org.camunda.bpm.engine.impl.cfg.StandaloneInMemProcessEngineConfiguration]: Value of nested property 'dmnEngineConfiguration' is null

Potential solutions:

Expose the DMN-related listeners on the ProcessEngineConfigurationImpl?
Alternatively call the init of the engine a little earlier?

Any views?

0 replies

falko · 2018-10-15T19:26:22Z

falko
Oct 15, 2018
Collaborator

With a process engine plugin, you should also be able to access the configuration after initialization.

0 replies

AntoCuc · 2018-10-16T15:00:33Z

AntoCuc
Oct 16, 2018
Author

@falko true. But that wouldn't help gathering coverage stats.

0 replies

falko · 2018-10-17T03:31:48Z

falko
Oct 17, 2018
Collaborator

Why not simply querying the history? That's how process test coverage started as well. On Oct 16, 2018, at 17:00, Antonino Cucchiara <[email protected]<mailto:[email protected]>> wrote: @falko<https://github.com/falko> true. But that wouldn't help gathering coverage stats. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#42 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AACcZgoAXnsUR_3b6ivWTCHAsuti5JLQks5ulfSXgaJpZM4XZCyh>.

0 replies

StephenOTT · 2018-10-17T11:29:14Z

StephenOTT
Oct 17, 2018

What is the expected output of DMN coverage statistics?

You are essentially looking to test inputs and expected output?

Something like: https://github.com/DigitalState/Camunda-Spock-Testing/tree/master/DMN

Would like to understand what the DMN coverage looks like so it can be added to https://github.com/DigitalState/camunda-coverage-generation

0 replies

AntoCuc · 2018-10-17T13:19:16Z

AntoCuc
Oct 17, 2018
Author

Hi @StephenOTT

What is the expected output of DMN coverage statistics?

The expected output would be a percentage assertable in a similar fashion to the BPM process coverage.

You are essentially looking to test inputs and expected output?

Nope. That behavior can already be obtained as in the unit test sample I wrote above.

Would like to understand what the DMN coverage looks like so it can be added to [...]

At the moment I am not trying to address the visual aspect of it. Rather the practical unit testing in order to integrate DMN with Continous Deployment systems.

0 replies

StephenOTT · 2018-10-17T13:42:09Z

StephenOTT
Oct 17, 2018

@AntoCuc but what is your percentage logic based on? The coverage percentage calc seems to be very different compared to bpmn.

0 replies

AntoCuc · 2018-10-17T14:03:18Z

AntoCuc
Oct 17, 2018
Author

@StephenOTT

what is your percentage logic based on?

My understanding is that:
BPMN calculates coverage percentage on a per-method basis and per-class basis vs. the full BPMN diagram.

I am planning to approach DMN in a similar fashion. Ie. the per-method approach will display decisions coverage of a test method in the context of the diagram. In a similar vein the per-class approach will display an aggregate decision coverage in the context of the diagram.

Am I missing something here?

0 replies

StephenOTT · 2018-10-17T17:38:45Z

StephenOTT
Oct 17, 2018

@AntoCuc maybe i am missing something :s . I am missing what is actually being tested in this case.
What methods and class level stats are being covered? Are you talking about the expressions and scripts you are adding into a DMN?

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DMN Engine Rule #170

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 11 comments

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

DMN Engine Rule #170

AntoCuc Oct 12, 2018

Replies: 11 comments

falko Oct 12, 2018 Collaborator

AntoCuc Oct 12, 2018 Author

AntoCuc Oct 15, 2018 Author

falko Oct 15, 2018 Collaborator

AntoCuc Oct 16, 2018 Author

falko Oct 17, 2018 Collaborator

StephenOTT Oct 17, 2018

AntoCuc Oct 17, 2018 Author

StephenOTT Oct 17, 2018

AntoCuc Oct 17, 2018 Author

StephenOTT Oct 17, 2018

AntoCuc
Oct 12, 2018

falko
Oct 12, 2018
Collaborator

AntoCuc
Oct 12, 2018
Author

AntoCuc
Oct 15, 2018
Author

falko
Oct 15, 2018
Collaborator

AntoCuc
Oct 16, 2018
Author

falko
Oct 17, 2018
Collaborator

StephenOTT
Oct 17, 2018

AntoCuc
Oct 17, 2018
Author

StephenOTT
Oct 17, 2018

AntoCuc
Oct 17, 2018
Author

StephenOTT
Oct 17, 2018