Beginner's guide to working with Evosuite plus plus

Introduction

This page will aim to cover:

1. A brief overview of how Evosuite++ works.
2. A brief overview of how the Evosuite++ project is structured.
3. How to look into the workings of Evosuite++ using the SF100 benchmark dataset.

A brief overview of how Evosuite++ works

Evosuite++ aims to generate unit test cases for methods to maximise branch coverage using search algorithms to gradually improve test cases to achieve coverage.

How the Evosuite++ project is structured

<TBD>

How to look into the workings of Evosuite++

One way to go about it is by looking at feature.objectconstruction.testgeneration.testcase.SF100OverallTest under evosuite-shell. This class provides an example of how to run Evosuite++ in the EvoObj configuration on a target method in some project in the SF100 benchmark dataset. Below is an example of the relevant code:

@Before
public void beforeTest() {
    // This MUST be set to allow breakpoints while debugging.
	Properties.CLIENT_ON_THREAD = true;
	Properties.STATISTICS_BACKEND = StatisticsBackend.DEBUG;
    ...
}

...

@Test
public void 
testBugExample() {
	// The target project
	String projectId = SF100Project.P1;	
	String[] targetMethods = new String[]{
		...
		// The target method 
		"com.ib.client.Contract#equals(Ljava/lang/Object;)Z"
		...
	};

	// Evosuite++ configuration
	int repeatTime = 5;
	int budget = 100;
	Long seed = null;
	String fitnessApproach = "branch";
	boolean aor = false;
	List<EvoTestResult> results = CommonTestUtil.evoTestSingleMethod(projectId, targetMethods, fitnessApproach, repeatTime, budget, true, seed, aor, "generateSuite", "Evosuite", "DynaMOSA");

	// Outcome reporting
	double coverage = 0;
	double initCoverage = 0;
	double time = 0;
	double iteration  = 0;
	for(EvoTestResult res: results) {
		coverage += res.getCoverage();
		initCoverage += res.getInitialCoverage();
		time += res.getTime();
		iteration += res.getAge();
	}
			
	System.out.println("applied object rule: " + aor);
	System.out.println("overall legitimization budget: " + Properties.TOTAL_LEGITIMIZATION_BUDGET);
	System.out.println("coverage: " + coverage/repeatTime);
	System.out.println("initCoverage: " + initCoverage/repeatTime);
	System.out.println("time: " + time/repeatTime);
	System.out.println("iteration: " + iteration/repeatTime);
}

Running this as a JUnit Test (in Eclipse, this can be done by accessing Run > Run As > JUnit Test) will trigger running Evosuite++ in the EvoObj configuration on the target method (in this case, com.ib.client.Contract#equals). The output can be viewed in the output window (either in your shell, on in the Console window if on Eclipse).

Sometimes, you might want to view the test cases as they evolve. This can be done by setting breakpoints in the appropriate class. In our case, this is org.evosuite.ga.metaheuristics.mosa.DynaMOSA. Evosuite++ works by evolving the test cases iteratively; in our example, this is done in org.evosuite.ga.metaheuristics.mosa.DynaMOSA#evolve. As such, we can see how the population changes with each iteration by setting a breakpoint at the printBestFitness() call in line 187, in the evolve method. When the breakpoint is reached, we can see the current population by inspecting this.population.