-
Notifications
You must be signed in to change notification settings - Fork 344
Regression Testing
Using EvoSuite's Regression test suite generation (EvoSuiteR), you can generate a test suite revealing differences between two versions of a Java class. This tutorial guides you through generating a differential test using EvoSuiteR, on the command-line.
- Java 8 JDK
- EvoSuite itself (Download Latest evosuite.jar)
- A pair of compiled Java classes
Given that EvoSuiteR requires a pair of versions, the general structure of the commands are as follows:
java -jar evosuite.jar -regressionSuite -projectCP "$originalCP" -Dregressioncp="$regressionCP" <target> [options]In the command above, $originalCP refers to the Java class path to the original version of the program, and $regressionCP refers to the class path to the modified version of the program.
The term original in our context refers to the version which EvoSuiteR considers as correct, and the term regression refers to the version which EvoSuiteR considers to have potential regressions. Thus, the generated tests are intended to pass on the original version, and fail on the regression version.
Consider you have a Java class my.pkg.HelloWorld, that is compiled and available under original/build/classes/my/pkg/HelloWorld.class. After you make a change to this class, you compile a new version under new/build/classes/my/pkg/HelloWorld.class.
To generate a regression test suite revealing the change between the two classes, you can run the following command:
java -jar evosuite.jar -regressionSuite -projectCP "original/build/classes" -Dregressioncp="new/build/classes" -class my.pkg.HelloWorldClass dependencies can also be provided to the projectCP/regressioncp, using the Java class-path structure. Additional options such as the search budget can be provided using the same properties as EvoSuite options.
EvoSuiteR can also be used to generate tests for all class pairs between two .jar packages.
Assuming we have a program a/project.jar and a new version of this program b/project.jar, and the two programs have different set of dependencies – which for the sake of brevity, we only include one dependency, but as many dependencies can be put on the classpath – the following command can be used to generate a test suite for all pairs of classes across
the two versions:
java -jar evosuite.jar -regressionSuite -projectCP "a/project.jar:a/dependency.jar" -Dregressioncp="b/project.jar:b/dependency.jar" -target project.jarWhen generating differential tests across two versions of a program, it is likely that not all classes have been modified across the two versions. EvoSuiteR has a functionality to only generate tests for pairs of classes where the underlying code has been changed. To enable this, you can add the option -Dregression_skip_similar=true.