Possible bug when removing testMethods from the TestCluster

[gdenaro73]

Context

Please provide below a detailed introduction to the issue itself, and describe what you were doing when the issue happened. Or, what do you want to achieve?

I am guessing that I bumped into a bug within class org.evosuite.ga.archive.Archive. I describe my findings below.

Class org.evosuite.ga.archive.Archive includes the behavior of informing the TestCluster that a 'targetMethod' is fully covered (line 122, method removeNonCoveredTargetOfAMethod that calls method ignoreMethodCall) and, if so, that method gets removed from the collection testMethods (e.g. line 374). Note that testMethods is the collection that the TestCluster uses to select new random methods for inclusion in test cases.

This seems wrong because, even if a method is fully covered, it may anyway include side effects that are relevant for covering targets in other methods. For example, here is a case that I bumped into while running EvoSuite against a sample program.

In the initial phases of the test process some helper methods (they were setter methods) of the class under test were removed from the testMethods, since they easily got fully covered. At the same time, since EvoSuite was experiencing that those methods were not useful for covering new branches (at least before covering some challenging branches that dominated a relevant portion of the code) they also eventually disappeared from the individuals in the current population. A few minutes later, EvoSuite identified a test case that hit the (above mentioned) challenging branches, thus opening the possibility of testing all the code that depended on those branches. However, at that point, it was impossible for EvoSuite to generate test cases that included the needed setters (neither via mutations, nor as new random test cases) as those methods were not available in TestCluster.targetMethods. So EvoSuite was stuck and could not progress further.