add Flacoco as fault localization engine

[andre15silva]

fixes #216

[andre15silva]

Blocked by ASSERT-KTH/flacoco#81

[andre15silva]

There is a compatibility issue between flacoco and nopol.

flacoco requires access to the binaries, and nopol does not have this information available in any place other than being embedded in the classpath itself

[monperrus]

interesting, to be discussed.

[andre15silva]

interesting, to be discussed.

FYI, the new online mode (ASSERT-KTH/flacoco#86) does not need access to the binaries, only the names of the classes to include/exclude. Since we have access to the source files here in nopol that is taken care of.

[monperrus]

interesting

[andre15silva]

In the cm5 bug we find the correct line, but nopol itself isn't able to generate a patch.

From the paper, the buggy code is if (u * v == 0) {, which corresponds to org.apache.commons.math.util.MathUtils:412

This line is included in the results for both GZoltar and Flacoco (flacoco results below):

Failing tests:
[Manual]TestMethod=org.apache.commons.math.util.MathUtilsTest#testGcd_0
[Manual]TestMethod=org.apache.commons.math.util.MathUtilsTest#testGcd_3
[Manual]TestMethod=org.apache.commons.math.util.MathUtilsTest#testGcd_1
[Manual]TestMethod=org.apache.commons.math.util.MathUtilsTest#testGcd_4
SourceLocation org.apache.commons.math.util.MathUtils:413:0.5547001962252291
SourceLocation org.apache.commons.math.util.MathUtils:412:0.30151134457776363

[andre15silva]

Also note that Flacoco is not facing the same issue as GZoltar in #223 , and is able to compute the right failing tests.

[monperrus]

is Nopol in SMT or Dynamoth mode in this run?

[andre15silva]

is Nopol in SMT or Dynamoth mode in this run?

SMT is the default, but with I've tried with Dynamoth and it doesn't find a patch for cm5 either.

[andre15silva]

• cl2: Not working due to a bug in Jacoco (Missing line coverage for several examples jacoco/jacoco#1223 (comment))
• cl4: Not working due to a bug in Jacoco (Missing line coverage for several examples jacoco/jacoco#1223 (comment))
• cm1: Not working due to a bug in Jacoco (Missing line coverage for several examples jacoco/jacoco#1223 (comment))
• cm5: Not working due to a bug in Nopol (add Flacoco as fault localization engine #220 (comment))

[monperrus]

Per our discussion, will merge when CI is green.

[andre15silva]

For the record, cl4 works if we use jacoco/jacoco#321

[andre15silva]

cm7 is failing with flacoco 1.0.0 but works with flacoco 0.0.1-SNAPSHOT

[andre15silva]

• Fix example7Fix
• Use GZoltar for the ignored TSE bugs
• update to flacoco new release

[andre15silva]

Ready for review @monperrus

[monperrus]

Yeah, that's great @andre15silva thanks.