“Empirically revisiting the test independence assumption”
by Sai Zhang, Darioush Jalali, Jochen Wuttke, Kıvanç Muşlu, Wing Lam, Michael D. Ernst, and David Notkin.Two tests are independent (non-interfering) if running one test does not change whether the other test fails or succeeds. If tests are dependent, then reordering them (as in test prioritization or test selection) could cause a previously-passing test to suddenly start failing, even though the program has not changed.
Both practitioners and researchers generally ignore the problem of test dependence: they assume that test cases are independent. We developed presents algorithms and tools for detecting dependent tests. Our paper shows that test dependence does occur in real-world test suites, and it shows that test dependence can cause both false alarms and failed alarms in prioritized (reordered) test suites.
You can also read the abstract.
“Efficient mutation analysis by propagating and partitioning infected execution states”
by René Just, Michael D. Ernst, and Gordon Fraser.Abstract:
Given two test suites, mutation analysis ranks one of them as better than the other. Mutation analysis is notoriously compute-intensive, because it runs many variants of the program over each test suite. This paper reduces the cost of mutation analysis by 40%, by eliminating redundant executions that reveal no information.
The technique works by first running an instrumented version of the program over the test suite. This pre-processing step more than pays for itself by enabling other executions to be skipped:
- mutated expressions that produce the same value as the original program expression did,
- mutanted expressions that produce a different value, but a containing expression produces the same value as it did in the original program, and
- different mutants that all compute the same value (all but one of which can be skipped).
You can also read the abstract.
“A type system for format strings”
by Konstantin Weitz, Gene Kim, Siwakorn Srisakaokul, and Michael D. Ernst.Most programming languages support format strings, but their use is error-prone. Using the wrong format string syntax or passing the wrong number or type of arguments leads to unintelligible text output, program crashes, or security vulnerabilities.
We have designed and implemented a type system that indicates erroneous usage, or statically guarantees that calls to these APIs will never fail. The annotation burden for the user of our type system is low, and it found over 100 bugs in open-source projects.
You can also read the abstract.
