Compare with competitors
What is the difference between Infection and other PHP mutation testing frameworks?
Well, there is only one competitor at the moment - Humbug.
The main and the best difference is that Infection uses AST (Abstract Syntax Tree) to mutate the code. It brings so much value to the framework:
- Much easier to support code
- Much easier to write new Mutators
- Much easier to handle false-positives and different edge cases, e.g. deciding when mutation should be done or should not in difficult situation
Let’s look at the following examples to understand how AST helps here. This is an implementation of mutator in Infection:
The code is very clear. The class is small. Here we are working with objects that represent nodes of our code in an Object Oriented way.
Let’s look at the Humbug’s implementation of the same mutator that replaces
class Addition extends MutatorAbstract
Do you see the difference? Just imagine how difficult to understand and support such code.
I don’t say that Humbug’s code is bad, no. Here I just want to show you that using AST brings so much value and simplifies the code a lot.
If you still don’t believe that AST is a thing, just compare these two mutators that replace
return functionCall(); with
functionCall(); return null;:
- Humbug: https://github.com/humbug/humbug/blob/1.0.0-alpha2/src/Mutator/ReturnValue/FunctionCall.php
- Infection: https://github.com/infection/infection/blob/0.2.1/src/Mutator/ReturnValue/FunctionCall.php
I spent much time understanding how we can get rid of PHPUnit and PhpSpec dependencies for production build of Infection.
In order to run particular (not all) tests from the suite for each Mutation, Humbug subscribes to the PHPUnit process with its own listener and filters out not needed tests. So it has a listener that extends PHPUnit’s
\PHPUnit_Framework_BaseTestListener class. This couples Humbug with PHPUnit, even with particular version(s).
Obviously, if the code you are mutating uses a different (incompatible) version of PHPUnit, you can not use Humbug in this case.
Ok, the answer is
XML reports. PHPUnit generates the same (almost ;))
XML for running test suite where we can find information about executed lines of code, time of each test case, the original path to the tested file and so on. Moreover, PHPUnit allows adding a set of test files that need to be ran in its
So, instead of filtering tests in runtime, Infection just adds correct files to custom
phpunit.xml and runs the test suite with it. That’s it.
One of the best initiatives in the PHP community is a Developer Experience improvements.
Let’s have an example of
configure command of Infection. The typical mutation framework works like:
- you run mutation tool
- it does not find a config file and fails with an error message
- you run
configurecommand (or something similar) and create this config
- you run mutation tool again
So many annoying steps.
Thanks to Symfony’s Console component, it’s possible to run commands from another command. And this is exactly what we need. Let’s look how the process looks like in Infection:
- you run
infectionfor the first time
- it does not find
- it automatically runs
infection configurecommand without interrupting the main process
- it asks you some questions and tries to guess all possible values for you
- after configuration is done, it just continues with the initial command
So you run
infection only once.
Do you remember old versions of Symfony where
phpunit.xml.dist was located in
PhpUnitPathGuesser parses your
composer.json file and automatically suggests custom path where PHPUnit’s config file is found.
PSR-4, you can configure your source folders in
SourceDirGuesser automatically parses your
composer.json file and suggests all source folders for you (as a default value or by autocompletion feature of Symfony Console component).
There are several other guessers but I believe you see the point.