What's new in Infection 0.24.0

Jul 27, 2021

Release: https://github.com/infection/infection/releases/tag/0.24.0

BC Breaks

• Spread mutator has been renamed to SpreadOneItem

Allow using MSI Badge for multiple branches

By configuring logs.badge.branch with a /-delimited regular expression, we can now collect reports about a number of branches instead of just one.

{
    "logs": {
        "badge": {
            "branch": "/^release-.*$/"
        }
    }
}

If you provide a value that is not a regular expression starting and ending with /, a direct match will be performed:

{
    "logs": {
        "badge": {
            "branch": "main"
        }
    }
}

Major performance improvement for the projects with slow test suites, using --only-covering-test-cases option

Infection 0.24.0 dramatically improves performance when executed against functional (read - slow) tests.

One of the random real example:

- Time: 4m 18s. Memory: 0.05GB
+ Time: 1m 19s. Memory: 0.06GB

Problem

As you probably know, we always said that Infection runs only those tests that cover mutated line, not the all tests from an application.

It is partially true, but not exactly. PHPUnit’s XML file allows us to only filter original set of tests by <directory> or <file> .

And this is what we did starting from 2017.

PHPunit has no ability to configure which test cases to run in phpunit.xml, this is a not merged feature - https://github.com/sebastianbergmann/phpunit/pull/4449

Example of a phpunit.xml file generated by Infection for one of the Mutant:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit>
  <testsuites>
    <testsuite name="Infection testsuite with filtered tests">
      <file>/infection/PestTestFramework/tests/CalculatorTest.php</file>
    </testsuite>
  </testsuites>
    
  <filter>
    <whitelist>
      <directory>/infection/PestTestFramework/src/</directory>
    </whitelist>
  </filter>
</phpunit>

Notice in the example above, that we tell PHPUnit to run tests only from CalculatorPhpUnitTest.php file. But this file can contain 1 test that covers mutated line, and hundreds of other tests that still will be executed for a Mutant.

It wasn’t always noticeable because the majority of us used Infection for unit tests, which are very fast.

However, when you use Infection in the projects with thousands of functional or integration tests, this becomes a bottleneck.

Example: Imagine there is a file ProductApiTest.php that covers all the API endpoints with functional tests (Symfony WebTestCase tests).

• In this file, there is only 1 test case that covers mutated line. This test case takes 1s
• In this file, there are additionally other 100 test cases that together take 15s

So, with the previous Infection version (0.23.0), instead of running 1 test case for 1s, we ran all the tests from ProductApiTest.php for 1+15=16s.

Solution

Starting from Infection 0.24.0, for the example explained above, we can run only 1 test case for 1s, saving 15 from 16 seconds.

This feature can be enabled by --only-covering-test-cases option.

Technical details

Since PHPUnit doesn’t allow us to provide test cases on phpunit.xml file level, we can only do it on a command line level, using --filter option

• Command line on master:

  'phpunit' '--configuration' '/path-to/phpunit.xml'

• Command line on this PR:

  'phpunit' '--configuration' '/path-to/phpunit.xml' '--filter' '/App\\Test::test_case1|App\\Test::test_case2/'

Here is an example of the logs for the escaped mutant:

0.23.0 (previous old Infection version):

3) /srv/api/src/DataPersister/TableViewDataPersister.php:32    [M] InstanceOf_

--- Original
+++ New
@@ @@
     }
     public function persist($data, array $context = [])
     {
-        if ($data instanceof TableView && ($context['collection_operation_name'] ?? null) === 'post' && $this->security->getUser() instanceof Employee) {
+        if ($data instanceof TableView && ($context['collection_operation_name'] ?? null) === 'post' && true) {
             /** @var Employee $employee */
             $employee = $this->security->getUser();
             $data->setCreatedBy($employee);

$ '/srv/api/vendor/phpunit/phpunit/phpunit' '--configuration' '/tmp/infection/phpunit.xml'
  Test cache cleared
  PHPUnit 9.5.6 by Sebastian Bergmann and contributors.
  
  Random Seed:   1625689971
  
  Testing 
  .............................................................   61 / 1166 (  5%)
  .............................................................  122 / 1166 ( 10%)
  .............................................................  183 / 1166 ( 15%)
  .............................................................  244 / 1166 ( 20%)
  .............................................................  305 / 1166 ( 26%)
  .............................................................  366 / 1166 ( 31%)
  .............................................................  427 / 1166 ( 36%)
  .............................................................  488 / 1166 ( 41%)
  .............................................................  549 / 1166 ( 47%)
  .............................................................  610 / 1166 ( 52%)
  .............................................................  671 / 1166 ( 57%)
  ..............................

0.24.0 (new Infection version):

3) /srv/api/src/DataPersister/TableViewDataPersister.php:32    [M] InstanceOf_

--- Original
+++ New
@@ @@
     }
     public function persist($data, array $context = [])
     {
-        if ($data instanceof TableView && ($context['collection_operation_name'] ?? null) === 'post' && $this->security->getUser() instanceof Employee) {
+        if ($data instanceof TableView && ($context['collection_operation_name'] ?? null) === 'post' && true) {
             /** @var Employee $employee */
             $employee = $this->security->getUser();
             $data->setCreatedBy($employee);

$ '/srv/api/vendor/phpunit/phpunit/phpunit' '--configuration' '/tmp/infection/phpunit.xml' '--filter' '/App\\Tests\\[...very long filter]/'
  Test cache cleared
  PHPUnit 9.5.6 by Sebastian Bergmann and contributors.
  
  Random Seed:   1625689648
  
  Testing 
  ...............................................................  63 / 192 ( 32%)
  ............................................................... 126 / 192 ( 65%)
  ............................................................... 189 / 192 ( 98%)
  ...                                                             192 / 192 (100%)

So it is 192 executed tests (in 0.24.0) instead of 1166 (in 0.23.0).

We recommend trying this new --only-covering-test-cases option and see how it impacts your test suites.

For the fast test suites (with mainly unit tests) it can negatively impact performance, decreasing the speed of Infection. You can read the details in this PR

New Mutators

SpreadAssignment mutator

Removes a spread operator in an array expression and turns it into an assignment. For example:

$x = [...$collection];

Will be mutated to:

$x = $collection;

SpreadRemoval mutator

Removes a spread operator in an array expression. For example:

$x = [...$collection, 4, 5];

Will be mutated to:

$x = [$collection, 4, 5];

Enjoying Infection? Consider supporting us on GitHub Sponsors ♥️

https://github.com/sponsors/infection

Star