Determining Code Coverage With PHPUnit

There are lots of ways that we can judge other developers’ code. A lot of them are subjective like spacing, function names, and class names. When we do code reviews these are not helpful methods to judge the code and we instead need to use a better method. We need to use non-subjective metrics like lines of code or the complexity of the solution. However, as a proponent of test-driven development (TDD) my favorite metric is code coverage.

In this article, we’ll discuss what code coverage is, why it’s important, and run through how to determine your code coverage with PHPUnit.

What is Code Coverage?

Automated tests are a critical aspect of maintaining projects in the long term. By integrating test writing into our daily workflow using approaches like TDD we can quickly and confidently make changes to our code base. Test cases make sure that our code is doing what it should, reduces maintenance costs, keeps the quality high, and ensures we keep our code functional as it changes.

We need some kind of metric to make sure that we have enough tests in our code base. Code coverage is this metric.

Code coverage is a software testing metric that determines how much of your code is run during your test process. Code coverage is calculated as a percentage by taking the number of lines of code executed by our tests and dividing by the total number of lines of code times one hundred. The “total number of lines of code” part only includes lines that are performing operations and not declarations and white space.

As an example let’s look at this class with a basic constructor and function.

“`
class FirstName
{
protected string $value;

public function __construct(string $value)
{
$this->value = $value;
}

public function length(): int
{
return strlen($this->value);
}
}
“`

The test for this code “only” checks if the class can be initialized. This is a good first test but generally gets removed during refactoring as tests are added.

“`
class FirstNameTest extends TestCase
{
public function testCanInitialize(): void
{
$this->assertNotNull(new FirstName(“Name”));
}
}
“`

In our FirstName class, we only have 2 lines of executable code. The assignment operation in the constructor and the `strlen()` call in `length()`. All of the other lines are ignored when we calculate code coverage. The tests only call the constructor so only the assignment operation is covered. We calculate the coverage by dividing one by two times one hundred for 50% code coverage.

What’s a Good Coverage Number?

Now you may be wondering what a good code coverage number is. Unfortunately, this is one of those annoying questions that have an answer of “it depends” because there are a lot of factors going into what your team thinks is best.

The most important thing to remember as you determine your ideal number is that good coverage does not always equal good tests. It’s easy to create a ton of tests but you’ll quickly find that maintaining the tests becomes a real hassle every time you make a change.

Each team will have to determine what’s best but generally, 75-90% coverage is considered good. Less than that and it’s easy to miss bugs but more than that can make tests fragile, expensive, and difficult to debug.

I like my teams to maintain ~80% code coverage on the application as a whole. To do this each PR needs to have at least 80% code coverage to be accepted and we quickly fix any that are less than that.

Some components in our application require more than that and need at least 95% code coverage. I feel that modules like billing that are mission-critical are worth the extra effort.

PHPUnit Example

Let’s work through how you can find the code coverage percentage of your code base.

PHPUnit provides code coverage support when we install it into our application but we also need to have either Xdebug or PCOV installed on our system. This is what PHPUnit uses to determine which lines are executed as our tests are run. Xdebug is a powerful tool for debugging your code (check back later for an article on how to use it) and is hopefully already installed so it’s an easy win. I also mention PCOV because it’s faster and uses less memory which can be critical when your code base gets larger.

PHPUnit provides a large number of options for outputting the code coverage. Some like the `–coverage-text` or `–coverage-html` options are human consumable reports of our code coverage and others like `–coverage-clover` or `–coverage-xml` are specifically made for other programs to consume them.

“`
./vendor/bin/phpunit –help | grep coverage
–coverage-clover Generate code coverage report in Clover XML format
–coverage-cobertura Generate code coverage report in Cobertura XML format
–coverage-crap4j Generate code coverage report in Crap4J XML format
–coverage-html

The `–coverage-text` option is a great starting point especially when our code base is small. It’s nice because it gives us overall code coverage numbers as well as a breakdown by file. You’ll find that this is less and less helpful as the code base grows.

Let’s check our FirstName and FirstNameTest examples from before using the `–coverage-text` switch. If you’re using XDebug 3 you’ll need to make sure you specify the mode in the command line.

“`
XDEBUG_MODE=coverage ./vendor/bin/phpunit –coverage-text
PHPUnit 9.5.26 by Sebastian Bergmann and contributors.

Runtime: PHP 8.1.6 with Xdebug 3.1.6
Configuration: /Users/scottkeck-warren/tmp/composer-test/phpunit.xml

. 1 / 1 (100%)

Time: 00:00.058, Memory: 8.00 MB

OK (1 test, 1 assertion)

Code Coverage Report:
2022-12-09 01:02:02

Summary:
Classes: 0.00% (0/1)
Methods: 50.00% (1/2)
Lines: 50.00% (1/2)

ScottValueObjects\FirstName
Methods: 50.00% ( 1/ 2) Lines: 50.00% ( 1/ 2)
“`

Notice that this verifies our manual calculation from before to show us that our code base has 50% code coverage. It also shows which files have how much coverage.

If we add a LastName class with no tests we can see how that will affect our percentage as well.

“`
Summary:
Classes: 0.00% (0/2)
Methods: 25.00% (1/4)
Lines: 25.00% (1/4)

ScottValueObjects\FirstName
Methods: 50.00% ( 1/ 2) Lines: 50.00% ( 1/ 2)
ScottValueObjects\LastName
Methods: 0.00% ( 0/ 2) Lines: 0.00% ( 0/ 2)
“`

As the project grows it will be helpful to see the coverage in a more user-friendly way. This is where the `–coverage-html` switch comes in handy. This generates an HTML report of our code base. All we have to do is specify the directory we want saved to.

“`
XDEBUG_MODE=coverage ./vendor/bin/phpunit –coverage-html=coverage

Generating code coverage report in HTML format … done [00:00.019]
“`

Opening the “index.html” file in the directory will display the report. It starts on a page that allows us to browse our code coverage based on file and directory. This helps find whole modules we can improve.

Clicking the “(Dashboard)” link in the top navigation gives us a helpful overview in chart form to find specific problem files.

The “Complexity” chart is an excellent option for finding files that need more coverage. The chart shows how much complexity each one of the classes has and what is the test coverage percentage for that file. Files that have high complexity but low code coverage are prime candidates for more tests.

Other reports can help you narrow down your focus if you need to increase your code coverage.

GitHub Integration

One of the best pieces of advice I can give you in this article is to make code coverage part of your continuous integration (CI) pipeline. Your continuous integration pipeline can automatically fail if you don’t reach a high enough percentage of coverage.

In a future article, we’ll discuss how to use a SaaS solution to do this.

What you need to know

1. Code coverage is an indicator of how tested our code is
2. Expressed as a percentage
3. Each team needs to pick its threshold
4. Use tools to integrate it into your CI pipeline and pull request process to maximize its usefulness

The post Determining Code Coverage With PHPUnit appeared first on php[architect].