Code Coverage – Stages of Life for Teams

Code Coverage is a conversation you’ll have on most development teams today with a wide range of opinions. Some people hate code coverage, others see it as absolutely mandatory and teams love to argue over what coverage numbers to settle on. We seem to have a culture of setting the code coverage standard at the beginning of projects and sticking to them. I would argue that they need to be reviewed as the project and the team evolves.

Is there one set code coverage level requirement that works for any team? 90% is usually a good number if you are on a project with no deadline. This works for projects that can strive to be perfect like many open source projects. This doesn’t always work when you are dealing with projects for clients that have deadlines though.

When a project first starts there can be an issue with some developers that just don’t write tests on your team or just don’t have the skill. If that’s the case I would suggest a high code coverage number just to get people writing unit tests. That’s the first step to unit testing, start writing! This will of course need to be followed up with senior level developers helping get everyone up to speed on the “proper” way to unit test.

Eventually, projects with high code coverage number requirements may start getting tests that are simply there to hit the code coverage number. These tests often don’t test anything of any value and are simply there to hit some random use case that doesn’t necessarily to need to be tested. These clutter the project and can often take more time to write than they are worth to test. This is the point in the project where lowering the code coverage number could be beneficial.

Lowering the number once everyone is writing unit tests can have benefits. The first and most important one is that developers can now spend their valuable time (especially with deadlines) writing actual good unit tests as opposed to tests that handle random edge cases. The quality of the tests in general tend to go up with a lower number (as long as you still have code reviews). Once again, keep in mind that this is for projects with deadlines. Not open source projects with no deadlines and endless time to write perfect tests. Another benefit is less clutter. You can get rid of a large number of “pointless” tests.

If the team’s project gets past the initial deadline and goes into maintenance mode the team can always raise the coverage percentage back up to a high standard. At this point in the project there is usually a distinct, agreed upon style to write the tests in. It’s usually easier to write good tests and there is usually just more time to write better code. Team’s just need to make sure there they are now writing tests that have quality, and don’t just satisfy quantity.

At the end of the day you have to know your team’s situation. Their are a ton of variables to account for including skill level, time and training. Setting the coverage to 90% or 100% isn’t the right call for every situation.