Code coverage is an important aspect of good engineering practice. Without it, you are flying a bit blind writing tests.
What is Coverage?
coverage is a mature testing utility originally written by Gareth Rees and now maintained by Ned Batchelder and others. It shows what lines are invoked during project execution, typically while running tests. But coverage provides utility beyond showing what lines were executed during tests. Coverage enhances your development process and encourages good testing practices. If you have never used coverage, we encourage you to read the coverage "Quick Start" reference.
What Coverage Does Not Do
Coverage does not prove tests are correct. That is the job of the developer and peer review. Coverage does not prove your code works correctly. Again, that is the job of the designer and developers and their tests. Given these truths, why use coverage at all?
Reasons for Coverage
1. Coverage Proves Tests Exercise Code
Coverage shows that code was executed, and the most immediate benefit is exposure of bad code by that execution. As more code is executed in tests, more typos, cut-n-paste, and similar errors are surfaced.
2. Coverage Reveals Test Omissions
Coverage also reveals un-executed application code. Consider an updated if-then-block. Quite often the common branch is well tested but the exception is not. In my experience the unexecuted branch may not be as well thought out as the common branch and may contain a bug. Coverage highlights these omissions and allows you to improve robustness by adding tests proving "unusual" situations are handled properly before they appear to users.
3. Coverage Enhances Refactoring
An application with good coverage is much easier to refactor than an application with poor coverage. The approach taken to refactor code is different when coverage is low. You must be more careful with assumptions and need (often inconsistent) manual testing to prove the code produces the same result as before. Coverage gives you a good idea of how to approach refactoring a system.
4. Coverage Educates Developers
Beyond proving correctness, well crafted tests demonstrate proper API usage. Tests show how to use the code and provide valuable insight into how things work. Increasing test coverage expands internal project documentation, providing more leverage for developers.
5. Increase Coverage, Improve Code Correctness
A 64% coverage does not mean 64% of your code works correctly. It just means 64% of your code has been executed during the test suite. Are all those tests correct? Unlikely, as we all have written flawed tests during development. Yet there is a very good chance that a large percentage of those tests are correct. When test coverage increases from 64% to 70% you almost certainly have a more robust and reliable system. Increasing coverage with carefully constructed tests is a worthy goal.
Coverage Recommendations
The coverage utility is available from PyPi. You invoke coverage in a command line interface and interpret the results locally. Alternatively you can sign up for a coverage web service. We have had success with codecov.io integrating with our github repositories and providing automated coverage reports.
Cover all your code! Cheers!