How do you know your code gives the right answers?
... what about after you make changes?
Legacy code: "code without tests" (Michael Feathers, Working Effectively with Legacy Code)
Testing your code
when to test: always.
where to test: external test suite.
Example: tests subdirectory inside package.
Perfect is the enemy of good; a basic level of tests is better than nothing. But a rigorous test suite will save you time and potential problems in the long run.
Why test?
Testing is a core principle of scientific software; it ensures results are trustworthy.
Scientific and engineering software is used for planes, power plants, satellites, and decisionmaking. Thus, correctness of this software is pretty important.
And we all know how easy it is to have mistakes in code without realizing it...
What and how to test?
def kepler_loc(p1, p2, dt, t):
'''Use Kepler's Laws to predict location of celestial body'''
...
return p3
def test_kepler_loc():
p1 = jupiter(two_days_ago)
p2 = jupiter(yesterday)
exp = jupiter(today)
obs = kepler_loc(p1, p2, 1, 1)
if exp != obs:
raise ValueError("Jupiter is not where it should be!")
What is a test?
Tests compare expected outputs versus observed outputs for known inputs. They do not inspect the body of the function directly. In fact, the body of a function does not even have to exist for a valid test to be written.
def test_func():
exp = get_expected()
obs = func(*args, **kwargs)
assert exp == obs
Good idea: test through assertions
For exactness:
def test_kepler_loc():
p1 = jupiter(two_days_ago)
p2 = jupiter(yesterday)
exp = jupiter(today)
obs = kepler_loc(p1, p2, 1, 1)
assert exp == obs
For approximate exactness:
import numpy as np
def test_kepler_loc():
p1 = jupiter(two_days_ago)
p2 = jupiter(yesterday)
exp = jupiter(today)
obs = kepler_loc(p1, p2, 1, 1)
assert np.allclose(exp, obs)
Test using pytest
# content of test_sample.py
def inc(x):
return x + 1
def test_answer():
assert inc(3) == 5
$ pytest
pytest finds all testing modules and functions, and runs them.
Kinds of tests
interior test: precise points/values do not matter
edge test: test examines beginning or end of a range
Best practice: test all edges and at least one interior point.
Also corner cases: two or more edge cases combined.
import numpy as np
def sinc2d(x, y):
'''(Describe the function here)'''
if x == 0.0 and y == 0.0:
return 1.0
elif x == 0.0:
return np.sin(y) / y
elif y == 0.0:
return np.sin(x) / x
else:
return (np.sin(x) / x) * (np.sin(y) / y)
import numpy as np
from mod import sinc2d
def def test_internal():
exp = (2.0 / np.pi) * (-2.0 / (3.0 * np.pi))
obs = sinc2d(np.pi / 2.0, 3.0 * np.pi / 2.0)
assert np.allclose(exp, obs)
def test_edge_x():
exp = (-2.0 / (3.0 * np.pi))
obs = sinc2d(0.0, 3.0 * np.pi / 2.0)
assert np.allclose(exp, obs)
def test_edge_y():
exp = (2.0 / np.pi)
obs = sinc2d(np.pi / 2.0, 0.0)
assert np.allclose(exp, obs)
def test_corner():
exp = 1.0
obs = sinc2d(0.0, 0.0)
assert np.allclose(exp, obs)
Test generators
import numpy as np
import pytest
# contents of add.py
def add2(x, y):
return x + y
class Test(object):
@pytest.mark.parametrize('exp, x, y', [
(4, 2, 2),
(5, -5, 10),
(42, 40, 2),
(16, 3, 13),
(-128, 0, -128),
])
def test_add2(self, x, y, exp):
obs = add2(x, y)
assert np.allclose(exp, obs)
$ pytest
Types of tests
- unit test: interrogate individual functions and methods
- integration test: verify that multiple pieces of the code work together
- regression test: confirm that results match prior code results (which are assumed correct)
Test-Driven Development (TDD)
Write the tests first.
Before you write any lines of a function, first write the test for that function.
from numpy import allclose
from mod import std
def test_std1():
obs = std([0.0, 2.0])
exp = 1.0
assert np.allclose(exp, obs)
def std(vals):
# this must be cheating.
return 1.0
def test_std1():
obs = std([0.0, 2.0])
exp = 1.0
assert np.allclose(exp, obs)
def test_std2():
obs = std()
exp = 0.0
assert np.allclose(exp, obs)
def test_std3():
obs = std([0.0, 4.0])
exp = 2.0
assert np.allclose(exp, obs)
def std(vals):
# a bit better, but still not quite generic
if len(vals) == 0:
return 0.0
return vals[-1] / 2.0
def test_std1():
obs = std([0.0, 2.0])
exp = 1.0
assert np.allclose(exp, obs)
def test_std2():
obs = std()
exp = 0.0
assert np.allclose(exp, obs)
def test_std3():
obs = std([0.0, 4.0])
exp = 2.0
assert np.allclose(exp, obs)
def test_std4():
obs = std([1.0, 3.0])
exp = 1.0
assert np.allclose(exp, obs)
def test_std4():
obs = std([1.0, 1.0, 1.0])
exp = 0.0
assert np.allclose(exp, obs)
def std(vals):
# finally some math
n = len(vals)
if n == 0:
reutnr 0.0
mu = sum(vals) / n
var = 0.0
for val in vals:
var = var + (val - mu)**2
return (var / n)**0.5
Test coverage
Meaning: percentage of code for which a test exists, determined by number of line executed during tests
pytest-cov
Instructions:
-
Install
pytest-covusingpip/conda -
pytest -vv --cov=./ - Look at coverage; are you at or near 100%?
-
Get more detailed information by having it create a report:
pytest -vv --cov=./ --cov-report html
Example
# content of test_sample.py
def inc(x):
if x < 0:
return x - 1
return x + 1
def test_answer():
assert inc(3) == 4
$ pytest -vv test_sample.py --cov=./
main.py::test_answer PASSED
---------- coverage: platform darwin, python 3.5.4-final-0 -----------
Name Stmts Miss Cover
-----------------------------
main.py 6 1 83%
Coverage report
# content of test_sample.py
def inc(x):
if x < 0:
return x - 1
return x + 1
def test_answer():
assert inc(3) == 4
def test_answer_negative():
assert inc(-2) == -3
$ pytest -vv test_sample.py --cov=./
main.py::test_answer PASSED
main.py::test_answer_negative PASSED
---------- coverage: platform darwin, python 3.5.4-final-0 -----------
Name Stmts Miss Cover
-----------------------------
main.py 8 0 100%