How to set up a Python project with tests, code coverage, and GitHub actions

Repository structure

We start off with defining the structure of ou repository with all the needed files. I will explain in a minute what each file does and what the contents are.

pytest-ci-example/
├── .github/
│   └── workflows/
│       └── pytest-ci.yml
├── examples/
│   ├── __init__.py
│   └── util.py
├── tests/
│   └── test_util.py
├── pytest.ini
├── requirements.txt

The root folder is just the name of the repository pytest-ci-example. Then we have all the subfolder and files:

Define examples content & tests

Let us first create some functions in examples/util.py that will be the heart of our package. I use some simple math functions just for the example.

                    def sum(a, b):
    """Returns the sum of a and b."""
    return a + b

def subtract(a, b):
    """Returns the difference of a and b."""
    return a - b

def multiply(a, b):
    """Returns the product of a and b."""
    return a * b

Now we we will also add an examples/__init__.py file in our examples folder to ensure that examples is recognised as a python module and can be imported in other python files.

Now we are ready to write tests for our new functions. We will use the widely used pytest for this and can use the following

                    from examples import util
import pytest

@pytest.mark.parametrize("a, b, expected", [
    (1, 2, 3),
    (0, 0, 0),
    (-1, 1, 0),
    (1.5, 2.5, 4.0),
])
def test_sum(a, b, expected):
    """Test the sum function."""
    assert util.sum(a, b) == expected

@pytest.mark.parametrize("a, b, expected", [
    (2, 1, 1),
    (0, 0, 0),
    (1, -1, 2),
    (2.5, 1.5, 1.0),
])
def test_subtract(a, b, expected):
    """Test the subtract function."""
    assert util.subtract(a, b) == expected

A few explanations are necessary here. The most important thing is that with pytest we create a test file for every .py file we have as module. The naming always has to start with test_ so that pytest identifies the script as a test scrupt. To clearly identify tests the naming convention for test scripts for a module is test_module.py or in our case test_util.py.

Next, we import pytest and import our util module to the test file. We define tests for our sum and subtract functions and again use the test_ prefix to clearly identify which test covers which function. We then use the @pytest.mark.parametrize decorator to be able to execute the tests for different sets of parameters. These are handed over in the list together with the expected result. In the last line of every test we use the assert keyword to check if our function also has the expected result.

The last step to prepare for the test execution is to define our settings for pytest. We will do this in the pytest.ini file in root which looks like this

                    [pytest]
addopts = --cov=examples --cov-report=xml --cov-report=term
testpaths = tests
pythonpath = .

Here we define the settings for the test coverage report in addopts where the --cov argument defines the module that should be scanned for the coverage report, --cov-report defines the format of the report and the term keyword ensures that our report is also displayed in the terminal on execution. For integration of the report into GitHub or Azure DevOps often the xml version is needed, if you want to view your report directly you can also use html as format.

Execute tests locally

Now we have everything in place to execute our tests locally. The only thing left is to create a virtual environment and install the packages we defined in the requirements file. For this example we only need the pytest and pytest-cov package. That means our requirements file looks like this:

                        pytest
pytest-cov

We then create the environment and install the packages in the terminal with

                        python -m venv venv
venv/Scripts/activate
pip install -r requirements.txt

Then we are ready to run the tests from our root directory

                        pytest -v

The output should then look something like this

configfile: pytest.ini
testpaths: tests
plugins: cov-6.1.1
collected 8 items

tests/test_util.py::test_sum[1-2-3] PASSED                                [ 12%] 
tests/test_util.py::test_sum[0-0-0] PASSED                                [ 25%] 
tests/test_util.py::test_sum[-1-1-0] PASSED                               [ 37%] 
tests/test_util.py::test_sum[1.5-2.5-4.0] PASSED                          [ 50%] 
tests/test_util.py::test_subtract[2-1-1] PASSED                           [ 62%] 
tests/test_util.py::test_subtract[0-0-0] PASSED                           [ 75%] 
tests/test_util.py::test_subtract[1--1-2] PASSED                          [ 87%] 
tests/test_util.py::test_subtract[2.5-1.5-1.0] PASSED                     [100%] 

================================ tests coverage ================================
_______________ coverage: platform win32, python 3.13.1-final-0 ________________

Name                   Stmts   Miss  Cover
------------------------------------------
examples\__init__.py       0      0   100%
examples\util.py           6      1    83%
------------------------------------------
TOTAL                      6      1    83%
Coverage XML written to file coverage.xml
============================== 8 passed in 0.24s ===============================

In the upper part we see all of our tests due to the verbose setting in pytest. We cann see all combinations are executed and all tests have passed. Below that we see our coverage report. We achieve 83% test coverage which is due to the fact the we did not write a test for our multiply function.

Now you are ready to test whatever python functons you write! The next step will be to automate this process so that our tests run everytime we push to out remote repository.

Get badge for test coverage in readme file

You will often see code coverage badges on public repos to give a quick hint on your test coverage and state of your latest build. They look something like this: For that badge I used codecov, however there are also other tools like coveralls. Both of these are free to use for open-source projects. For codecov you need to head over to their homepage, login using your GitHub account and then you can create either a global secret for your GitHub account or a specific secret just for your repository. You cann then add it as a secret CODECOV_TOKEN in your repository by navigating to settings -> Secrets and variables -> Actions.

Once the secret is set you can add the following code snippet at the bottom of your workflow yml file

                    - name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v5
with:
    token: ${{ secrets.CODECOV_TOKEN }}
    slug: your-github-name/your-repo-name

If you navigate over to your codecov page to Cofniguration -> Badges & Graphs you can get the embedding link for you repository badge and add it to your readme or docs page, whatever you like best.

This is it! Now you can show off your code coverage on your next project. You can view and clone all of the code I showed here pytest-ci-example.

In a future article we will go on step further to package, build and publish our python project to make it easibly accessible for our users.