Tyler Tries to Automate Testing

Join me in this installment of “Tyler Tries” where I learn to automate testing which enforces pre-commit hooks, runs pytest and displays test passing and code coverage percentage badges in the project’s README file.

---

TL;DR

I have implemented some GitHub Actions that runs pre-commit which checks the following:

• Trailing whitespace
• Ensures files end in a new line
• Verifies proper yaml syntax
• Ensures no debug statements were left behind
• Checks that python test files are named properly
• Runs the ruff linter and ruff formatter
• Runs mypy a static type checker

Next, it runs pytest to run python tests and Coverage.py to measure code coverage.

Lastly, it runs smokeshow which is used to create a temporary website based on the code coverage report. This enables the ability to use coverage badge to show the code coverage percent badge in the README.md file.

Inspiration

Whenever I look at a python repos such as: FastAPI, Pydantic, rich

I was always curious as to how they were able to display badges in the README.md which showed information such as tests passing, code coverage, version etc. Something about these badges gave of this instant feel of a polished project. The little details matter.

I wanted to start adding some polish to my own projects so I dove a little deeper into how I can implement this myself, but where do I start?

“good artists borrow, great artists steal.” - Pablo Picasso

With most new things I learn, I personally start with a concrete example. A reference point that I use as a blueprint for what I am trying to build. This helps me overcome blank canvas syndrome.

My favorite way of finding examples is by looking at prominent people in the domain and see how they do it. For python projects I personally like to look at projects by:

• Charlie Marsh
• Anthony Sottile
• Will McGugan
• Sebastián Ramírez
• Simon Willison
• Ted Conbeer
• Daniel Reeves
• Samuel Colvin
• Pamela Fox

There are so many others and I encourage you to leave a comment of some of your favorite people in the python space!

For this project I used the full-stack-fastapi-template as my main reference.

Pre-commit

I have grown very fond of pre-commit to run my static analysis tests. It makes it really nice to run all sorts of checks and it manages the installation of any tools needed. A great example is ruff, my goto python linter and formatter. Ruff already has ruff-pre-commit which makes it as easy as adding the following lines to my .pre-commit-config.yaml file to run ruff as a pre-commit hook.

  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.4.8
    hooks:
      - id: ruff
        args:
          - --fix
      - id: ruff-format

Testing and Code Coverage

Testing can be a controversial topic for many developers… and to be honest it’s an area I need to improve on. Recently, I have actually grown to like it. There is something about running tests and seeing all green that is so so satisfying.

Pytest is the defacto standard when it comes to implementing tests in python. I am less experienced with code coverage and this was actually the first time I had used Coverage.py or any code coverage tool for that matter.

Note

I need to come back and look into how Coverage.py figures out code coverage percentage

To get the badges to display in the README.md file I copied the links from the fastapi-template replacing the urls with my repository url and I added the smokeshow.yml GitHub Action which creates the temporary website from the code coverage html report so the code coverage badge can display the percentage.

Tip

You can use pipx to run python commands without having to install the package

pipx run smokeshow generate-key

Tip

I have noticed in other people’s GitHub Action files that ${{ secrets.GITHUB_TOKEN }} is referenced. I always tried to create a secret with that name in my repository with a GitHub token value. But it would error out saying you can’t have a secret that starts with GitHub. Well, it turns out this secret is already created for you and you don’t need to create anything.

Summary

This blog post was written primarily so I can refer back to how I did this. Making it public so others can hopefully get something out of my experience. More importantly I wanted to shared how I approached doing something I have never done before.