Tyler Tries Publishing a Python Package to PyPI
TL;DR
I developed a python package, harlequin_trino, and published it The Python Package Index (PyPI). This provides a Trino adapter for Harelquin a SQL IDE for your terminal.
What is a Python Package Anyway?
One of the many great things about the python community is the plethora of packages available that you can install. These packages extend the functionality of python that is not available in the standard library. This is beneficial for developers as it saves time from writing code from scratch for common tasks. Instead you can leverage the hard work of others who have solved similar problems.
The most common place to publish python packages is to PyPI which is described as:
A repository of software for the Python programming language. PyPI helps you find and install software and shared by the Python community.
The most popular way to install python packages is to use pip which is the package installer for python. Now package installation, dependencies, python versions, and virtual environments is a touchy subject for many python devs… I am not going to into that here but instead talk about my experiencing developing my first ever python package and publishing it to PyPI.
Harelquin
Harelquin is a SQL IDE for your terminal. Originally, it was developed strictly for DuckDB but the creator, Ted Conbeer, changed that by adding the feature of database adapters. There is even a nice harlequin-adapter-template to get adapter authors started with creating their own.
This really intrigued me as I am always looking for ways to contribute to open-source as I find it a great way to learn and give back to the community.
This project in particular interested me because I am a big fan of it and because of the extra thought and care Ted has put into the project to set up contributors for success:
- Contributing To Harelquin
- Creating an Adapter
- Example adapters that use the adapter-template: Postgres, MySQL
Ted here has created one of the biggest pits of success ever.
I decided on creating a Trino adapter simply because I could run it locally in a docker container and I didn’t have to sign up for an account.
After I forked the adapter template I started following the commits that were made to the other adapter projects to see what changes were made and replicating them to my project, while modifying any database specific stuff. Two things that really helped me along the way:
- Makefile that was used in the other adapters which was helpful as it gave me insight to the commands I should be running to ensure my adapter is working. At the end I was basically just going through one by one each error from pytest or mypy and making sure everything was all ✅
- GitHub Actions made it ridiculously easy to make releases and publish to PyPI. At first I didn’t understand what they did but after ChatGPT explained it to me I was off to the races. So the actual process for making a release and publishing to PyPI is completely automated for me. The only manual thing I have to do is kick off the release workflow when I want to create a new release.
Summary
Long story short my experience building my first ever python package and publishing it to PyPI was easy because of the hard work that Ted put into setting me up for success. This post is less about my experience and more about what you can do as an open source maintainer to encourage high quality contributions.