Featured image of post How To Create An *Actual* CLI In Python
Platform Engineering

How To Create An *Actual* CLI In Python

Boost developer productivity and adoption

Why A CLI?

So you’ve successfully written your python-based tool. It does something really useful that you want the rest of your engineering team to use and worship. Or maybe it’s going to be open source and you’re hoping you’ve built the next Docker/Terraform. CLI’s are usually used in the following ways:

  • Invoked locally to aid a developer on their own machine
  • Invoked in a pipeline to execute something desirable in CI/CD

After two weeks and just two stars of your GitHub repo (both from the GitHub accounts you set up for Mom and Dad), Bob comes to your desk saying “Hey! Tell me about that project you keep spamming in the #general channel! How can I use it?”. As you’re explaining that he needs to check out the repo, build the venv then execute X, Y or Z in a Python shell he interrupts you — “Python shell? I’m a Java dev and haven’t worked with Python for years!”

So as you can see, when you build a tool in Python, you are immediately limiting it’s adoption to engineers comfortable with Python. You may take for granted the preparation required to run your tool:

  • The correct version of Python installed and on the right $PATH etc
  • A virtual env needs creating with the correct tool (Pipenv, Poetry etc…)
  • Your project needs to be in your $PYTHONPATH
  • You need to open a Python shell and import your module
1
2
3
4
5
6
7
8
9

$ git checkout ...
$ cd ...
$ curl -sSL https://install.python-poetry.org | python3 -
$ poetry install
...
$ poetry shell
$ python3
>> from X import *
>> ... (Now you can get started)

This is all time-consuming and will hurt the tool’s adoption. So you decide to convert it into a CLI.

Common “CLI” Options

When researching how to create a CLI from a Python project you will come across the following options:

All of these will allow you to invoke your Python tool using the following syntax:

1

python3 -m <my_module> -a -b --sea easy_as 123

Where you can define flags (see a, b (shorthand) and sea) and arguments (easy_as and 123).

But what will a user need to prepare in order to use your new “CLI”?..

  • The correct version of Python installed and on the right $PATH etc
  • A virtual env needs creating with the correct tool (Pipenv, Poetry etc…)
  • Your project needs to be in your $PYTHONPATH
1
2
3
4
5
6
7

$ git checkout ..
$ cd ...
$ curl -sSL https://install.python-poetry.org | python3 -
$ poetry install
...
$ poetry shell
$ python3 -m <my_module> -a -b --sea easy_as 123

Sound familiar? All you’ve achieved is removing the need to open a Python shell in order to use the tool. Technically this is a CLI as you are interacting with the command line rather than a Python shell, but you don’t need to go through these steps to run “docker build” or “terraform deploy”, so why should users when using your tool?

The True CLI Option

Now comes the punchline. Pyinstaller is your new best friend.

Pyinstaller will bundle up a python project into either a binary (unix) or .exe (Windows), including the version of Python it requires, and all it’s packages. When building it, you pass an argument pointing to the file that defines your CLI in either Typer, Argparse or Click (see above). When your bin/exe is executed, it will execute the given in a python environment containing everything it needs. Your machine doesn’t even require Python to be installed, it runs with it’s own!

So all of a sudden, Java-based Bob is happy to adopt your tool as he doesn’t need to come from a Python background to use it!

Steps to Convert Your Python CLI into an Actual CLI

Create A Python-based “CLI”

To begin you need a working python-based CLI. I’d recommend creating a file called cli.py at the root of your src directory. For a good example check out data-file-merge (only going to promote my own open-source project once!). It has the cli file exactly where I’ve described as it uses argparse.

Invoke the PyInstaller CLI

Pyinstaller can be invoked by the following:

1

$ poetry run pyinstaller src/cli.py --onefile --name <your_project_name>

This tells pyinstaller to create the single-file bundle named <your_project_name>, executing your cli.py file when invoked.

You should now see a file in ./dist/<your_project_name> at the root of your repo.

I’d recommend adding this command into a MakeFile. See data-file-merge’s MakeFile (sorry last plug I promise!), specifically the create-cli command.

Test your Actual CLI

Your CLI is invokable! To invoke simply enter the path to this bundle (in a command line!, not a Python shell or even a Python venv!) with any flags and arguments your CLI expects.

1

$ ./dist/<my_module> -a -b --sea easy_as 123

Build your Actual CLI in CI/CD

Add the CLI build step into your CI/CD. See data-file-merge’s CI/CD (whoops I lied! Last time!). Notice how I created 3 separate jobs:

  • create-cli-mac
  • create-cli-unix
  • create-cli-windows

This is because PyInstaller is not a cross-complier. If you build on mac, you get a binary that runs on mac. If you build on Windows, you get an .exe that runs on Windows. This is why I used CircleCI for my pipeline. CircleCI spins up each job in a defined container, so I spin up a mac, linux and windows container in order to create each CLI bin/.exe.

Distribute your Actual CLI

Your users need installation instructions. After building my CLI bin/.exe files in CircleCI, I store the files as pipeline artifacts. I can then add them as binaries in a GitHub release of my repo. The project wiki has installation instructions that tell a user how to install the CLI for their particular OS.

I just copied the installation instructions for Terraform. I’ll let their documentation team do the work for me.

Conclusion

And there we have it! Your Python project now generates an actual CLI as part of it’s CI/CD which anyone can use with minimal set up!

Example Repo — data-file-merge

Twitter — @Serverless_Sam

GitHub — @ServerlessSam

© 2020 - 2024 Serverless Sam