How to upload a Python Library to PyPI?
How to upload a Python Library to PyPI?
Quick PyPI Overview
If you have been exposed to python in any meaningful way, you may have heard of PyPI, which is short for the Python Package Index. This is a repository for the python community, which allows software authors to upload Python libraries and distribute their software. Aadditionally, PyPI can be used to find and install existing software. PyPI is generally accessed through the use of python’s package manager (pip).
Proper Package Structure
Before uploading the Python module, we need to understand how to properly structure and build a package. This will require us to first create and name our package, which should be located in our project directory. Here Is an example of the structure:
|–top level directory
|———-our package directory:
|———__init__.py # this file is needed in a package
|———our_module1.py
|———our_module2.py
This is the basic structure of a conventional package. We must have the “__init__.py” file included (you do not have a conventional package without one!). The “__init__.py” file gives us control over what functions and classes are exposed at the top-level allowing access to key functionalities of our software, and reducing the need for a user to dig for them as they are immediately accessible. It allows us to run necessary initializations (think of this like putting batteries in the remote before you can actually use it).
Please keep in mind that python package structure is constantly changing. The newest standard and most common way to set up a package is to use a “src” directory first, as this more accurately represents the users’ machine when they download and install the package as it is in their site-packages directory. On your machine, you will build everything in your top directory, but on the users’ machine, the package will live in their site-packages folder. Imagine you had some dependencies that lived in your top development directory, all your tests would be working correctly, but when the user downloads your package, those missing dependencies break their instance. Using a “src” folder ensures you do not make this mistake. An example of the “src”:
|–top level directory
|———-src:
|———-our package directory:
|———__init__.py # this file is needed in a package
|———our_module1.py
|———our_module2.py
Namespace Packages
Namespace packages are a bit different since they are comprised of sub packages that can be logically grouped together yet work separately. For example, you may have a namespace package called “wrangler”. This namespace package has no “__init__.py” file but has 3 sub-packages that all have functionality for data wrangling (these subpackages do have an “__init__”).
A user may only need tools from sub-package3, so they can simply pip install subpackage3. Each sub-package must be uploaded to PyPI separately. To be clear, a namespace package is for organization during development and really has no influence for the end-user. Below is an example:
|–top level directory
|———-wrangler:
|———-wrangler1:
|———__init__.py # this file is needed in a package
|———our_module1.py
|———our_module2.py
|———-wrangler2:
|———__init__.py # this file is needed in a package
|———our_module1.py
|———our_module2.py
Simple Requirements
First, we must create an account by visiting PyPi (this is required). You will receive a verification email to complete your account activation. Once you have an account, you can begin to upload your packages or modules to PyPI. The requirements to upload are:
- Utilize a unique name for your package or module (this should be done during development).
- Define the license type you will use for the package so users understand the legalities around using and distributing your software. This should be in a text file named “LICENSE” in your project directory.
- A README.md: This is a markdown file and should have the project title and a brief description of its purpose and functionality. You may choose to use a “README.rst” which is more complex but better suited for uploading to PyPI. An “.rst” file has more functionalities for styling so you can provide better documentation.
Package Configurations
The “pyproject.toml” file is the new standard for setting your package configurations, meta data (project version, software author, and software description), and dependencies (pandas, matplotlib, pywinauto). Previously we would use the setup.py file or the setup.cfg, both of which are now being deprecated, however setup.py may still be used as it runs a script, which can be useful for complex projects that need some initializations.
The “.toml” file removes the burden of manual package installation and instead allows pip to handle everything for the end-user. Having this task automated allows developers to focus more on writing their code versus setting up their environment. Lastly the “.toml” file allows for exclusions to be defined. The “.toml” file can be quite complex and does require a bit setup, however all documentation and information can be found here:
Build and Publish to PyPI
To publish a package to PyPI we must build it into what’s called a wheel file. This file will contain our ready to install package, which will look something like “my_cool_package.whl”.
Wheel files are binary in nature, already come compiled, and thus are extremely fast (which is always a good thing for developers). Because of the whl format, we increase the likelihood of a successful installation for the end-user. We can use the build module to perform this conversion by running pip install build then we simply run python -m build.
Once these src and wheel files have been built, we are finally ready to upload a Python library to PyPI using twine.
- We must pip install twine and then use twine to check the build by running; twine check dist/* (useful for finding illegal configurations).
- Next, we upload by running: twine upload dist/*,
- Enter your username and password and wait for the process to complete. That’s it, your package is now accessible on PyPI and can installed via pip install your_newly_uploaded_packag.
