Useful Python Tools and Libraries

Page content

Key tools and libraries for the Python programming language.

Using Python Command-Line Applications

Use pipx to install Python command-line tools.

The pipx utility uses the pip and virtual environment facilities that are included with Python itself.

Online Development Environments

These services enable you to work on Python projects in a cloud-based environment:

To quickly share pieces of code, use Pastebin.

Working on Python Projects

Use pipenv for Virtual Environments

Install pipenv to manage your Python projects. It ensures that each of your Python projects use a separate set of packages, and provides other features to help you maintain your work, such as checking the code for style and warning you about security issues in libraries.

The pipenv tool uses the pip and virtual environment facilities that are included with Python itself, so it is compatible with other Python utilities.

The Python Guide tutorial shows you how to work with pipenv.

Poetry may replace pipenv, but it is not yet fully supported by code editors and third-party tools.

Working with Multiple Versions of Python

If you need to develop with several different versions of Python, use pyenv. This tool enables you to install multiple versions of Python on the same system, and switch between them.

Managing Projects

Code Quality

If you install the Python extension for Visual Studio Code, it will offer to use Pylint and autopep8 to check and format your code. Use Black instead of autopep8 for new projects.

Tools for Testing Python Code

  • Pytest for testing
  • Tox runs sets of tests in multiple Python environments
  • Coverage for measuring the test coverage of code
  • Faker - Generates fake data
  • mypy - Static type checking, using Python type annotations
  • Bandit to check your code for common security issues
  • Safety to check your project dependencies for known security vulnerabilities

Developing Command-line Applications

Use the Click framework to build command-line tools with Python.

Python Fire enables you to add a command-line interface to existing code.

Building Web Applications

For simple Websites and services, use the Flask framework. Flask itself provides the basic package of features that you need for a Web application, and the framework has a wide range of extensions to add more capabilities.

Use either Django or Pyramid for larger projects. Django provides a set of custom tools and libraries that closely integrate together. The main Django framework has features for content Websites, whilst Django REST Framework is specifically for building APIs. Pyramid offers a modular framework for integrating third-party Python libraries together into custom Web applications.

All of these Web frameworks follow the WSGI standard for synchronous Python Web applications. This enables you to choose between many options for hosting your Python projects. In each case, a WSGI server such as Gunicorn runs your code on the host system.

Several commercial services offer fully managed systems, so that you can deploy applications without any setting up or maintaining any servers yourself. These services include Google App Engine, Heroku and Python Anywhere.

If you need more control over the infrastructure that your application uses, consider using containers and a Kubernetes service, rather than setting up a cluster of Web servers. Kubernetes services include Red Hat OpenShift, DigitalOcean Kubernetes and Google Kubernetes Engine.

Full Stack Python provides a comprehensive guide to building Web applications with Python.

Whichever Web framework you use, add the extension to implement Web security features.

Asynchronous Web Applications

Treat Python asynchronous Web frameworks as experimental technology. The standards and libraries are still being developed for asynchronous Python.

Starlette, Sanic and Quart are probably the most well-known asynchronous frameworks. Quart is specifically designed to use the same API as Flask, whilst Starlette and Sanic are new designs.

Both Starlette and Quart implement the ASGI standard. The Uvicorn server implements ASGI, so that it can host applications that are built with any asynchronous framework that follows ASGI.

The aiohttp library provides an asynchronous HTTP client and server, but it does not currently support ASGI.

Developing Web Clients

Use the requests library for your Web client software, such as downloading files or working with APIs. The HTTP support in the Python standard library is for low-level code.

If you need to get information from Websites that do not provide an API, use Scrapy. You can then extract content from the pages with the Beautiful Soup library.

Working with SQL Databases

The Python standard library includes a version of SQLite. This means that you can use SQL databases in any Python project without installing a separate database product. Avoid underestimating SQLite: it is a robust and efficient database that will handle gigabytes of data.

To access other types of SQL databases such as PostgreSQL, MySQL, or Oracle, install the Python driver that the vendor of the database recommends. Each Python driver will require a particular client library. For example, the recommended Python driver for PostgreSQL is psycopg, which requires a copy of the libpq library.

Connecting to Microsoft SQL Server: Microsoft recommend that you use the ODBC adapter for SQL Server.

Whichever brand of SQL database you work with, use a database toolkit, rather than writing data access and schema management code yourself.

SQLAlchemy is the standard Python library for database programming. You may use the declarative portion of SQLAlchemy like a standard ORM, but it has many more capabilities.

If your project has limited or specific needs, consider alternatives to SQLAlchemy. Peewee offers a lightweight alternative to SQLAlchemy for accessing the most popular brands of Open Source database. For Django projects, use the Object-Relational Mapper (ORM) that is provided with Django, because this is integrated with the other features of the framework.

Building Graphical Desktop Applications

Consider using wxPython to build graphical interfaces for your applications. The Tk interface toolkit that is supplied with the Python standard library is rather basic and dated. If you have advanced needs, you may prefer QT for Python, which enables you to make use of the QT libraries.

If you are building 2D games, try pygame. This provides a simple toolkit for games and multimedia applications.

Microsoft Windows Integration

Python includes support for some features that are unique to Microsoft Windows, but not all of them. To use Python with other features of Windows, such as COM and the Registry, install the win32 Extensions.

Packaging Applications

Use PyInstaller to create applications that you can give to other people. PyInstaller creates stand-alone executables that include Python itself, your code, and any other dependencies. Follow the steps described in the tutorial to add PyInstaller to your project.

If you only need to deploy your application to systems that you know will have a suitable version of Python, use shiv. The shiv utility packages your application and dependencies into a ZIP file that Python interpreters will run.

To run your Python server applications on Kubernetes, package them as containers.

Other Useful Libraries

  • Jinja2 - Text templating, for generating HTML and other formats
  • Matplotlib - Plotting 2D graphs and charts
  • Pendulum - Date and time parsing
  • Pillow - Image processing
  • python-dotenv - Load environment variables from files
  • pytz - Olson timezone database
  • PyYAML - YAML support for Python
  • Reportlab - PDF generation

Learning Resources

This article lists useful learning resources for Python.