Project Setup with Python

If possible use a project tool to set up and develop your Python projects. It will implement these concepts for you. If you decide not to use a project tool, set up your projects to follow the best practices in this article.

Use a pyproject.toml File #

Create a pyproject.toml file in the root directory of each Python project. Use this file as the central place to store configuration information about the project and the tools that it uses. For example, you list the dependencies of your project in the pyproject.toml file.

Python project tools like uv, PDM and Hatch automatically create and use a pyproject.toml file.

The pyOpenSci project documentation on pyproject.toml provides an introduction to the file format. The various features of pyproject.toml files are defined these PEPs: PEP 517, PEP 518, PEP 621 and PEP 660.

Libraries: Create a Directory Structure That Uses the src Layout #

Python itself does not require a specific directory structure for your projects. The Python packaging documentation describes two popular directory structures: the src layout and the flat layout. The pyOpenSci project documentation on directory structures explains the practical differences between the two.

Use the src layout for a project that creates Python wheel packages. This requires you to use editable installs of the packages in your project. PDM, uv and Hatch support editable installs.

By default, uv will create a project with the flat layout. Use the –lib flag to create a project with the src layout.

Use Virtual Environments for Development #

The virtual environments feature enables you to define one or more separate sets of packages for each Python project, and switch between them. This ensures that a set of packages that you use for a specific purpose do not conflict with any other Python packages on the system. Always use Python virtual environments for your projects.

Several tools automate virtual environments. The mise version manager includes support for virtual environments. The pyenv version manager supports virtual environments with the virtualenv plugin. If you use a tool like uv, PDM or Hatch to develop your projects, these also manage Python virtual environments for you.

You can set up and use virtual environments with venv, which is part of the Python standard library. This is a manual process.

Use Requirements Files to Install Packages Into Environments #

Avoid using pip commands to install individual packages into virtual environments. If you use uv, PDM or Hatch to develop your project, they can manage the contents of virtual environments for development and testing.

For other cases, use requirements files. A requirements file can specify the exact version and hash for each required package.

You run a tool to read the dependencies in the pyproject.toml file and generate a requirements file that lists the specific packages that are needed to provide those dependencies for the Python version and operating system. PDM, pip-tools and uv include features to create requirements files.

You can then use pip-sync or the sync feature of uv to make the packages in a target virtual environment match the list in the requirements file. This process ensures that any extra packages are removed from the virtual environment.

You can also run pip install with a requirements file. This only attempts to install the specified packages. For example, these commands install the packages that are specified by the file requirements-macos-dev.txt into the virtual environment .venv-dev:

source ./.venv-dev/bin/activate
python3 -m pip install --require-virtualenv -r requirements-macos-dev.txt

pip-compile: Use the Correct Virtual Environment #

If you do not already have a tool that can create requirements files, you can use the pip-compile utility that is provided by pip-tools.

To ensure that it calculates the correct requirements for your application, the pip-compile tool must be run in a virtual environment that includes your application package. This means that you cannot use pipx to install pip-compile.