↓Skip to main content

Evolving Projects By Using Shared Copier Templates

·13 mins
Table of Contents

Copier is a command-line tool and a Python library that enables you to automate the maintenance of large numbers of software projects, so that they evolve together over time. The documentation for Copier describes it as a code lifecycle management tool.

Copier uses the idea of generating the files in projects from templates, which is implemented by tools like cookiecutter, but adds these capabilities:

  • You can apply multiple Copier templates to a software project.
  • You can reapply new versions of a Copier template to projects that have already used that template. Copier updates the files in the project to match those in the newer version of the template.
  • Each update can run migrations or other defined tasks.

This means that you can systematically evolve your projects by defining specific capabilities or configurations for tools as Copier templates that projects can share, then updating the shared templates as needed and applying the new versions of the template to all of the relevant projects.

For example, you could define a Copier template that provides configurations for quality tools like Ruff and apply new versions of that template to all of your projects, so that the configurations for every project are synchronised.

How It Works #

A Copier template is a Git repository that has a configuration file, along with template files and directories. Each Git repository must only contain one Copier template, because Copier uses Git tags for version information.

Copier uses Jinja for templating the files and directories, with extensions. The configuration file includes definitions for the questions that Copier will ask a user when they apply a template to a project. The responses to these questions become variables that the templates can use. You can also use the external data option to set variables for the Jinja templating.

You can use any extension that is available for Jinja, and include custom extensions in specific Copier templates. Consider using copier-pydantic to integate Pydantic models into Copier templating.

Once a template is applied for the first time, Copier creates an answers file in the project. This answers file enables Copier to manage updates to new versions of the template. Here is an example of a Copier answers file:

_commit: v0.7.0
_src_path: git@github.com:my-username/copier-mycompany-myteam-aws-lambda-py.git
lambda_name: example-lambda
use_ruff: true

The answers file stores the address of the template and the version of the template that you used, along with your most recent responses to the questions in the template.

You can apply Copier templates to the repositories that hold Copier templates. This includes applying a template to the repository that holds itself. This means that you can use Copier to manage the projects for Copier templates in the same way that you manage other software projects.

Using a Copier Template #

You can either run the Copier command-line tool, or use the library with Python. The examples here use the command-line. See the section on using Copier for details about how to run the command-line tool.

The first time that you run the Copier command-line tool to add a template to a project, you specify the address of the Git repository that contains the template, like this:

copier copy git@github.com:my-username/copier-mycompany-myteam-aws-lambda-py.git my-project

Copier then prompts you for answers to the questions that are defined in the template. Once you have responded to all of the questions it will create the files and directories from the template in the target directory, including the answers file.

You can also provide responses on the command-line or in a data file.

You can update a project at any time. Run Copier, specifying the answers file in the project that tracks the relevant template. Copier will read the address of the template repository from the answers file, and begin the update process. It fetches either the latest version of the template, or the version of the template that you specify:

# Apply the latest version of the template that is specified in the answers file
copier update -a .copier/.copier-answers-mycompany-myteam-aws-lambda-py.yaml

# Apply version v1.2.3 of the template that is specified in the answers file
copier update -a .copier/.copier-answers-mycompany-myteam-aws-lambda-py.yaml -r v1.2.3

Copier then reads the questions that are defined for the new version of the template. By default, it prompts the user for responses, but we can provide responses by other means in order to automate updates. If a question existed in the previous version, it sets the previous response from the answers file as the default response.

Once it has responses to all of the questions that are defined in the template it performs an update. The update can also include running migrations or other defined tasks.

Copier only updates a project if the version of the template is higher than the version that has already been applied to that project. It does not support downgrading to previous versions of a template.

You can safely use multiple Copier templates on the same project. Templates can use the responses that users have provided to other templates, as explained in a later section.

These features mean that you can maintain large numbers of projects that are composed from sets of Copier templates. The developers that work on these projects can update them as needed, or you can use automation to run template updates and commit the changes.

If you use Renovate it can update projects with the latest versions of Copier templates, in the same way that it updates dependencies.

Versioning Your Copier Templates #

Copier reads the Git tags on a template repository to determine the available versions. By default, it will use the current release found in the Git version tags for the template, excluding pre-releases. Migrations also rely on the version of the template to determine whether to run.

Copier expects the version tags to follow the format of a Python version specifier.

These features mean that you should use Semantic Versioning for template repositories. To identify the tags that are for versions, give each version tag a prefix of v. Set up an automated release tool to handle versioning for you, with Git tags that have a v prefix, e.g. v1.2.3.

If you use Semantic Versioning then it also allows you to indicate whether a new version of a template is a minor release, or a major new version that could introduce breaking changes. Humans and automations can then decide when and how to carry out updates. The section on automating updates explains how Copier upgrades projects.

For development and testing Copier templates, you can use the option to fetch a Copier template from any Git ref, such as a branch name or a tag.

Automating Updates #

Use tasks and migrations to trigger actions that will not be carried out by Copier itself.

You can call the command-line tool with any scripts or tools that you wish, or write automations with Python that use the package for Copier. You may decide to use different approaches for different templates, or different classes of update. For example, you might choose a one approach for minor versions that should not require any changes to answers and another for the major version changes that could introduce breaking changes.

Consider using Renovate to manage updates for Copier templates that do not require any responses.

Each time that Copier updates a project to a different version of a template, it reads the configuration for the new version of the template. It then uses any external data to set variables and reads the questions that are defined for that version of the template. If Copier has responses to all of the questions in the template, it does not prompt the user.

Copier accepts responses from the command-line and from YAML data files. If you think that it is safe to use the existing responses for an update, you can also use the skip answered option to make Copier automatically use those responses from the answers file.

Running Copier #

To run Copier on a development system, use a Python tool like pipx or uv. If you are using uv, call Copier with uvx. Alternatively, you can create your own container image that includes Copier along with Python, Git and any other required dependencies.

This command uses pipx to run copier copy:

pipx run copier==9.17.1 copy git@github.com:my-username/copier-mycompany-myteam-aws-lambda-py.git my-project

This command uses uv to run copier copy:

uvx copier==9.17.1 copy git@github.com:my-username/copier-mycompany-myteam-aws-lambda-py.git my-project

Both pipx run and uvx download Copier to a cache, so that you do not need to manage a Python virtual environment.

If you use extra Jinja filters in a Copier template, you will need include these packages into the virtual environment that pipx or uvx maintains for Copier.

By default, Copier disables features that allow arbitrary code execution, including tasks and migrations. You must use the –trust flag to enable these to run.

Use the –pretend flag to carry out a dry-run, where Copier lists the changes that it would make but does not apply them.

Copier supports fetching templates from remote Git repositories over HTTPS and SSH, as well as from repositories on the local system. Use SSH authentication to access Copier templates in private Git repositories. If you do not use an SSH agent, you can pass credentials.

Using Existing Responses for Updates #

If you think that it is safe to use the existing responses for an update, you can use the skip answered option to make Copier automatically use the responses from the answers file:

copier update -A -a .copier/.copier-answers-mycompany-myteam-aws-lambda-py.yaml

This option does not handle responses to new questions, because the answers file will not have a response for them.

Updating Specific Files #

Use –exclude to update a single file in a project from a template:

copier copy --exclude '*' --exclude '!file-i-want' ./template ./destination

For example:

copier copy -a .copier/.copier-answers-mycompany-myteam-aws-lambda-py.yaml --exclude '*' --exclude '!.pre-commit-config.yaml' git@github.com:my-username/copier-mycompany-myteam-aws-lambda-py.git .

Creating a Copier Template #

This process does not require the Copier tool.

A Copier template is a Git repository with a structure like this:

|- template/
|   |
|   |- [{_copier_conf.answers_file}].jinja
|   |
|   |- <templated files and directories...>
|
|- .gitignore
|- LICENSE.md
|- README.md
|- copier.yaml

To create a Copier template:

  1. First, create a Git repository to hold the template. By convention, the name of this template repository should start with copier-. To simplify management, set the name of the repository to match the identifier of the answers file. For example: copier-mycompany-myteam-aws-lambda-py when the answers file will be identified as mycompany-myteam-aws-lambda-py.
  2. Create a copier.yaml configuration file in the root of the template repository. See below for an example configuration file. To avoid conflicts with other Copier templates that projects may use, the _answers_file must specify a name that is unique to each template.
  3. Create a directory called template/ in the repository to hold the files and directories that make up the template.
  4. Create a template answers file with the name [{_copier_conf.answers_file}].jinja in the template/ directory. Use this exact name. The file name in the template must include the delimiters, because the file in each project is completely managed by Copier. See below for an example answers file.
  5. Optional: Set up automated releases for the template repository to ensure that there are Git tags for versions.
  6. Optional: Add metadata to the project for the template repository. For example, if it is hosted on GitHub, add the GitHub Topic copier-template.

Copier supports having multiple template directories in the same template repository. This requires that every set of files uses the same questions. To avoid issues occurring later on, I would recommend only having one template directory for each template repository.

Example Configuration File for the Template #

This is an example of a copier.yaml file:

---
# Configuration for Copier Template
#
# See:
#
# https://copier.readthedocs.io/en/stable/
#

# This template uses the configuration format introduced in Copier version 9.
# Specify the minimum version of Copier that you will support.
_min_copier_version: "9.17"

# Use this subdirectory of the template repository as the root directory of the template.
_subdirectory: template

# Path of the answers file in projects that use this template.
# The file name must be unique to avoid conflicts with other Copier templates.
# Start the name of the file with .copier-answers so that Renovate can detect it.
# Copier will automatically create the .copier/ directory that this path specifies.
_answers_file: .copier/.copier-answers-myorg-myteam-aws-lambda-py.yaml

# Use alternate template delimiters.
# This avoids conflicts with templating in the managed files.
_envops:
  block_end_string: "%]"
  block_start_string: "[%"
  comment_end_string: "#]"
  comment_start_string: "[#"
  variable_end_string: "}]"
  variable_start_string: "[{"

# Create these files from the template, if they are not already present.
# If one of these files exists, it will not be updated from the template.
_skip_if_exists:
  - LICENSE.md
  - README.md

To avoid conflicts with other Copier templates that projects may use, the _answers_file must specify a unique name.

You also define the questions for the template in the configuration file. The Copier maintainers recommend that you use the well-known variable names for common variables. Set default answers for questions as much as possible, because they minimise user effort and increase consistency.

Copier supports including other YAML files into the configuration. To avoid issues, you should only do this if you have a specific requirement that makes it necessary.

Example Template Answers File #

Always create a template answers file. It must render the answers that are provided to YAML:

---
# Maintained by Copier: NEVER EDIT THIS FILE
#
# See:
#
# https://copier.readthedocs.io/en/stable/updating/#never-change-the-answers-file-manually

[{ _copier_answers|to_nice_yaml -}]

Copier must template the name of the answers file from _copier_conf.answers_file. This means that if you use square brackets as delimiters, the file itself will be called [{_copier_conf.answers_file}].jinja in the template repository.

Managing Copier Answers Files #

Each Copier template must have a separate answers file in the projects that use it. templates. This means that every Copier template must specify a unique name for the answers file that it creates. The names of the answers files become particularly important if projects may use templates that are maintained by different groups.

Here are some guidelines for naming Copier templates answers files:

  • Start the name of each answers file with .copier-answers, so that Renovate and other tools can identify it as a Copier answers file.
  • Use hyphens as separators.
  • Include namespaces in the template name, such as your company and your team.
  • Include a unique identifier for the template.
  • Use the same name for the answers file and the repository.
  • Always use the file extension .yaml, so that the file is identified as a YAML file.

For example, you could name the answers file for a Copier template for AWS Lambda projects as .copier-answers-mycompany-myteam-aws-lambda-py.yaml, so that the template has the namespaces mycompany and myteam and the template identifier aws-lambda-py. The repository could then have the name copier-mycompany-myteam-aws-lambda-py.

This article also suggests that you set answers files with a path that includes a .copier/ directory. This simplifies automation and reduces clutter in the root directory of each project. For example:

_answers_file: .copier/.copier-answers-myorg-myteam-aws-lambda-py.yaml

By default, Copier puts answers files in the root directory of projects. If the _answers_file path includes directories then Copier automatically creates those directories in a project when it renders the answers file.

Referencing Other Copier Templates #

A Copier template can reference other templates that have already been applied to the project:

# Template loads answers from the previous template into the _external_data object
_external_data:
  # A dynamic path. Make sure you answer that question
  # before the first access to the data (with `_external_data.parent_tpl`)
  parent_tpl: "[{ parent_tpl_answers_file }]"

# Ask the user where the answers file for the previous template is located
parent_tpl_answers_file:
  help: Where did you store answers for the parent template?
  default: .copier/.copier-answers-example-parent.yaml

# Answers can then reference answers in the other template through _external_data
project_name:
  type: str
  help: Your project name
  default: "[{ _external_data.parent_tpl.project_name }]"

Automating Version Tags #

Always automate the release process for your templates. This ensures that every version of a Copier template has a Git tag in the valid format. Popular tools for release automation include:

I provide an article on using Python Semantic Release with GitLab.

Resources #

Here are some useful articles and tutorials about Copier.

Official Documentation #

Tutorials #

Tips and Suggestions #

Media #

Example Templates #