Introduction

What is pyproject.nix

Pyproject.nix is a collection of Nix utilities to work with Python project metadata in Nix. It mainly targets PEP-621 compliant pyproject.toml files and data formats, but also implement support for other & legacy formats such as Poetry & requirements.txt.

Pyproject.nix aims to be a swiss army knife of simple customizable utilities that works together with the nixpkgs Python infrastructure.

Foreword

This documentation only helps you to get started with pyproject.nix. As it's a toolkit with many use cases not every use case can be documented fully.

This documentation is centered around packaging Python applications & managing development environments. For other use cases see the reference documentation.

Concepts

pyproject.nix introduces a few high level abstract concepts. The best way to get started is to understand these concepts and how they fit together.

Project

A project attribute set is a high-level representation of a project that includes:

The parsed pyproject.toml file
Parsed dependencies
Project root directory

It can can be loaded from many different sources:

PEP-621 pyproject.toml
PEP-621 pyproject.toml with PDM extensions
Poetry pyproject.toml
requirements.txt

Validators

Validators work on dependency constraints as defined in a project and offers validation for them. This can be useful to check that a package set is compilant with the specification.

Renderers

A renderer takes a project together with a Python interpreter derivation and renders it into a form understood by various pieces of nixpkgs Python infrastructure.

For example: The buildPythonPackage renderer returns an attribute set that can be passed to either nixpkgs function buildPythonPackage or buildPythonApplication.

There might be information missing from what a renderer returned depending on what can be computed from the project. If any attributes are missing you can manually merge your own attribute set with what the renderer returned.

Tying it together

For a concrete example use see Use cases -> pyproject.toml.

pyproject.toml

It's possible to develop PEP-621 compliant Python projects without using any Python package manager except Nix.

This example loads pyproject.toml to create an environment using python.withPackages and a consumable package using python.pkgs.buildPythonPackage.

flake.nix

{
  description = "A basic flake using pyproject.toml project metadata";

  inputs.pyproject-nix.url = "github:nix-community/pyproject.nix";
  inputs.pyproject-nix.inputs.nixpkgs.follows = "nixpkgs";

  outputs = { nixpkgs, pyproject-nix, ... }:
    let
      inherit (nixpkgs) lib;

      # Loads pyproject.toml into a high-level project representation
      # Do you notice how this is not tied to any `system` attribute or package sets?
      # That is because `project` refers to a pure data representation.
      project = pyproject-nix.lib.project.loadPyproject {
        # Read & unmarshal pyproject.toml relative to this project root.
        # projectRoot is also used to set `src` for renderers such as buildPythonPackage.
        projectRoot = ./.;
      };

      # This example is only using x86_64-linux
      pkgs = nixpkgs.legacyPackages.x86_64-linux;

      # We are using the default nixpkgs Python3 interpreter & package set.
      #
      # This means that you are purposefully ignoring:
      # - Version bounds
      # - Dependency sources (meaning local path dependencies won't resolve to the local path)
      #
      # To use packages from local sources see "Overriding Python packages" in the nixpkgs manual:
      # https://nixos.org/manual/nixpkgs/stable/#reference
      #
      # Or use an overlay generator such as pdm2nix:
      # https://github.com/adisbladis/pdm2nix
      python = pkgs.python3;

    in
    {
      # Create a development shell containing dependencies from `pyproject.toml`
      devShells.x86_64-linux.default =
        let
          # Returns a function that can be passed to `python.withPackages`
          arg = project.renderers.withPackages { inherit python; };

          # Returns a wrapped environment (virtualenv like) with all our packages
          pythonEnv = python.withPackages arg;

        in
        # Create a devShell like normal.
        pkgs.mkShell {
          packages = [ pythonEnv ];
        };

      # Build our package using `buildPythonPackage
      packages.x86_64-linux.default =
        let
          # Returns an attribute set that can be passed to `buildPythonPackage`.
          attrs = project.renderers.buildPythonPackage { inherit python; };
        in
        # Pass attributes to buildPythonPackage.
          # Here is a good spot to add on any missing or custom attributes.
        python.pkgs.buildPythonPackage (attrs // {
          env.CUSTOM_ENVVAR = "hello";
        });
    };
}

pyproject.toml

[project]
name = "spam"
version = "2020.0.0"
description = "Lovely Spam! Wonderful Spam!"
readme = "README.rst"
requires-python = ">=3.8"
license = {file = "LICENSE.txt"}
keywords = ["egg", "bacon", "sausage", "tomatoes", "Lobster Thermidor"]
authors = [
  {email = "hi@pradyunsg.me"},
  {name = "Tzu-ping Chung"}
]
maintainers = [
  {name = "Brett Cannon", email = "brett@python.org"}
]
classifiers = [
  "Development Status :: 4 - Beta",
  "Programming Language :: Python"
]

dependencies = [
  "httpx",
  "gidgethub[httpx]>4.0.0",
  "django>2.1; os_name != 'nt'",
  "django>2.0; os_name == 'nt'"
]

[project.optional-dependencies]
test = [
  "pytest < 5.0.0",
  "pytest-cov[all]"
]

[project.urls]
homepage = "https://example.com"
documentation = "https://readthedocs.org"
repository = "https://github.com"
changelog = "https://github.com/me/spam/blob/master/CHANGELOG.md"

[project.scripts]
spam-cli = "spam:main_cli"

[project.gui-scripts]
spam-gui = "spam:main_gui"

[project.entry-points."spam.magical"]
tomatoes = "spam:main_tomatoes"

requirements.txt

Many projects comes without proper packaging and use requirements.txt files to declare their dependencies.

This example loads requirements.txt to create an environment using python.withPackages with packages from nixpkgs.

flake.nix

{
  description = "Construct development shell from requirements.txt";

  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

  inputs.pyproject-nix.url = "github:nix-community/pyproject.nix";

  outputs =
    { nixpkgs
    , pyproject-nix
    , ...
    }:
    let
      # Load/parse requirements.txt
      project = pyproject-nix.lib.project.loadRequirementsTxt {
        projectRoot = ./.;
      };

      pkgs = nixpkgs.legacyPackages.x86_64-linux;
      python = pkgs.python3;

      pythonEnv =
        # Assert that versions from nixpkgs matches what's described in requirements.txt
        # In projects that are overly strict about pinning it might be best to remove this assertion entirely.
        assert project.validators.validateVersionConstraints { inherit python; } == { }; (
          # Render requirements.txt into a Python withPackages environment
          pkgs.python3.withPackages (project.renderers.withPackages {
            inherit python;
          })
        );

    in
    {
      devShells.x86_64-linux.default =
        pkgs.mkShell {
          packages = [
            pythonEnv
          ];
        };
    };
}

FAQ

How does package name mapping from Python to Nixpkgs work?

Package names are normalized according to the PyPA normalization specification. Nixpkgs also uses the same normalization but has some legacy package names that do not follow normalization guidelines.

The other case where the automatic mapping goes wrong is when the Nixpkgs python.pkgs set does not contain a dependency. One such example is ruff, a Python linter written in Rust.

Nixpkgs has ruff on the top-level (pkgs), but not in python3.pkgs. In such cases you can use an overlay to add the package to the Python set:

let
  python = pkgs.python3.override {
    packageOverrides = self: super: {
      ruff = pkgs.ruff;
    };
  };
in ...

How do you treat `dynamic` attributes?

Pyproject.nix makes no attempt at parsing dynamic fields as it does not have the required knowledge to infer these.

When using the withPackages renderer most fields that may be dynamic are not even relevant and won't cause issues. At other times, like when using the buildPythonPackage renderer problems occur as there is no way for the renderer to create the version attribute.

let
  project = pyproject.project.loadPyproject { pyproject = lib.importTOML ./pyproject.toml; };
  python = pkgs.python3;
  attrs = pyproject.renderers.buildPythonPackage { inherit python project; };
in python.pkgs.buildPythonPackage attrs

Will result in an error from buildPythonpackage because version is missing:

error: attribute 'version' missing

at /nix/store/gna8i238i3nnz6cizcayyfyfdzn28la5-nixpkgs/pkgs/development/interpreters/python/mk-python-derivation.nix:31:28:

    30|
    31| { name ? "${attrs.pname}-${attrs.version}"
      |                            ^
    32|

In these cases you can manually add attributes to the attribute set returned by the renderer:

let
  project = pyproject.project.loadPyproject { pyproject = lib.importTOML ./pyproject.toml; };
  python = pkgs.python3;
  attrs = pyproject.renderers.buildPythonPackage { inherit python project; };
in python.pkgs.buildPythonPackage (attrs // {
  version = "1.0";  # Not dynamically inferred
})

Reference documentation

The reference documentation is split up into two main categories:

User facing APIs

Contains high-level representations and has notions of things like a project (a fully parsed pyproject.toml) and further operations done on the project level.

Standards APIs

Contains parsers, evaluators & utility functions for dealing with Python packaging standards defined by the through the PEP process & from PyPA.

project

`lib.project.loadPyproject`

Type: loadPyproject :: AttrSet -> AttrSet

Load dependencies from a PEP-621 pyproject.toml.

structured function argument

: pyproject

: The unmarshaled contents of pyproject.toml

extrasAttrPaths

: Example: extrasAttrPaths = [ "tool.pdm.dev-dependencies" ];

projectRoot

: Path to project root

::: {.example #function-library-example-lib.project.loadPyproject}

`lib.project.loadPyproject` usage example

# loadPyproject { pyproject = lib.importTOML }
{
  dependencies = { }; # Parsed dependency structure in the schema of `lib.pep621.parseDependencies`
  build-systems = [ ];  # Returned by `lib.pep518.parseBuildSystems`
  pyproject = { }; # The unmarshaled contents of pyproject.toml
  projectRoot = null; # Path to project root
  requires-python = null; # requires-python as parsed by pep621.parseRequiresPython
}

:::

`lib.project.loadPDMPyproject`

Type: loadPDMPyproject :: AttrSet -> AttrSet

Load dependencies from a PDM pyproject.toml.

structured function argument

: pyproject

: The unmarshaled contents of pyproject.toml

projectRoot

: Path to project root

pdmLock

: The unmarshaled contents of pdm.lock

::: {.example #function-library-example-lib.project.loadPDMPyproject}

`lib.project.loadPDMPyproject` usage example

# loadPyproject { pyproject = lib.importTOML }
{
  dependencies = { }; # Parsed dependency structure in the schema of `lib.pep621.parseDependencies`
  build-systems = [ ];  # Returned by `lib.pep518.parseBuildSystems`
  pyproject = { }; # The unmarshaled contents of pyproject.toml
  projectRoot = null; # Path to project root
  requires-python = null; # requires-python as parsed by pep621.parseRequiresPython
}

:::

`lib.project.loadPoetryPyproject`

Type: loadPoetryPyproject :: AttrSet -> AttrSet

Load dependencies from a Poetry pyproject.toml.

structured function argument

: pyproject

: The unmarshaled contents of pyproject.toml

projectRoot

: Path to project root

poetryLock

: The unmarshaled contents of pyproject.toml

::: {.example #function-library-example-lib.project.loadPoetryPyproject}

`lib.project.loadPoetryPyproject` usage example

# loadPoetryPyproject { pyproject = lib.importTOML }
{
  dependencies = { }; # Parsed dependency structure in the schema of `lib.pep621.parseDependencies`
  build-systems = [ ];  # Returned by `lib.pep518.parseBuildSystems`
  pyproject = { }; # The unmarshaled contents of pyproject.toml
  projectRoot = null; # Path to project root
  requires-python = null; # requires-python as parsed by pep621.parseRequiresPython
}

:::

`lib.project.loadRequirementsTxt`

Type: loadRequirementsTxt :: AttrSet -> AttrSet

Load dependencies from a requirements.txt.

Note that as requirements.txt is lacking important project metadata this is incompatible with some renderers.

structured function argument

: requirements

: The contents of requirements.txt

projectRoot

: Path to project root

::: {.example #function-library-example-lib.project.loadRequirementsTxt}

`lib.project.loadRequirementsTxt` usage example

# loadRequirementstxt { requirements = builtins.readFile ./requirements.txt; root = ./.; }
{
  dependencies = { }; # Parsed dependency structure in the schema of `lib.pep621.parseDependencies`
  build-systems = [ ];  # Returned by `lib.pep518.parseBuildSystems`
  pyproject = null; # The unmarshaled contents of pyproject.toml
  projectRoot = null; # Path to project root
  requires-python = null; # requires-python as parsed by pep621.parseRequiresPython
}

:::

`lib.project.loadPyprojectDynamic`

Type: loadPyprojectDynamic :: AttrSet -> AttrSet

Load dependencies from a either a PEP-621 or Poetry pyproject.toml file. This function is intended for 2nix authors that wants to include local pyproject.toml files but don't know up front whether they're from Poetry or PEP-621.

structured function argument

: pyproject

: The unmarshaled contents of pyproject.toml

projectRoot

: Path to project root

::: {.example #function-library-example-lib.project.loadPyprojectDynamic}

`lib.project.loadPyprojectDynamic` usage example

# loadPyprojectDynamic { pyproject = lib.importTOML }
{
  dependencies = { }; # Parsed dependency structure in the schema of `lib.pep621.parseDependencies`
  build-systems = [ ];  # Returned by `lib.pep518.parseBuildSystems`
  pyproject = { }; # The unmarshaled contents of pyproject.toml
  projectRoot = null; # Path to project root
  requires-python = null; # requires-python as parsed by pep621.parseRequiresPython
}

:::

renderers

`lib.renderers.withPackages`

Type: withPackages :: AttrSet -> lambda

Renders a project as an argument that can be passed to withPackages

Evaluates PEP-508 environment markers to select correct dependencies for the platform but does not validate version constraints. For validation see lib.validators.

structured function argument

: project

: Project metadata as returned by lib.project.loadPyproject

python

: Python derivation

extras

: Python extras (optionals) to enable

extraPackages

: Extra withPackages function

::: {.example #function-library-example-lib.renderers.withPackages}

`lib.renderers.withPackages` usage example

# withPackages (lib.project.loadPyproject { ... })
  «lambda @ «string»:1:1»

:::

`lib.renderers.buildPythonPackage`

Type: buildPythonPackage :: AttrSet -> AttrSet

Renders a project as an argument that can be passed to buildPythonPackage/buildPythonApplication.

Evaluates PEP-508 environment markers to select correct dependencies for the platform but does not validate version constraints. For validation see lib.validators.

structured function argument

: project

: Project metadata as returned by lib.project.loadPyproject

python

: Python derivation

extras

: Python extras (optionals) to enable

extrasAttrMappings

: Map a Python extras group name to a Nix attribute set like: { dev = "checkInputs"; } This is intended to be used with optionals such as test dependencies that you might want to add to checkInputs instead of propagatedBuildInputs

format

: Which package format to pass to buildPythonPackage If the format is "wheel" PEP-518 build-systems are excluded from the build.

::: {.example #function-library-example-lib.renderers.buildPythonPackage}

`lib.renderers.buildPythonPackage` usage example

# buildPythonPackage { project = lib.project.loadPyproject ...; python = pkgs.python3;  }
  { pname = "blinker"; version = "1.3.3.7"; propagatedBuildInputs = [ ]; }

:::

validators

`lib.validators.validateVersionConstraints`

Type: validateVersionConstraints :: AttrSet -> AttrSet

Validates the Python package set held by Python (python.pkgs) against the parsed project.

Returns an attribute set where the name is the Python package derivation pname and the value is a list of the mismatching conditions.

structured function argument

: project

: Project metadata as returned by lib.project.loadPyproject

python

: Python derivation

extras

: Python extras (optionals) to enable

::: {.example #function-library-example-lib.validators.validateVersionConstraints}

`lib.validators.validateVersionConstraints` usage example

# validateVersionConstraints (lib.project.loadPyproject { ... })
{
  resolvelib = {
    # conditions as returned by `lib.pep440.parseVersionCond`
    conditions = [ { op = ">="; version = { dev = null; epoch = 0; local = null; post = null; pre = null; release = [ 1 0 1 ]; }; } ];
    # Version from Python package set
    version = "0.5.5";
  };
  unearth = {
    conditions = [ { op = ">="; version = { dev = null; epoch = 0; local = null; post = null; pre = null; release = [ 0 10 0 ]; }; } ];
    version = "0.9.1";
  };
}

:::

Reference documentation

The reference documentation is split up into two main categories:

User facing APIs

Contains high-level representations and has notions of things like a project (a fully parsed pyproject.toml) and further operations done on the project level.

Standards APIs

Contains parsers, evaluators & utility functions for dealing with Python packaging standards defined by the through the PEP process & from PyPA.

pep440

`lib.pep440.parseVersion`

Type: parseVersion :: string -> AttrSet

Parse a version according to PEP-440.

version

: Function argument

::: {.example #function-library-example-lib.pep440.parseVersion}

`lib.pep440.parseVersion` usage example

# parseVersion "3.0.0rc1"
{
  dev = null;
  epoch = 0;
  local = null;
  post = null;
  pre = {
    type = "rc";
    value = 1;
  };
  release = [ 3 0 0 ];
}

:::

`lib.pep440.parseVersionCond`

Type: parseVersionCond :: string -> AttrSet

Parse a version conditional.

cond

: Function argument

::: {.example #function-library-example-lib.pep440.parseVersionCond}

`lib.pep440.parseVersionCond` usage example

# parseVersionCond ">=3.0.0rc1"
{
  op = ">=";
  version = {
    dev = null;
    epoch = 0;
    local = null;
    post = null;
    pre = {
      type = "rc";
      value = 1;
    };
    release = [ 3 0 0 ];
  };
}

:::

`lib.pep440.parseVersionConds`

Type: parseVersionConds :: string -> [AttrSet]

Parse a list of version conditionals separated by commas.

conds

: Function argument

::: {.example #function-library-example-lib.pep440.parseVersionConds}

`lib.pep440.parseVersionConds` usage example

# parseVersionConds ">=3.0.0rc1,<=4.0"
[
  {
    op = ">=";
    version = {
      dev = null;
      epoch = 0;
      local = null;
      post = null;
      pre = {
        type = "rc";
        value = 1;
      };
      release = [ 3 0 0 ];
    };
  }
  {
    op = "<=";
    version = {
      dev = null;
      epoch = 0;
      local = null;
      post = null;
      pre = null;
      release = [ 4 0 ];
    };
  }
]

:::

`lib.pep440.compareVersions`

Type: compareVersions :: AttrSet -> AttrSet -> int

Compare two versions as parsed by parseVersion according to PEP-440.

Returns:

-1 for less than
0 for equality
1 for greater than

a

: Function argument

b

: Function argument

::: {.example #function-library-example-lib.pep440.compareVersions}

`lib.pep440.compareVersions` usage example

# compareVersions (parseVersion "3.0.0") (parseVersion "3.0.0")
0

:::

`lib.pep440.comparators`

Type: operators.${operator} :: AttrSet -> AttrSet -> bool

Map comparison operators as strings to a comparator function.

Attributes:

Compatible release clause: ~=
Version matching clause: ==
Version exclusion clause: !=
Inclusive ordered comparison clause: <=, >=
Exclusive ordered comparison clause: <, >
Arbitrary equality clause: ===

::: {.example #function-library-example-lib.pep440.comparators}

`lib.pep440.comparators` usage example

# comparators."==" (parseVersion "3.0.0") (parseVersion "3.0.0")
true

:::

pep508

`lib.pep508.parseMarkers`

Type: parseMarkers :: string -> AttrSet

Parse PEP 508 markers into an AST.

input

: Function argument

::: {.example #function-library-example-lib.pep508.parseMarkers}

`lib.pep508.parseMarkers` usage example

# parseMarkers "(os_name=='a' or os_name=='b') and os_name=='c'"
{
  lhs = {
    lhs = {
      lhs = {
        type = "variable";
        value = "os_name";
      };
      op = "==";
      rhs = {
        type = "string";
        value = "a";
      };
      type = "compare";
    };
    op = "or";
    rhs = {
      lhs = {
        type = "variable";
        value = "os_name";
      };
      op = "==";
      rhs = {
        type = "string";
        value = "b";
      };
      type = "compare";
    };
    type = "boolOp";
  };
  op = "and";
  rhs = {
    lhs = {
      type = "variable";
      value = "os_name";
    };
    op = "==";
    rhs = {
      type = "string";
      value = "c";
    };
    type = "compare";
  };
  type = "boolOp";
}

:::

`lib.pep508.parseString`

Type: parseString :: string -> AttrSet

Parse a PEP-508 dependency string.

input

: Function argument

::: {.example #function-library-example-lib.pep508.parseString}

`lib.pep508.parseString` usage example

# parseString "cachecontrol[filecache]>=0.13.0"
{
  conditions = [
    {
      op = ">=";
      version = {
        dev = null;
        epoch = 0;
        local = null;
        post = null;
        pre = null;
        release = [ 0 13 0 ];
      };
    }
  ];
  markers = null;
  name = "cachecontrol";
  extras = [ "filecache" ];
  url = null;
}

:::

`lib.pep508.mkEnviron`

Type: mkEnviron :: derivation -> AttrSet

Create an attrset of platform variables. As described in https://peps.python.org/pep-0508/#environment-markers.

python

: Function argument

::: {.example #function-library-example-lib.pep508.mkEnviron}

`lib.pep508.mkEnviron` usage example

# mkEnviron pkgs.python3
{
  implementation_name = {
    type = "string";
    value = "cpython";
  };
  implementation_version = {
    type = "version";
    value = {
      dev = null;
      epoch = 0;
      local = null;
      post = null;
      pre = null;
      release = [ 3 10 12 ];
    };
  };
  os_name = {
    type = "string";
    value = "posix";
  };
  platform_machine = {
    type = "string";
    value = "x86_64";
  };
  platform_python_implementation = {
    type = "string";
    value = "CPython";
  };
  # platform_release maps to platform.release() which returns
  # the running kernel version on Linux.
  # Because this field is not reproducible it's left empty.
  platform_release = {
    type = "string";
    value = "";
  };
  platform_system = {
    type = "string";
    value = "Linux";
  };
  # platform_version maps to platform.version() which also returns
  # the running kernel version on Linux.
  # Because this field is not reproducible it's left empty.
  platform_version = {
    type = "version";
    value = {
      dev = null;
      epoch = 0;
      local = null;
      post = null;
      pre = null;
      release = [ ];
    };
  };
  python_full_version = {
    type = "version";
    value = {
      dev = null;
      epoch = 0;
      local = null;
      post = null;
      pre = null;
      release = [ 3 10 12 ];
    };
  };
  python_version = {
    type = "version";
    value = {
      dev = null;
      epoch = 0;
      local = null;
      post = null;
      pre = null;
      release = [ 3 10 ];
    };
  };
  sys_platform = {
    type = "string";
    value = "linux";
  };
}

:::

`lib.pep508.evalMarkers`

Type: evalMarkers :: AttrSet -> AttrSet -> bool

Evaluate an environment as returned by mkEnviron against markers as returend by parseMarkers.

environ

: Function argument

value

: Function argument

::: {.example #function-library-example-lib.pep508.evalMarkers}

`lib.pep508.evalMarkers` usage example

# evalMarkers (mkEnviron pkgs.python3) (parseMarkers "python_version < \"3.11\"")
true

:::

pep518

`lib.pep518.parseBuildSystems`

Type: readPyproject :: AttrSet -> list

Parse PEP-518 build-system.requires from pyproject.toml.

pyproject

: Function argument

::: {.example #function-library-example-lib.pep518.parseBuildSystems}

`lib.pep518.parseBuildSystems` usage example

# parseBuildSystems (lib.importTOML ./pyproject.toml)
  [ ]  # List of parsed PEP-508 strings as returned by `lib.pep508.parseString`.

:::

pep599

`lib.pep599.manyLinuxTargetMachines`

Map Nixpkgs CPU values to target machines known to be supported for manylinux* wheels (a.k.a. uname -m), in nixpkgs found under the attribute stdenv.targetPlatform.parsed.cpu.name

::: {.example #function-library-example-lib.pep599.manyLinuxTargetMachines}

`lib.pep599.manyLinuxTargetMachines` usage example

# legacyAliases.powerpc64
"ppc64"

:::

pep600

`lib.pep600.legacyAliases`

Type: legacyAliases.${tag} :: AttrSet -> string

Map legacy (pre PEP-600) platform tags to PEP-600 compliant ones.

https://peps.python.org/pep-0600/#legacy-manylinux-tags

::: {.example #function-library-example-lib.pep600.legacyAliases}

`lib.pep600.legacyAliases` usage example

# legacyAliases."manylinux1_x86_64" or "manylinux1_x86_64"
"manylinux_2_5_x86_64"

:::

`lib.pep600.manyLinuxTagCompatible`

Type: manyLinuxTagCompatible :: AttrSet -> derivation -> string -> bool

Check if a manylinux tag is compatible with a given stdenv.

platform

: Platform attrset (lib.systems.elaborate "x86_64-linux")

libc

: Libc derivation

tag

: Platform tag string

::: {.example #function-library-example-lib.pep600.manyLinuxTagCompatible}

`lib.pep600.manyLinuxTagCompatible` usage example

# manyLinuxTagCompatible pkgs.stdenv.targetPlatform pkgs.stdenv.cc.libc "manylinux_2_5_x86_64"
true

:::

pep621

`lib.pep621.parseDependencies`

Type: parseDependencies :: AttrSet -> AttrSet

Parse dependencies from pyproject.toml.

structured function argument

: pyproject

: Function argument

extrasAttrPaths

: Function argument

::: {.example #function-library-example-lib.pep621.parseDependencies}

`lib.pep621.parseDependencies` usage example

# parseDependencies {
#
#   pyproject = (lib.importTOML ./pyproject.toml);
#   # Don't just look at `project.optional-dependencies` for groups, also look at these:
#   extrasAttrPaths = [ "tool.pdm.dev-dependencies" ];
# }
{
  dependencies = [ ];  # List of parsed PEP-508 strings (lib.pep508.parseString)
  extras = {
    dev = [ ];  # List of parsed PEP-508 strings (lib.pep508.parseString)
  };
  build-systems = [ ];  # PEP-518 build-systems (List of parsed PEP-508 strings)
}

:::

`lib.pep621.parseRequiresPython`

Type: parseRequiresPython :: AttrSet -> list

Parse project.python-requires from pyproject.toml

pyproject

: Function argument

::: {.example #function-library-example-lib.pep621.parseRequiresPython}

`lib.pep621.parseRequiresPython` usage example

#  parseRequiresPython (lib.importTOML ./pyproject.toml)
[ ]  # List of conditions as returned by `lib.pep440.parseVersionCond`

:::

`lib.pep621.getDependenciesNames`

Type: getDependenciesNames :: AttrSet -> AttrSet

Takes a dependency structure as returned by lib.pep621.parseDependencies and transforms it into a structure with it's package names.

::: {.example #function-library-example-lib.pep621.getDependenciesNames}

`lib.pep621.getDependenciesNames` usage example

# getDependenciesNames (pep621.parseDependencies { pyproject = (lib.importTOML ./pyproject.toml); })
{
  dependencies = [ "requests" ];
  extras = {
    dev = [ "pytest" ];
  };
  build-systems = [ "poetry-core" ];
}

:::

`lib.pep621.filterDependenciesByEnviron`

Type: filterDependenciesByEnviron :: AttrSet -> AttrSet -> AttrSet

Filter dependencies not relevant for this environment.

environ

: Environ as created by lib.pep508.mkEnviron.

extras

: Extras as a list of strings

dependencies

: Dependencies as parsed by lib.pep621.parseDependencies.

::: {.example #function-library-example-lib.pep621.filterDependenciesByEnviron}

`lib.pep621.filterDependenciesByEnviron` usage example

# filterDependenciesByEnviron (lib.pep508.mkEnviron pkgs.python3) (lib.pep621.parseDependencies (lib.importTOML ./pyproject.toml))
{ }  # Structure omitted in docs

:::

`lib.pep621.filterDependenciesByExtras`

Type: filterDependenciesByExtras :: list[string] -> AttrSet -> AttrSet

Filter dependencies by their extras groups.

extras

: Extras groups as a list of strings.

dependencies

: Dependencies as parsed by lib.pep621.parseDependencies.

::: {.example #function-library-example-lib.pep621.filterDependenciesByExtras}

`lib.pep621.filterDependenciesByExtras` usage example

# filterDependenciesByExtras [ "dev" ] (lib.pep621.parseDependencies (lib.importTOML ./pyproject.toml))
{ }  # Structure omitted in docs

:::

`lib.pep621.filterDependencies`

Type: filterDependencies :: AttrSet -> AttrSet

Aggregate of filterDependencies & filterDependenciesByExtras

structured function argument

: dependencies

: Dependencies as parsed by lib.pep621.parseDependencies

environ

: Environ as created by lib.pep508.mkEnviron

extras

: Extras as a list of strings

::: {.example #function-library-example-lib.pep621.filterDependencies}

`lib.pep621.filterDependencies` usage example

# filterDependencies {
#   dependencies = lib.pep621.parseDependencies (lib.importTOML ./pyproject.toml);
#   environ = lib.pep508.mkEnviron pkgs.python;
#   extras = [ "dev" ];
# }
{ }  # Structure omitted in docs

:::

pep656

`lib.pep656.muslLinuxTagCompatible`

Type: muslLinuxTagCompatible :: AttrSet -> derivation -> string -> bool

Check if a musllinux tag is compatible with a given stdenv.

platform

: Platform attrset (lib.systems.elaborate "x86_64-linux")

libc

: Libc derivation

tag

: Platform tag string

::: {.example #function-library-example-lib.pep656.muslLinuxTagCompatible}

`lib.pep656.muslLinuxTagCompatible` usage example

# muslLinuxTagCompatible pkgs.stdenv.targetPlatform pkgs.stdenv.cc.libc "musllinux_1_1_x86_64"
true

:::

poetry

`lib.poetry.translatePoetryProject`

Type: translatePoetryProject :: AttrSet -> lambda

Translate a Pyproject.toml from Poetry to PEP-621 project metadata. This function transposes a PEP-621 project table on top of an existing Pyproject.toml populated with data from tool.poetry. Notably does not translate dependencies/optional-dependencies.

For parsing dependencies from Poetry see lib.poetry.parseDependencies.

pyproject

: Function argument

::: {.example #function-library-example-lib.poetry.translatePoetryProject}

`lib.poetry.translatePoetryProject` usage example

# translatePoetryProject (lib.importTOML ./pyproject.toml)
{ }  # TOML contents, structure omitted. See PEP-621 for more information on data members.

:::

`lib.poetry.parseDependencies`

Type: parseDependencies :: AttrSet -> AttrSet

Parse dependencies from pyproject.toml (Poetry edition). This function is analogous to lib.pep621.parseDependencies.

pyproject

: Function argument

::: {.example #function-library-example-lib.poetry.parseDependencies}

`lib.poetry.parseDependencies` usage example

# parseDependencies {
#
#   pyproject = (lib.importTOML ./pyproject.toml);
# }
{
  dependencies = [ ];  # List of parsed PEP-508 strings (lib.pep508.parseString)
  extras = {
    dev = [ ];  # List of parsed PEP-508 strings (lib.pep508.parseString)
  };
  build-systems = [ ];  # PEP-518 build-systems (List of parsed PEP-508 strings)
}

:::

`lib.poetry.parseVersionCond`

Type: parseVersionCond :: string -> [ AttrSet ]

Parse a version conditional. Supports additional non-standard operators ^ and ~ used by Poetry.

Because some expressions desugar to multiple expressions parseVersionCond returns a list.

cond

: Function argument

pypa

`lib.pypa.normalizePackageName`

Type: normalizePackageName :: string -> string

Normalize package name as documented in https://packaging.python.org/en/latest/specifications/name-normalization/#normalization

::: {.example #function-library-example-lib.pypa.normalizePackageName}

`lib.pypa.normalizePackageName` usage example

# readPyproject "Friendly-Bard"
"friendly-bard"

:::

`lib.pypa.parsePythonTag`

Type: parsePythonTag :: string -> AttrSet

Parse Python tags.

As described in https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#python-tag.

tag

: Function argument

::: {.example #function-library-example-lib.pypa.parsePythonTag}

`lib.pypa.parsePythonTag` usage example

# parsePythonTag "cp37"
{
  implementation = "cpython";
  version = "37";
}

:::

`lib.pypa.parseABITag`

Type: parseABITag :: string -> AttrSet

Parse ABI tags.

As described in https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#python-tag.

tag

: Function argument

::: {.example #function-library-example-lib.pypa.parseABITag}

`lib.pypa.parseABITag` usage example

# parseABITag "cp37dmu"
{
  rest = "dmu";
  implementation = "cp";
  version = "37";
}

:::

`lib.pypa.isSdistFileName`

Type: isSdistFileName :: string -> bool

Check whether string is a sdist file or not.

name

: The filename string

::: {.example #function-library-example-lib.pypa.isSdistFileName}

`lib.pypa.isSdistFileName` usage example

# isSdistFileName "cryptography-41.0.1.tar.gz"
true

:::

`lib.pypa.matchWheelFileName`

Type: matchWheelFileName :: string -> [ string ]

Regex match a wheel file name, returning a list of match groups. Returns null if no match.

name

: Function argument

`lib.pypa.isWheelFileName`

Type: isWheelFileName :: string -> bool

Check whether string is a wheel file or not.

name

: The filename string

::: {.example #function-library-example-lib.pypa.isWheelFileName}

`lib.pypa.isWheelFileName` usage example

# isWheelFileName "cryptography-41.0.1-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"
true

:::

`lib.pypa.parseWheelFileName`

Type: parseFileName :: string -> AttrSet

Parse PEP-427 wheel file names.

name

: The wheel filename is {distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl.

::: {.example #function-library-example-lib.pypa.parseWheelFileName}

`lib.pypa.parseWheelFileName` usage example

# parseFileName "cryptography-41.0.1-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"
 {
  abiTag = {  # Parsed by pypa.parseABITag
    implementation = "abi";
    version = "3";
    rest = "";
  };
  buildTag = null;
  distribution = "cryptography";
  languageTags = [  # Parsed by pypa.parsePythonTag
    {
      implementation = "cpython";
      version = "37";
    }
  ];
  platformTags = [ "manylinux_2_17_aarch64" "manylinux2014_aarch64" ];
  version = "41.0.1";
}

:::

`lib.pypa.isABITagCompatible`

Type: isABITagCompatible :: derivation -> string -> bool

Check whether an ABI tag is compatible with this python interpreter.

python

: Python interpreter derivation

abiTag

: ABI tag string

::: {.example #function-library-example-lib.pypa.isABITagCompatible}

`lib.pypa.isABITagCompatible` usage example

# isABITagCompatible pkgs.python3 (pypa.parseABITag "cp37")
true

:::

`lib.pypa.isPlatformTagCompatible`

Type: isPlatformTagCompatible :: AttrSet -> derivation -> string -> bool

Check whether a platform tag is compatible with this python interpreter.

platform

: Platform attrset (lib.systems.elaborate "x86_64-linux")

libc

: Libc derivation

platformTag

: Python tag

::: {.example #function-library-example-lib.pypa.isPlatformTagCompatible}

`lib.pypa.isPlatformTagCompatible` usage example

# isPlatformTagCompatible pkgs.python3 "manylinux2014_x86_64"
true

:::

`lib.pypa.isPythonTagCompatible`

Type: isPythonTagCompatible :: derivation -> AttrSet -> bool

Check whether a Python language tag is compatible with this Python interpreter.

python

: Python interpreter derivation

pythonTag

: Python tag

::: {.example #function-library-example-lib.pypa.isPythonTagCompatible}

`lib.pypa.isPythonTagCompatible` usage example

# isPythonTagCompatible pkgs.python3 (pypa.parsePythonTag "py3")
true

:::

`lib.pypa.isWheelFileCompatible`

Type: isWheelFileCompatible :: derivation -> AttrSet -> bool

Check whether wheel file name is compatible with this python interpreter.

platform

: Platform attrset (lib.systems.elaborate "x86_64-linux")

libc

: Libc derivation

python

: Python interpreter derivation

file

: The parsed wheel filename

::: {.example #function-library-example-lib.pypa.isWheelFileCompatible}

`lib.pypa.isWheelFileCompatible` usage example

# isWheelFileCompatible pkgs.python3 (pypa.parseWheelFileName "Pillow-9.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl")
true

:::

`lib.pypa.selectWheels`

Type: selectWheels :: AttrSet -> derivation -> [ AttrSet ] -> [ AttrSet ]

Select compatible wheels from a list and return them in priority order.

platform

: Platform attrset (lib.systems.elaborate "x86_64-linux")

python

: Python interpreter derivation

files

: List of files as parsed by parseWheelFileName

::: {.example #function-library-example-lib.pypa.selectWheels}

`lib.pypa.selectWheels` usage example

# selectWheels (lib.systems.elaborate "x86_64-linux") [ (pypa.parseWheelFileName "Pillow-9.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl") ]
[ (pypa.parseWheelFileName "Pillow-9.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl") ]

:::

eggs

`lib.eggs.matchEggFileName`

Type: matchEggFileName :: string -> [ string ]

Regex match an egg file name, returning a list of match groups. Returns null if no match.

name

: Function argument

`lib.eggs.isEggFileName`

Type: isEggFileName :: string -> bool

Check whether string is an egg file or not.

name

: The filename string

::: {.example #function-library-example-lib.eggs.isEggFileName}

`lib.eggs.isEggFileName` usage example

# isEggFileName "cryptography-41.0.1-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"
false

:::

`lib.eggs.parseEggFileName`

Type: parsehEggFileName :: string -> AttrSet

Parse an egg file name.

name

: Function argument

::: {.example #function-library-example-lib.eggs.parseEggFileName}

`lib.eggs.parseEggFileName` usage example

# parseEggFileName

:::

`lib.eggs.selectEggs`

Type: selectEggs :: derivation -> [ AttrSet ] -> [ AttrSet ]

Select compatible eggs from a list and return them in priority order.

python

: Python interpreter derivation

files

: List of files parsed by parseEggFileName

pip

`lib.pip.parseRequirementsTxt`

Type: parseRequirementsTxt :: AttrSet -> list

Parse dependencies from requirements.txt

requirements

: The contents of or path to requirements.txt

::: {.example #function-library-example-lib.pip.parseRequirementsTxt}

`lib.pip.parseRequirementsTxt` usage example

# parseRequirements ./requirements.txt
[ { flags = []; requirement = {}; # Returned by pep508.parseString } ]

:::

Hacking

This document outlines hacking on pyproject.nix itself, and lays out it's project structure.

Getting started

To start hacking run nix develop -c hivemind to run the project in watch mode.

This will start up two processes:

A Nix-unit test runner
A documentation server available at http://localhost:3000

Project structure & testing

All Nix code lives in lib/. Each file has an implementation and a test suite. The attribute path to a an attribute parseVersion in lib/pep440.nix would be lib.pep440.parseVersion.

A function in lib/test.nix maps over the public interface of the library and the test suite to generate coverage tests, ensuring that every exported symbol has at least one test covering it.

Integration tests meaning tests that perform environment constructions & builds lives in test/ and are exposed through Flake checks.

The manual you are reading right now is built from the doc/ directory. To edit a specific page see the "Edit this page on GitHub" link in the footer for each respective page.

Running tests

Run the entire unit test suite $ nix-unit --flake .#libTests
Run unit tests for an individual function $ nix-unit --flake .#libTests.pep440.parseVersion
Run integration tests $ nix flake check

Formatter

Before submitting a PR format the code with nix fmt and ensure Flake checks pass with nix flake check.