Build your python project with nix in 10 minutes

!!! Warning: dream2nix is unstable software. While simple UX is one of our main focus points, the APIs are still under development. Do expect changes that will break your setup.

This guide walks you through the process of setting up nix for your project using dream2nix. This will allow your project's build and dev-environment to be reproduced by machines of other developers or CI systems with high accuracy.

Outline

  1. Install nix
  2. Navigate to your python project
  3. Initialize the dream2nix flake
  4. Define target platform(s)
  5. List the available packages
  6. Build the project
  7. Create a development shell
  8. Resolve impurities

Install nix

If you don't have nix already, check out nixos.org/download.html on how to install it.

Enable the nix flakes feature

For internal dependency management dream2nix requires the experimental nix feature flakes being enabled.

export NIX_CONFIG="extras-experimental-features = flakes nix-command"

If you find yourself using dream2nix regularly, you can permanently save these settings by adding the following line to your /etc/nix/nix.conf:

experimental-features = flakes nix-command

In this example I will clone the python project httpie to /tmp/my_project as an example.

> git clone https://github.com/httpie/httpie /tmp/my_project
> cd ./my_project

Initialize the dream2nix flake.nix

> nix flake init -t github:nix-community/dream2nix#simple
wrote: /tmp/my_project/flake.nix

Great, this created a new file flake.nix which is like a recipe that tells nix how to build our python project or how to assemble a development environment for it. By modifying this file, we can tweak settings and change the way our package gets built by nix. But for now we just go with the defaults.

Define target platform(s)

We have the flake setup, now we need to define the supported systems, this is necessary because nix can do multi platform and cross-platform builds so we need to tell it what can be built and where.

There are 2 ways to do this, either with a nix_systems file, or we can write the target platforms inline to our flake.nix.

nix_systems

We can create a nix_systems file with the current system:

> nix eval --impure --raw --expr 'builtins.currentSystem' > ./nix_systems
> git add ./nix_systems

The nix_systems file is simply a list of the supported systems, for example:

x86_64-linux

Remember to add the file ./nix_systems to git, or it won't be picked up by nix. If you want to support more platforms later, just add more lines to that file.

inline

Alternatively, we can define the targets in the flake.nix like so:

{
  inputs.dream2nix.url = "github:nix-community/dream2nix";
  outputs = inp:
    inp.dream2nix.lib.makeFlakeOutputs {
      systems = ["x86_64-linux"];         # <- This line.
      config.projectRoot = ./.;
      source = ./.;
    };
}

This has the advantage of keeping all the configuration in a single file.

List the available packages

Let's get an overview of what the flake.nix allows us to do with our project.

> nix flake show
warning: Git tree '/tmp/my_project' is dirty
warning: creating lock file '/tmp/my_project/flake.lock'
warning: Git tree '/tmp/my_project' is dirty
git+file:///tmp/my_project
├───packages
│   └───x86_64-linux
│       ├───main: package 'main'
│       └───resolveImpure: package 'resolve'
└───projectsJson: unknown

What we can observe here:

  1. warning: Git tree '/tmp/my_project' is dirty Nix warns us that the current git repo has uncommited changes. Thats fine, because we like to experiment for now. This warning will go away as soon as we commit our changes.

  2. warning: creating lock file '/tmp/my_project/flake.lock' Our flake.nix imported external libraries. The versions of these libraries have now been locked inside a new file flake.lock. We should later commit this file to the repo, in order to allow others to reproduce our build exactly.

  3.   git+file:///tmp/my_project
      ├───packages
      │   └───x86_64-linux
      │       ├───main: package 'main'
      │       └───resolveImpure: package 'resolve'
      └───projectsJson: unknown
    

    Similar like a .json file defines a structure of data, our flake.nix defines a structure of nix attributes which are things that we can build or run with nix. We can see that it contains packages for my current platform x86_64-linux.

    The packages which we can see here is my python package and a package called resolveImpure, which is a special package provided by dream2nix which we will learn more about later.

Build the project

Let's try building our project. If you get an error about unresolved impurities, see Resolve Impurities

> nix build .#default

Congratulations, your build artifacts will now be accessible via the ./result directory. If your project contains executables, you can run these via ./result/bin/executable-name. If you want to develop on your python project, see Create a development shell

Create a development shell

Nix can provide you with a development shell containing all your project's dependencies. First, ensure that your project is resolved, then execute the following command.

> nix develop -c $SHELL

The -c $SHELL part is only necessary if you use a different shell than bash and would like to bring that shell with you into the dev environment.

Resolve impurities

If you try to build, you might run into the following error.

> nix build .#default
error: The python package main contains unresolved impurities.
       Resolve by running the .resolve attribute of this derivation
       or by resolving all impure projects by running the `resolveImpure` package

Oops. It seems like our project does not contain enough information for dream2nix to construct a reproducible build. But this is not a problem as we can fix this by using the resolveImpure package that dream2nix provides.

> nix run .#resolveImpure
...
adding file to git: dream2nix-packages/main/dream-lock.json

Fine, that created a new file dream-lock.json which is a lock file specifically for our python project. If we later add any dependencies, we will have to re-run resolveImpure to update this lock file.

Now everything should be ready to Build the Project