<tt>libnixf</tt>, The nix frontend

This constitutes a frontend for the Nix language, encompassing a lexer, parser, and semantic analysis, with a prime emphasis on enhancing the editing experience.

Background & Motivation

Over the years, the Nix community has seen a variety of parsers developed by different hackers, however, not all of them are suitable for editing purposes. For example, the official parser, generated by yacc, consists of declarative files (with some grammar rules included). Yacc-generated parsers prioritize speed but can lack robust error handling capabilities. Consequently, when editing files within your workspace, which may be incomplete and riddled with errors, this deficiency becomes apparent. The need for a new Nix parser arises from challenges in implementing robust error recovery and providing helpful error hints.

For instance, addressing error recovery regarding missing semicolons:

{
    a = 1
    b = 2
}

The official parser will complain there are some errors happens in the second eq sign =, not the first one:

nix-repl> {
              a = 1
              b = 2
error: syntax error, unexpected '=', expecting ';'
       at «string»:3:7:
            2|     a = 1
            3|     b = 2
             |       ^

But we actually know the error happens right after "1", right?

The second challenge is, the offical nix parser stops right after the first error token encountered. However for language servers, you may want to see it is going on parsing, recovered from previous errors.

Modules

Currently the package has these following components.

• Basic (Diagnostic Types, Node Types, ...)
• Parse (The parsing algorithm implementation)
• Sema (Semantic Analysis, e.g. duplicated keys)
• Bytecode (experimental)

The Basic module

This module mainly exposing nixf interfaces, and it's definitions are shared among other modules.

Diagnostic and Fixes

A Diagnostic have a set of Notes attached to it. And each diagnostic has a set of fixes. Each fix has a name briefly describe the plan. For example

• "insert ;"
• "remove ...".

And each fix has a series of Edits. The removing range, and the new text to replace it.

These "fixes" could be applied automatically, and also may pop a "💡" and let you choose that fix.

AST Nodes

Each node in the AST is owned by its parent and becomes immutable immediately after construction.

Additionally, the libnixf AST incorporates semantic information, which is populated by Semantic Actions within the Sema module.

The Parse module

The parsing algorithm

This is a recursive descent parser not completely consistent with the official one in case of error recovery.

The objective is twofold:

1. Ensure a consistent experience when code can be accurately evaluated.
2. Offer a significantly improved experience when code cannot be correctly evaluated.

For instance, the following text will undergo different parsing outcomes in libnixf.

{
    a = 1  # <-- look at here
    b = 2;
}

In the official Nix parser, it would be parses as a = (Call 1, b) for the a attribute. So you will see the strange diagnostic:

nix-repl> {
              a = 1
              b = 2
error: syntax error, unexpected '=', expecting ';'
       at «string»:3:7:
            2|     a = 1
            3|     b = 2
             |       ^

However, in the libnixf parser, since 1 will definitely not be evaluated as a lambda, it will not be called. Thus, the result would be:

{
    a = 1  # <-- missing semicolon ";"
    b = 2; # Correctly parse this as "b = 2"
}

Standalone tools provided along libnixf

nixf-tidy

Usage

nixf-tidy
 
--pretty-print  Pretty print the JSON data
--variable-lookup  Peform Variable Lookup Analysis on the source code, including detecting dead code, and unused variables.

Basic CLI usage:

cat foo.nix | nixf-tidy --variable-lookup

A dedicated tool tailored for linting Nix projects, nixf-tidy is crafted specifically for seamless integration into CI pipelines like GitHub Actions or commit hooks. As such, it prioritizes functionality over flashy CLI output. Tasks such as rendering code lines or applying color distinctions are beyond the scope of nixf-tidy.

The JSON fields provided by nixf-tidy are documented here.

Use nixf-tidy in your workflow

Please update this section if you want to take fancy "rendering" job for libnixf, and link your projects!