System Requirements: nixos-anywhere

Install NixOS everywhere via ssh

Documentation Index

Requirements

Source Machine

1. Supported Systems:

   • Linux or macOS computers with Nix installed.
   • NixOS
   • Windows systems using WSL2.

   Note: Should be able to build nix derivations for the target platform. Otherwise --build-on-remote can be used.

2. Nix Installation: If Nix is not yet installed on your system, refer to the nix installation page.

Destination Machine

The machine must be reachable over the public internet or local network. Nixos-anywhere does not support wifi networks. If a VPN is needed, define a custom installer via the --kexec flag which connects to your VPN.

1. Direct Boot Option:

   • Must be already running a NixOS installer.

2. Alternative Boot Options: If not booting directly from a NixOS installer image:

   • Architecture & Support: Must be operating on:
     • x86-64 or aarch64 Linux systems with kexec support. Note: While most x86-64 Linux systems support kexec, if you're using an architecture other than those mentioned, you may need to specify a different kexec image manually.
   • Memory Requirements:
     • At least 1 GB of RAM (excluding swap space).

Quickstart Guide: nixos-anywhere

Install NixOS everywhere via ssh

Documentation Index

Introduction

This guide documents a simple installation of NixOS using nixos-anywhere on a target machine running x86_64 Linux with kexec support. The example used in this guide installs NixOS on a Hetzner cloud machine. The configuration may be different for some other instances. We will be including further examples in the How To Guide as and when they are available.

You will need:

• A flake that controls the actions to be performed
• A disk configuration containing details of the file system that will be created on the new server.
• A target machine that is reachable via SSH, either using keys or a password, and the privilege to either log in directly as root or a user with password-less sudo.

nixos-anywhere doesn’t need to be installed. You can run it directly from the Github repository.

Details of the flake, the disk configuration and the CLI command are discussed below.

Steps required to run nixos-anywhere

1. Enable Flakes

Check if your nix has flakes enabled by running nix flake. It will tell you if it's not. To enable flakes, refer to the NixOS Wiki.

2. Initialize a Flake

The easiest way to start is to copy our example flake.nix into a new directory. This example is tailored for a virtual machine setup similar to one on Hetzner Cloud, so you might need to adapt it for your setup.

If you already have a flake, you can use it by adding disko configuration to it.

3. Configure your SSH key

If you cloned our nixos-anywhere-example you will also replace the SSH key like this: In your configuration, locate the line that reads:

# change this to your ssh key
            "CHANGE"

Replace the text CHANGE with your own SSH key. This is crucial, as you will not be able to log into the target machine post-installation without it.

4. Configure Storage

In the same directory, create a file called disk-config.nix. This file will define the disk layout for the disko tool, which is used by nixos-anywhere to partition, format, and mount the disks.

For a basic installation, you can copy the contents from the example provided here. This configuration sets up a standard GPT (GUID Partition Table) that is compatible with both EFI and BIOS systems and mounts the disk as /dev/sda. You may need to adjust /dev/sda to match the correct disk on your machine. To identify the disk, run the lsblk command and replace sda with the actual disk name.

For example, on this machine, we would select nvme0n1 as the disk:

NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
nvme0n1     259:0    0  1.8T  0 disk

If this setup does not match your requirements, you can choose an example that better suits your disk layout from the disko examples. For more detailed information, refer to the disko documentation.

5. Lock your Flake

nix flake lock

This will download your flake dependencies and make a flake.lock file that describes how to reproducibly build your system.

Optionally, you can commit these files to a repo such as Github, or you can simply reference your local directory when you run nixos-anywhere. This example uses a local directory on the source machine.

6. Connectivity to the Target Machine

nixos-anywhere will create a temporary SSH key to use for the installation. If your SSH key is not found, you will be asked for your password. If you are using a non-root user, you must have access to sudo without a password. To avoid SSH password prompts, set the SSHPASS environment variable to your password and add --env-password to the nixos-anywhere command.

7. (Optional) Test your NixOS and Disko configuration

Skip this step and continue with Step 8, if you don't have a hardware configuration (hardware-configuration.nix or facter.json) generated yet or make sure you don't import non-existing hardware-configuration.nix or facter.json during running the vm test.

The following command will automatically test your nixos configuration and run disko inside a virtual machine, where

• <path to configuration> is the path to the directory or repository containing flake.nix and disk-config.nix

• <configuration name> must match the name that immediately follows the text nixosConfigurations. in the flake, as indicated by the comment in the example.

nix run github:nix-community/nixos-anywhere -- --flake <path to configuration>#<configuration name> --vm-test

8. Prepare Hardware Configuration

If you're not using a virtual machine, it's recommended to allow nixos-anywhere to generate a hardware configuration during installation. This ensures that essential drivers, such as those required for disk detection, are properly configured.

To enable nixos-anywhere to integrate its generated configuration into your NixOS setup, you need to include an import for the hardware configuration beforehand.

Here’s an example:

 nixosConfigurations.generic = nixpkgs.lib.nixosSystem {
   system = "x86_64-linux";
   modules = [
     disko.nixosModules.disko
     ./configuration.nix
+    ./hardware-configuration.nix
   ];
 };

When running nixos-anywhere, this file is automatically generated by including the following flags in your command: --generate-hardware-config nixos-generate-config ./hardware-configuration.nix. The second flag, ./hardware-configuration.nix, specifies where nixos-generate-config will store the configuration. Adjust this path to reflect the location where you want the hardware-configuration.nix for this machine to be saved.

8.1 nixos-facter

As an alternative to nixos-generate-config, you can use the experimental nixos-facter command, which offers more comprehensive hardware reports and advanced configuration options.

To use nixos-facter, add the following to your flake inputs:

 {
+ inputs.nixos-facter-modules.url = "github:numtide/nixos-facter-modules";
 }

Next, import the module into your configuration and specify facter.json as the path where the hardware report will be saved:

 nixosConfigurations.generic-nixos-facter = nixpkgs.lib.nixosSystem {
   system = "x86_64-linux";
   modules = [
     disko.nixosModules.disko
     ./configuration.nix
+    nixos-facter-modules.nixosModules.facter
+    { config.facter.reportPath = ./facter.json }
   ];
 };

To generate the configuration for nixos-facter with nixos-anywhere, use the following flags: --generate-hardware-config nixos-facter ./facter.json. The second flag, ./facter.json, specifies where nixos-generate-config will store the hardware report. Adjust this path to suit the location where you want the facter.json to be saved.

9. Run it

You can now run nixos-anywhere from the command line as shown below, where:

• <path to configuration> is the path to the directory or repository containing flake.nix and disk-config.nix

• <configuration name> must match the name that immediately follows the text nixosConfigurations. in the flake, as indicated by the comment in the example.

• <ip address> is the IP address of the target machine.

nix run github:nix-community/nixos-anywhere -- --flake <path to configuration>#<configuration name> --target-host root@<ip address>

The command would look  like this if you had created your files in a directory named /home/mydir/test and the IP address of your target machine is 37.27.18.135:

nix run github:nix-community/nixos-anywhere -- --flake /home/mydir/test#hetzner-cloud --target-host root@37.27.18.135

If you also need to generate hardware configuration amend flags for nixos-generate-config:

nix run github:nix-community/nixos-anywhere -- --generate-hardware-config nixos-generate-config ./hardware-configuration.nix --flake <path to configuration>#<configuration name> --target-host root@<ip address>

Or these flags if you are using nixos-facter instead:

nix run github:nix-community/nixos-anywhere -- --generate-hardware-config nixos-facter ./facter.json  --flake <path to configuration>#<configuration name> --target-host root@<ip address>

Adjust the location of ./hardware-configuration.nix and ./facter.json accordingly.

nixos-anywhere will then run, showing various output messages at each stage. It may take some time to complete, depending on Internet speeds. It should finish by showing the messages below before returning to the command prompt.

Installation finished. No error reported.
Warning: Permanently added '<ip-address>' (ED25519) to the list of known hosts

When this happens, the target server will have been overwritten with a new installation of NixOS. Note that the server's public SSH key will have changed.

If you have previously accessed this server using SSH, you may see the following message the next time you try to log in to the target.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.
Please contact your system administrator.
Add correct host key in ~/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in ~/.ssh/known_hosts:6
  remove with:
  ssh-keygen -f ~/.ssh/known_hosts" -R "<ip address>"
Host key for <ip_address> has changed and you have requested strict checking.
Host key verification failed.

This is because the known_hosts file in the .ssh directory now contains a mismatch, since the server has been overwritten. To solve this, use a text editor to remove the old entry from the known_hosts file (or use the command ssh-keygen -R <ip_address>). The next connection attempt will then treat this as a new server.

The error message line Offending ECDSA key in ~/.ssh/known_hosts:6 gives the line number that needs to be removed from the known_hosts file (line 6 in this example).

Finished!

nixos-anywhere's job is now done, as it is a tool to install NixOS onto the target machine.

Any future changes to the configuration should be made to your flake. You would reference this flake when using the NixOS nixos-rebuild command or a separate 3rd party deployment tool of your choice i.e. deploy-rs, colmena, nixinate.

To update on the machine locally (replace <URL to your flake> with your flake i.e. .# if your flake is in the current directory):

nixos-rebuild switch --flake <URL to your flake>

To update remotely you will need to have configured an ssh server and your ssh key for the root user:

nixos-rebuild switch --flake <URL to your flake> --target-host "root@<ip address>"

See the Nix documentation for use of the flake URL-like syntax.

For more information on different use cases of nixos-anywhere please refer to the How to Guide, and for more technical information and explanation of known error messages, refer to the Reference Manual.

How To Guide: nixos-anywhere

Install NixOS everywhere via ssh

Documentation Index

Contents

Installing on a machine with no operating system

Copying files to the new installation

Using your own kexec image

Repair installations without wiping data

Secrets and full disk encryption

Use without flakes

Terraform

Nix-channels / NIX_PATH

IPv6-only targets

Installing on a machine with no operating system

If your machine doesn't currently have an operating system installed, you can still run nixos-anywhere remotely to automate the install. To do this, you would first need to boot the target machine from the standard NixOS installer. You can either boot from a USB or use netboot.

The NixOS installation guide has detailed instructions on how to boot the installer.

When you run nixos-anywhere, it will determine whether a NixOS installer is present by checking whether the /etc/os-release file contains the identifier VARIANT_ID=installer. This identifier is available on releases NixOS 23.05 or later.

If an installer is detected, nixos-anywhere will not attempt to kexec into its own image. This is particularly useful for targets that don't have enough RAM for kexec or don't support kexec.

NixOS starts an SSH server on the installer by default, but you need to set a password in order to access it. To set a password for the nixos user, run the following command in a terminal on the NixOS machine:

passwd

If you don't know the IP address of the installer on your network, you can find it by running the following command:

$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:12:34:56 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic noprefixroute eth0
       valid_lft 86385sec preferred_lft 75585sec
    inet6 fec0::5054:ff:fe12:3456/64 scope site dynamic mngtmpaddr noprefixroute
       valid_lft 86385sec preferred_lft 14385sec
    inet6 fe80::5054:ff:fe12:3456/64 scope link
       valid_lft forever preferred_lft forever

This will display the IP addresses assigned to your network interface(s), including the IP address of the installer. In the example output below, the installer's IP addresses are 10.0.2.15, fec0::5054:ff:fe12:3456, and fe80::5054:ff:fe12:3456%eth0:

To test if you can connect and your password works, you can use the following SSH command (replace the IP address with your own):

ssh -v nixos@fec0::5054:ff:fe12:3456

You can then use the IP address to run nixos-anywhere like this:

nix run github:nix-community/nixos-anywhere -- --flake '.#myconfig' --target-host nixos@fec0::5054:ff:fe12:3456

This example assumes a flake in the current directory containing a configuration named myconfig.

Using your own kexec image

By default, nixos-anywhere downloads the kexec image from the NixOS images repository.

However, you can provide your own kexec image file if you need to use a different one. This is particularly useful for architectures other than x86_64 and aarch64, since they don't have a pre-build image.

To do this, use the --kexec command line switch followed by the path to your image file. The image will be uploaded prior to execution.

Here's an example command that demonstrates how to use a custom kexec image with nixos-anywhere:

nix run github:nix-community/nixos-anywhere -- \
  --kexec "$(nix build --print-out-paths github:nix-community/nixos-images#packages.aarch64-linux.kexec-installer-nixos-unstable-noninteractive)/nixos-kexec-installer-noninteractive-aarch64-linux.tar.gz" \
  --flake 'github:your-user/your-repo#your-system' \
  root@yourip

Make sure to replace github:your-user/your-repo#your-system with the appropriate Flake URL representing your NixOS configuration.

The example above assumes that your local machine can build for aarch64 in one of the following ways:

• Natively

• Through a remote builder

• By emulating the architecture with qemu using the following NixOS configuration:

{
  boot.binfmt.emulatedSystems = [ "aarch64-linux" ];
}

Secrets and full disk encryption

The nixos-anywhere utility offers the capability to install secrets onto a target machine. This feature is particularly beneficial when you want to bootstrap secrets management tools such as sops-nix or agenix, which rely on machine-specific secrets to decrypt other uploaded secrets.

Example: Decrypting an OpenSSH Host Key with pass

In this example, we demonstrate how to use a script to decrypt an OpenSSH host key from the pass password manager and subsequently pass it to nixos-anywhere during the installation process:

#!/usr/bin/env bash

# Create a temporary directory
temp=$(mktemp -d)

# Function to cleanup temporary directory on exit
cleanup() {
  rm -rf "$temp"
}
trap cleanup EXIT

# Create the directory where sshd expects to find the host keys
install -d -m755 "$temp/etc/ssh"

# Decrypt your private key from the password store and copy it to the temporary directory
pass ssh_host_ed25519_key > "$temp/etc/ssh/ssh_host_ed25519_key"

# Set the correct permissions so sshd will accept the key
chmod 600 "$temp/etc/ssh/ssh_host_ed25519_key"

# Install NixOS to the host system with our secrets
nixos-anywhere --extra-files "$temp" --flake '.#your-host' --target-host root@yourip

Example: Uploading Disk Encryption Secrets

In a similar vein, nixos-anywhere can upload disk encryption secrets, which are necessary during formatting with disko. Here's an example that demonstrates how to provide your disk encryption password as a file or via the pass utility to nixos-anywhere:

# Write your disk encryption password to a file
echo "my-super-safe-password" > /tmp/disk-1.key

# Call nixos-anywhere with disk encryption keys
nixos-anywhere \
  --disk-encryption-keys /tmp/disk-1.key /tmp/disk-1.key \
  --disk-encryption-keys /tmp/disk-2.key <(pass my-disk-encryption-password) \
  --flake '.#your-host' \
  root@yourip

In the above example, replace "my-super-safe-password" with your actual encryption password, and my-disk-encryption-password with the relevant entry in your pass password store. Also, ensure to replace '.#your-host' and root@yourip with your actual flake and IP address, respectively.

Example: Using existing SSH host keys

If the system contains existing trusted /etc/ssh/ssh_host_* SSH host keys and certificates, nixos-anywhere can copy them in case they are necessary during installation and system activation.

nixos-anywhere --copy-host-keys --flake '.#your-host' root@yourip

This would copy /etc/ssh/ssh_host_* to /mnt after kexec but before installation, ignoring files that already exist in destination.

Use without flakes

First, import the disko NixOS module in your NixOS configuration and define disko devices as described in the examples.

Let's assume that your NixOS configuration lives in configuration.nix and your target machine is called machine:

1. Download your favourite disk layout:

See https://github.com/nix-community/disko-templates/ for more examples:

The example below will work with both UEFI and BIOS-based systems.

curl https://raw.githubusercontent.com/nix-community/disko-templates/main/single-disk-ext4/disko-config.nix > ./disko-config.nix

2. Get a hardware-configuration.nix from on the target machine

• Option 1: If NixOS is not installed, boot into an installer without first installing NixOS.

• Option 2: Use the kexec tarball method, as described here.

• Generate Configuration: Run the following command on the target machine:

  nixos-generate-config --no-filesystems --dir /tmp/config
  

This creates the necessary configuration files under /tmp/config/. Copy /tmp/config/nixos/hardware-configuration.nix to your local machine into the same directory as disko-config.nix.

3. Set NixOS version to use

# default.nix
let
  # replace nixos-24.11 with your preferred nixos version or revision from here: https://status.nixos.org/
  nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/archive/refs/heads/nixos-24.11.tar.gz";
in
import (nixpkgs + "/nixos/lib/eval-config.nix") {
  modules = [ ./configuration.nix ];
}

4. Write a NixOS configuration

# configuration.nix
{
  imports = [
   "${fetchTarball "https://github.com/nix-community/disko/tarball/master"}/module.nix"
    ./disko-config.nix
    ./hardware-configuration.nix
  ];
  # Replace this with the system of the installation target you want to install!!!
  disko.devices.disk.main.device = "/dev/sda";

  # Set this to the NixOS version that you have set in the previous step.
  # For more information, see `man configuration.nix` or https://nixos.org/manual/nixos/stable/options#opt-system.stateVersion .
  system.stateVersion = "24.11";
}

5. Build and deploy with nixos-anywhere

Your current directory now should contain the following files from the previous step:

• configuration.nix, default.nix, disko-config.nix and hardware-configuration.nix

Run nixos-anywhere as follows:

nixos-anywhere --store-paths $(nix-build -A config.system.build.disko -A config.system.build.toplevel --no-out-link) root@machine

Terraform

The nixos-anywhere terraform modules allow you to use Terraform for installing and updating NixOS. It simplifies the deployment process by integrating nixos-anywhere functionality.

Our terraform module requires the null and external provider.

You can get these by from nixpkgs like this:

nix-shell -p '(pkgs.terraform.withPlugins (p: [ p.null p.external ]))'

You can add this expression the packages list in your devshell in flake.nix or in shell.nix.

Checkout out the module reference for examples and module parameter on how to use the modules.

Nix-channels / NIX_PATH

nixos-anywhere does not install channels onto the new system by default to save time and disk space. This for example results in errors like:

(stack trace truncated; use '--show-trace' to show the full trace)

error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)

at «none»:0: (source not available)

when using tools like nix-shell/nix-env that rely on NIX_PATH being set.

Solution 1: Set the NIX_PATH via nixos configuration (recommended)

Instead of stateful channels, one can also populate the NIX_PATH using nixos configuration instead:

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
  # ... other inputs

  outputs = inputs@{ nixpkgs, ... }:
    {
      nixosConfigurations.yoursystem = nixpkgs.lib.nixosSystem {
        system = "x86_64-linux"; # adapt to your actual system
        modules = [
          # This line will populate NIX_PATH
          { nix.nixPath = [ "nixpkgs=${inputs.nixpkgs}" ]; }
          # ... other modules and your configuration.nix
        ];
      };
    };
}

Advantage: This solution will be automatically kept up-to-date every time the flake is updated.

In your shell you will see something in your $NIX_PATH:

$ echo $NIX_PATH
/root/.nix-defexpr/channels:nixpkgs=/nix/store/8b61j28rpy11dg8hanbs2x710d8w3v0d-source

Solution 2: Manually add the channel

On the installed machine, run:

$ nix-channel --add https://nixos.org/channels/nixos-unstable nixos
$ nix-channel --update

NixOS-anywhere on IPv6-only targets

As GitHub engineers still haven't enabled the IPv6 switch, the kexec image hosted on GitHub, cannot be used unfortunately on IPv6-only hosts. However it is possible to use an IPv6 proxy for GitHub content like that:

nixos-anywhere \
  --kexec https://gh-v6.com/nix-community/nixos-images/releases/download/nixos-unstable/nixos-kexec-installer-noninteractive-x86_64-linux.tar.gz \
...

This proxy is hosted by numtide. It also works for IPv4.

Alternatively it is also possible to reference a local file:

nixos-anywhere \
  --kexec ./nixos-kexec-installer-noninteractive-x86_64-linux.tar.gz \
...

This tarball will be then uploaded via sftp to the target.

Reference Manual: nixos-anywhere

Install NixOS everywhere via ssh

Documentation Index

TODO: Populate this guide properly

Contents

Command Line Usage

Explanation of known error messages

Command Line Usage

Usage: nixos-anywhere [options] <ssh-host>

Options:

* -f, --flake <flake_uri>
  set the flake to install the system from.
* -i <identity_file>
  selects which SSH private key file to use.
* -p, --ssh-port <ssh_port>
  set the ssh port to connect with
* --ssh-option <ssh_option>
  set an ssh option
* -L, --print-build-logs
  print full build logs
* --env-password
  set a password used by ssh-copy-id, the password should be set by
  the environment variable SSHPASS
* -s, --store-paths <disko-script> <nixos-system>
  set the store paths to the disko-script and nixos-system directly
  if this is given, flake is not needed
* --kexec <path>
  use another kexec tarball to bootstrap NixOS
* --kexec-extra-flags
  extra flags to add into the call to kexec, e.g. "--no-sync"
* --post-kexec-ssh-port <ssh_port>
  after kexec is executed, use a custom ssh port to connect. Defaults to 22
* --copy-host-keys
  copy over existing /etc/ssh/ssh_host_* host keys to the installation
* --extra-files <path>
  contents of local <path> are recursively copied to the root (/) of the new NixOS installation. Existing files are overwritten
  Copied files will be owned by root. See documentation for details.
* --disk-encryption-keys <remote_path> <local_path>
  copy the contents of the file or pipe in local_path to remote_path in the installer environment,
  after kexec but before installation. Can be repeated.
* --no-substitute-on-destination
  disable passing --substitute-on-destination to nix-copy
* --debug
  enable debug output
* --option <key> <value>
  nix option to pass to every nix related command
* --from <store-uri>
  URL of the source Nix store to copy the nixos and disko closure from
* --build-on-remote
  build the closure on the remote machine instead of locally and copy-closuring it
* --vm-test
  build the system and test the disk configuration inside a VM without installing it to the target.
* --generate-hardware-config nixos-facter|nixos-generate-config <path>
  generate a hardware-configuration.nix file using the specified backend and write it to the specified path.
  The backend can be either 'nixos-facter' or 'nixos-generate-config'.
* --phases
  comma separated list of phases to run. Default is: kexec,disko,install,reboot
  kexec: kexec into the nixos installer
  disko: first unmount and destroy all filesystems on the disks we want to format, then run the create and mount mode
  install: install the system
  reboot: unmount the filesystems, export any ZFS pools and reboot the machine
* --disko-mode disko|mount|format
  set the disko mode to format, mount or destroy. Default is disko.
  disko: first unmount and destroy all filesystems on the disks we want to format, then run the create and mount mode
  mount: mount the partition at the specified root-mountpoint
  format: create partition tables, zpools, lvms, raids and filesystems (Experimental: Can be run increntally, but use with caution and good backups)

Explanation of known error messages

TODO: Add additional error messages and meanings. Fill in missing explanations

This section lists known error messages and their explanations. Some explanations may refer to the following CLI syntax:

nix run github:nix-community/nixos-anywhere -- --flake <path to configuration>#<configuration name> root@<ip address>

This list is not comprehensive. It's possible you may encounter errors that originate from the underlying operating system. These should be documented in the relevant operating system manual.