Welcome to Arion documentation

Introduction

Arion is a tool for building and running applications that consist of multiple docker containers using NixOS modules. It has special support for docker images that are built with Nix, for a smooth development experience and improved performance.

It is built on top of Docker Compose, which implements the container orchestration functionality.

Instead of configuring the compositions in YAML files like docker-compose.yaml, Arion uses the Nix language to declare the compositions. Because of this, Arion gives you the ability to declare your deployments, configuration and packaging in the same language. By replacing multiple tools with a single language, you decrease your mental load and you can more easily refactor and maintain your configurations.

Although Arion can be used as a Docker Compose with an improved configuration front end, there is more to be gained from integrating with Nix. In particular, the more structured approach of Nix compared to Dockerfiles allows the following:

Build components of your image in parallel, automatically
Share packages between images, regardless of the order they were added
Improve performance by skipping container image creation
Work with structured data instead of strings, templates and a multitude of expression languages
Refactor across deployments, configuration and packaging

Arion allows to compose containers with different granularity:

Minimal: Plain command using nixpkgs
NixOS: run only one systemd service
NixOS: run full OS
Docker image from DockerHub

Full NixOS is supported on * docker-compose + podman with docker socket (NixOS >= 21.05) * docker-compose + docker, before cgroupsv2 (NixOS < 21.05)

podman-compose support is currently WIP on a separate branch.

Installation

Nix

$ nix-env -iA arion -f https://github.com/hercules-ci/arion/tarball/master

NixOS

Add this module to your NixOS configuration:

{ pkgs, ... }: {
  environment.systemPackages = [
    pkgs.arion

     # Do install the docker CLI to talk to podman.
     # Not needed when virtualisation.docker.enable = true;
    pkgs.docker-client
  ];

  # Arion works with Docker, but for NixOS-based containers, you need Podman
  # since NixOS 21.05.
  virtualisation.docker.enable = false;
  virtualisation.podman.enable = true;
  virtualisation.podman.dockerSocket.enable = true;
  virtualisation.podman.defaultNetwork.dnsname.enable = true;

  # Use your username instead of `myuser`
  users.extraUsers.myuser.extraGroups = ["podman"];
}

Usage

Arion is configured declaratively with two files:

arion-pkgs.nix

Arion needs arion-pkgs.nix to import nixpkgs, for example:

import <nixpkgs> { system = "x86_64-linux"; }

or more sophisticated (recommended) setup with Niv.

arion-compose.nix

Describe containers using NixOS-style modules. There are a few options:

Minimal: Plain command using nixpkgs

examples/minimal/arion-compose.nix:

{ pkgs, ... }:
{
  config.services = {

    webserver = {
      service.useHostStore = true;
      service.command = [ "sh" "-c" ''
                  cd "$$WEB_ROOT"
                  ${pkgs.python3}/bin/python -m http.server
                '' ];
      service.ports = [
        "8000:8000" # host:container
      ];
      service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
    };
  };
}

NixOS: run only one systemd service

examples/nixos-unit/arion-compose.nix:

{
  services.webserver = { config, pkgs, ... }: {

    nixos.configuration = {config, pkgs, ...}: {
      boot.isContainer = true;
      services.nginx.enable = true;
      services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
      system.build.run-nginx = pkgs.writeScript "run-nginx" ''
        #!${pkgs.bash}/bin/bash
        PATH='${config.systemd.services.nginx.environment.PATH}'
        echo nginx:x:${toString config.users.users.nginx.uid}:${toString config.users.groups.nginx.gid}:nginx web server user:/var/empty:/bin/sh >>/etc/passwd
        echo nginx:x:${toString config.users.groups.nginx.gid}:nginx >>/etc/group
        ${config.systemd.services.nginx.runner}
      '';
    };
    service.command = [ config.nixos.build.run-nginx ];
    service.useHostStore = true;
    service.ports = [
      "8000:80" # host:container
    ];
  };
}

NixOS: run full OS

examples/full-nixos/arion-compose.nix:

{
  services.webserver = { pkgs, ... }: {
    nixos.useSystemd = true;
    nixos.configuration.boot.tmpOnTmpfs = true;
    nixos.configuration.services.nginx.enable = true;
    nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
    service.useHostStore = true;
    service.ports = [
      "8000:80" # host:container
    ];
  };
}

Docker image from DockerHub

{
  services.postgres = {
    service.image = "postgres:10";
    service.volumes = [ "${toString ./.}/postgres-data:/var/lib/postgresql/data" ];
    service.environment.POSTGRES_PASSWORD = "mydefaultpass";
  };
}

Run

Start containers and watch their logs:

$ arion up -d
$ arion logs -f

You can go to examples/*/ and run these commands to give it a quick try.

Inspect the config

While developing an arion project, you can make use of arion repl, which launches a nix repl on the project configuration.

$ arion repl
Launching a repl for you. To get started:

To see deployment-wide configuration
  type config. and use tab completion
To bring the top-level Nixpkgs attributes into scope
  type :a (config._module.args.pkgs) // { inherit config; }

Welcome to Nix. Type :? for help.

Loading '../../src/nix/eval-composition.nix'...
Added 5 variables.

nix-repl> config.services.webserver.service.command
[ "sh" "-c" "cd \"$$WEB_ROOT\"\n/nix/store/66fbv9mmx1j4hrn9y06kcp73c3yb196r-python3-3.8.9/bin/python -m http.server\n" ]

nix-repl>

Build with Nix

You can build a project with nix-build using an expression like

arion.build { modules = [ ./arion-compose.nix ]; pkgs = import ./arion-pkgs.nix; }

If you deploy with runArion, and your pkgs variable is equivalent to import ./arion-pkgs.nix, you can use:

let
  deployment = pkgs.effects.runArion { /* ... */ });
in deployment.prebuilt

Project Status

This project was born out of a process supervision need for local development environments while working on Hercules CI. (It was also born out of ancient Greek deities disguised as horses. More on that later.)

Arion can be used for simple single host deployments, using Docker’s TLS client verification, or virtualisation.podman.networkSocket options. Remote deployments do not support useHostStore, although an SSH-based deployment method could support this. Docker Swarm is not currently supported.

Arion has run successfully on Linux distributions other than NixOS, but we only perform CI for Arion on NixOS.

How it works

Arion is essentially a thin wrapper around Nix and docker-compose. When it runs, it does the following:

Evaluate the configuration using Nix, producing a docker-compose.yaml and a garbage collection root
Invoke docker-compose
Clean up the garbage collection root

Most of the interesting stuff happens in Arion’s Nix expressions, where it runs the module system (known from NixOS) and provides the configuration that makes the Docker Compose file do the things it needs to do.

One of the more interesting built-in modules is the host-store.nix module which performs the bind mounts to make the host Nix store available in the container.

FAQ

Do I need to use Hercules CI?

Nope, it’s just Nix and Docker Compose under the hood.

It does integrate nicely though.

What about garbage collection?

Arion removes the need for garbage collecting docker images, delegating this task to Nix when using service.useHostStore.

Arion creates a garbage collection root that it cleans up after completing the command. This means that arion up -d should not be used with useHostStore in production. Instead, disable useHostStore, which will use dockerTools to generate images that can be used in production.

(And we felt morally obliged to name our stuff after Greek mythology)