prev. article next article
webservices

advanced webservices on nixos revisited

30 Apr 2014

motivation

NixOS is all about containment. in NixOS practically every program is running in its own environment, seing only the very libraries it requires to work. as a result NixOS uses the dynamic linker very differently as every program more or less behaves like statically linked (expressed by dynamic linking to a very specific and unique path). for programming langues not using the dynamic linker (aka RPATH) there are different methods of containment like using shell scripts which export the PYTHONPATH variable for python script. while the mentioned containment methods are well established in the NixOS community, doing the same with webservices is not.

the focus of this posting is about having multiple seamless webservice(s) on the same NixOS or Nix based host. it is not about NixOps or Disnix where one could already push several webservices to several hosts on a webservice per host basis. my focus lies on finding a new way how to create isolated, individual environments which work side by side.

in this posting i want to share my thoughts about isolating webservices in the same way and discuss benefits/drawbacks of various approaches. it is also a followup posting to [1]. i’ve held a talk about declarative webservices [2] at the easterhegg14 but since it was in german here is the english writeup.

warning

please note: nixos-containers are implemented using systemd-nspawn and not LXC containers. the NixOS documentation [5] warns that commands ran as root in a container might damage the host system. they are meant for development currently and not for deployment. i use them for demonstration purpose and if someone wants to use them in production it could be a good idea to implement LXC into NixOS first.

new ideas for webservice packaging

approach 1 (what we already have)

approach 1 is implemented in [8], where you would use httpd with a serviceType like mediawiki. i’ve described this in detail in [1], so here be only the summary:

note: this is no news for Nixers but for devs coming from different distributions it indeed is.

discussion

example usage of approach 1: see webservice3 in [7] where i made use of the mediawiki serviceType.

approach 2 (what my easterhegg14 example implements)

how i think webservice composition should be approached:

example for ‘approach 2’ using apache as ‘reverse proxy’

let’s look at an excerpt from [7]:

# for debugging use:
#   journalctl status httpd.service
#   journalctl -b -u httpd

# and do not forget to start the containers manually (even the declarative containers)
#
# once you did nixos-rebuild switch; 
# for i in `seq 1 5`; do nixos-container start web$i; done
# for i in `seq 1 5`; do mkdir -p /var/lib/containers/web$i/webroot/; done
# for i in `seq 1 5`; do echo "<?php phpinfo(); ?>" > /var/lib/containers/web$i/webroot/index.php; done
# for i in `seq 1 5`; do echo "hello world from web$i" > /var/lib/containers/web$i/webroot/index.html; done

services.httpd = {
  enable = true;
  enableSSL = false;
  adminAddr = "web0@example.org";
  virtualHosts =
  [ 
    # webservice0 vhost
    # index.html works, browsing to index.php shows that php is not enabled
    { 
      serverAliases = ["webservice0"];
      documentRoot = "/webroot/";
      extraConfig = ''
        <Directory "/webroot/">
          Options -Indexes
          AllowOverride None
          Order allow,deny
          Allow from all
        </Directory>

        <IfModule mod_dir.c>
        DirectoryIndex index.html
        </IfModule>
      '';
    }
    # webservice1 vhost
    { 
      hostName = "webservice1";
      serverAliases = ["webservice1"];
      extraConfig = ''
        # prevent a forward proxy! 
        ProxyRequests off

        # User-Agent / browser identification is used from the original client
        ProxyVia Off
        ProxyPreserveHost On 

        <Proxy *>
        Order deny,allow
        Allow from all
        </Proxy>

        ProxyPass / http://192.168.101.11:80/
        ProxyPassReverse / http://192.168.101.11:80/
      '';
    }
    ## webservice2 vhost
    { 
      hostName = "webservice2";
      serverAliases = ["webservice2"];
      extraConfig = ''
        # prevent a forward proxy! 
        ProxyRequests off

        # User-Agent / browser identification is used from the original client
        ProxyVia Off
        ProxyPreserveHost On 

        <Proxy *>
        Order deny,allow
        Allow from all
        </Proxy>

        ProxyPass / http://192.168.102.11:80/
        ProxyPassReverse / http://192.168.102.11:80/
      '';
    }
    # webservice3 vhost
    { 
      hostName = "webservice3";
      serverAliases = ["webservice3"];
      extraConfig = ''
        # prevent a forward proxy! 
        ProxyRequests off

        # User-Agent / browser identification is used from the original client
        ProxyVia Off
        ProxyPreserveHost On 

        <Proxy *>
        Order deny,allow
        Allow from all
        </Proxy>

        ProxyPass / http://192.168.103.11:80/
        ProxyPassReverse / http://192.168.103.11:80/
      '';
    }
    # webservice4 vhost
    { 
      hostName = "webservice4";
      serverAliases = ["webservice4"];
      extraConfig = ''
        # prevent a forward proxy! 
        ProxyRequests off

        # User-Agent / browser identification is used from the original client
        ProxyVia Off
        ProxyPreserveHost On 

        <Proxy *>
        Order deny,allow
        Allow from all
        </Proxy>

        ProxyPass / http://192.168.104.11:80/
        ProxyPassReverse / http://192.168.104.11:80/
      '';
    }
    # webservice5 vhost
    { 
      hostName = "webservice5";
      serverAliases = ["webservice5"];
      extraConfig = ''
        # prevent a forward proxy! 
        ProxyRequests off

        # User-Agent / browser identification is used from the original client
        ProxyVia Off
        ProxyPreserveHost On 

        <Proxy *>
        Order deny,allow
        Allow from all
        </Proxy>

        ProxyPass / http://192.168.105.11:80/
        ProxyPassReverse / http://192.168.105.11:80/
      '';
    }

  ];
};

containers.web1 = {
  privateNetwork = true;
  hostAddress = "192.168.101.10";
  localAddress = "192.168.101.11";
  
  config = { config, pkgs, ... }: { 
    networking.firewall = {
      enable = true;
      allowedTCPPorts = [ 80 443 ];
    };
    services.httpd = {
      enable = true;
      enableSSL = false;
      adminAddr = "web1@example.org";
      documentRoot = "/webroot";
      # we override the php version for all uses of pkgs.php with this, 
      #  nix-env -qa --xml | grep php
      # lists available versions of php
      extraModules = [
        { name = "php5"; path = "${pkgs.php}/modules/libphp5.so"; }
      ];
    };
  };
};

containers.web2 = {
  privateNetwork = true;
  hostAddress = "192.168.102.10";
  localAddress = "192.168.102.11";
  
  config = { config, pkgs, ... }: { 
    # two additional programs are installed in the environment
    environment.systemPackages = with pkgs; [
      wget
      nmap
    ];
    networking.firewall = {
      enable = true;
      allowedTCPPorts = [ 80 443 ];
    };
    services.httpd = {
      enable = true;
      enableSSL = false;
      adminAddr = "web2@example.org";
      documentRoot = "/webroot";
      extraModules = [
        # here we are using php-5.3.28 instead of php-5.4.23
        { name = "php5"; path = "${pkgs.php53}/modules/libphp5.so"; }
      ];
    };
  };
};

# container with a mediawiki instance
containers.web3 = {
  privateNetwork = true;
  hostAddress = "192.168.103.10";
  localAddress = "192.168.103.11";
  
  config = { config, pkgs, ... }: { 
    networking.firewall = {
      enable = true;
      allowedTCPPorts = [ 80 443 ];
    };
    services.postgresql = {
      enable=true;
      package = pkgs.postgresql92;
      authentication = pkgs.lib.mkOverride 10 ''
        local mediawiki all ident map=mwusers
        local all all ident
      '';
      identMap = ''
        mwusers root   mediawiki
        mwusers wwwrun mediawiki
      '';
    };
    services.httpd = {
      enable = true;
      enableSSL = false;
      adminAddr = "web3@example.org";
      documentRoot = "/webroot";

      virtualHosts =
      [ 
        {
          serverAliases = ["webservice3"];

          extraConfig = ''
            RedirectMatch ^/$ /wiki
          '';
          extraSubservices =
          [
            {
              serviceType = "mediawiki";
              siteName = "webservice3";
            }
          ];
        }
      ];
    };
  };
};

# lighttpd hello world example
containers.web4 = {
  privateNetwork = true;
  hostAddress = "192.168.104.10";
  localAddress = "192.168.104.11";
  config = { config, pkgs, ... }: { 
    networking.firewall = {
      enable = true;
      allowedTCPPorts = [ 80 443 ];
    };
    services.lighttpd = {
      enable = true;
      document-root = "/webroot";
    };
  };
};

# nginx hello world example
containers.web5 = {
  privateNetwork = true;
  hostAddress = "192.168.105.10";
  localAddress = "192.168.105.11";
  config = { config, pkgs, ... }: { 
    networking.firewall = {
      enable = true;
      allowedTCPPorts = [ 80 443 ];
    };
    services.nginx = {
      enable = true;
      config = ''
        error_log  /webroot/error.log;
         
        events {}
         
        http {
          server {
            access_log /webroot/access.log;
            listen 80;
            root /webroot;
          }
        }
      '';
    };
  };
};

explanation:

you can use this configuration by copying the needed parts into your own configuration.nix and afterwards run:

nixos-rebuild switch

note: you have to adapt the extraHosts IP address.

then proceed with preparing the nixos-containers as shown in line 1 to 11

discussion

approach 3: how this compares to alternatives like docker and nix-docker

what is the difference between docker and nix-docker:

docker on NixOS already works pretty well. example (run all commands as root):

nix-env -i docker
docker -d 

then on a different shell you can start using docker:

docker images
docker pull ubuntu

and then run something

docker run -t -i --privileged ubuntu:12.10 bash

docker let’s you start webservices [10] from containers, like:

docker run -d -p 8080:80 my-site

these ports can obviously be added to the reverse-proxy setup as the internal nixos-containers.

discussion

conclusion

related to reverse-proxy setup:

NixOS webservice ideas:

update 5.5.2014:

declaratively described webservices can be used without containersi as well. i don’t like this since:

please see [11] where zef hemel did discuss this already.

thanks

thanks to:

links