docker-images/README.md
Starbeamrainbowlabs d9dcec2cc2
n8n: create basic Dockerfile
The one on the Docker hub doesn't like our setup very much
2022-11-05 01:52:44 +00:00

8.7 KiB

docker-images

Dockerfiles and scripts to build various Docker images I rely on.

This repository contains a bunch of Dockerfiles that I've written myself. I run them on my Hashicorp Consul and Nomad cluster, which is comprised of 5 Raspberry Pi 4B+ boards (3 server nodes for both Consul and Nomad).

System Requirements

  • Linux
  • Bash
  • Docker

Getting started

The script that does the magic is called imagebuilder.sh in the root of this repository. Run it to get usage information:

./imagebuilder.sh

FAQ

Why write your own Dockerfiles for services that already ship with one?

Multiple reasons:

  1. Educational purposes (learning how to write Dockerfiles; how Docker works, etc)
  2. Bandwidth reduction / speed: basing them on my custom base image that proxies apt through a local apt-cacher-ng instance
  3. Security: I know precisely how the Dockerfile works and everything it depends on, because I've written it myself
  4. Compatibility:
    • I use Hashicorp Nomad, so some of these Dockerfiles are written explicitly with Hashicorp Nomad in mind - e.g. the NOMAD_PORT_* environment variables.
    • My Hashicorp Nomad cluster is comprised chiefly of Raspberry Pis (currently running armv7l, but an upgrade to arm64 is planned eventually), and many Docker containers on the Docker Hub are built by default for amd64
  5. Maintainability: I want to ensure I keep my Docker images up-to-date, so I rebuild them myself regularly via my Continuous Integration server (see also, and also this)

Why do I need to run a private Docker registry for imagebuilder.sh to work?

imagebuilder.sh is designed to automatically build the specified Docker image and then push it to a private Docker registry because then the hosts in my Hashicorp Nomad

These Dockerfiles don't work for me!

These Dockerfiles are specific to my environment. They depend on a patched version of minideb as a base image, which this package is also responsible for building. The key changes to minideb include:

  1. Config directive to tell apt to use my local apt-cacher-ng instance to save bandwidth / speed things up
  2. Apt repository definition for my personal apt repository.

To set your own apt caching proxy address, do this before calling imagebuilder.sh build minideb:

export proxy_address="http://example.com:3142";

Note that an apt caching proxy is required for it to work. If you don't yet have one setup, I have a blog post about it here: Cluster, Part 5: Staying current | Automating apt updates and using apt-cacher-ng

I've found a security issue, how can I contact you?

Please use the contact details on my website and privately get in touch (don't leave a public comment on my blog): https://starbeamrainbowlabs.com/

Image Catalogue

Image Purpose
certbot Dockerised certbot via certbot-auto - currently doesn't build anymore because "certbot doesn't support your OS anymore" or something like that, but the version pushed to our private registry works just fine until we can rectify the issue
docker-registry-ui docker registry ui, dockerised
etherpad Dockerised etherpad (currently faulty, see this GitHub issue)
gossa Dockerised gossa
jellyfin Dockerised jellyfin
minetest Dockerised server for minetest,, currently under construction
minetest-mapserver Dockerised minetest-mapserver
minideb Our main base image for (most) other images. Built from minideb, but customised to use our local apt-cacher-ng instance.
minideb-node minideb with the latest Node.js installed via our apt repository
mosquitto Dockerised Mosquitto MQTT server
node-serve minideb-node with serve installed & set as the entrypoint
paperless-ng Dockerised paperless-ng - currently under construction
redis Dockerised redis, installs the latest stable version
shiori Dockerised shiori, built from source
imap-download Fetchmail, procmail, inotifywait, and munpack working together to download emails and extract attachments. The provided fetchmailrc must be owned by 10000:10000.
archivebox Dockerised ArchiveBox
tinyproxy Dockerised tinyproxy - config file should be mounted read-only to /srv/tinyproxy.conf
json2collectdmqtt Dockerised json2collectdmqtt - KEY=value environment file should be mounted to /srv/env

Docker container UID/GID map

UID GID Container Notes
3 3 docker-registry-ui
60 60 minetest-mapserver
70 70 etherpad
80 80 serve Static HTTP Server based on Node.js
85 85 mosquitto
90 90 jellyfin
95 95 shiori Shiori bookmark system, built from source
999 994 certbot The same user & group as fabio, because file permissions
2100 2100 redis
0 0 imap-download Uses fetchmail to download emails and extract attachments. The provided fetchmailrc must be owned by 10000:10000, and be chmodded to 0700.
10200 10200 archivebox
10300 10300 tinyproxy
10400 10400 json2collectdmqtt chown environment file to 10400:10400, as it's loaded by a startup shell script inside the container
10500 10500 n8n

Development Notes

At present, 3 image types are present:

  • base: A base image - a script is called with an auto-generated output directory as the 1st and only argument. When the script exits the directory is checked for a .tar.gz file
  • base-nopush: A variant of the above that doesn't automatically push to the docker registry on completion.
  • docker: A Dockerfile is is built with docker build before being pushed to the docker registry.
  • import: Imports a Docker image from another Docker registry.

Creating a new image

Each image should have it's own subdirectory inside the image directory. The following files should be present for a docker image type:

  • type.txt: Should contain the word docker
  • Dockerfile: The Dockerfile to build

Any other files are given to Docker as build context.

For base and base-nopush image types, the following files should be present:

  • IMAGE_NAME.sh: A .sh file named after the name of the parent directory. For example, the minideb image contains the script minideb.sh
  • type.txt: Should contain either the word base or base-nopush

For import image types, the file imagetype.txt should be present and contain the (fully qualified) name of the image to import.

Optionally, any image type can contain the following files:

  • dependents.txt: The names of images that depend on this image - 1 image name per file. This is read by my continuous integration system to queue rebuilds of dependent Docker containers once the current image has finished building & pushing automatically.
  • nomad.txt: Nomad jobs that should be restarted after the docker image is rebuilt. The format of nomad.txt is 1 definition per line in the form jobname taskname.

Licence

The contents of this repository is licenced under the Mozilla Public License 2.0 (MPL-2.0). This license can be found in the LICENSE file in this repository.