docker-images/README.md

# docker-images

> Dockerfiles and scripts to build various Docker images I rely on.

This repository contains a bunch of Dockerfiles that I've written myself. I run them on my Hashicorp [Consul](https://consul.io/) and [Nomad](https://www.nomadproject.io/) cluster, which is comprised of 5 Raspberry Pi 4B+ boards (3 server nodes for both Consul and Nomad).


## System Requirements
 - Linux
 - Bash
 - Docker


## Getting started
The script that does the magic is called `imagebuilder.sh` in the root of this repository. Run it to get usage information:

```bash
./imagebuilder.sh
```

## FAQ

### Why write your own Dockerfiles for services that already ship with one?
Multiple reasons:

1. Educational purposes (learning how to write Dockerfiles; how Docker works, etc)
2. Bandwidth reduction / speed: basing them on my custom base image that proxies apt through a local [apt-cacher-ng](https://wiki.debian.org/AptCacherNg) instance
3. Security: I know precisely how the Dockerfile works and everything it depends on, because I've written it myself
4. Compatibility:
	- I use [Hashicorp Nomad](https://www.nomadproject.io/), so some of these Dockerfiles are written explicitly with Hashicorp Nomad in mind - e.g. the [`NOMAD_PORT_*` environment variables](https://www.nomadproject.io/docs/job-specification/network#port-parameters).
	- My Hashicorp Nomad cluster is comprised chiefly of Raspberry Pis (currently running armv7l, but an upgrade to arm64 is planned eventually), and many Docker containers on the Docker Hub are built by default for amd64
5. Maintainability: I want to ensure I keep my Docker images up-to-date, so I rebuild them myself regularly via my [Continuous Integration server](https://laminar.ohwg.net/) ([see also](https://starbeamrainbowlabs.com/blog/article.php?article=posts/392-own-your-code-series-list.html), and [also this](https://starbeamrainbowlabs.com/blog/article.php?article=posts/451-cluster-11-lock-and-key.html))

### Why do I need to run a private Docker registry for `imagebuilder.sh` to work?
`imagebuilder.sh` is designed to automatically build the specified Docker image and then push it to a private Docker registry because then the hosts in my Hashicorp Nomad 

### These Dockerfiles don't work for me!
These Dockerfiles are specific to my environment. They depend on a patched version of `minideb` as a base image, which this package is also responsible for building. The key changes to `minideb` include:

1. Config directive to tell `apt` to use my local apt-cacher-ng instance to save bandwidth / speed things up
2. Apt repository definition for [my personal apt repository](https://apt.starbeamrainbowlabs.com/).

To set your own apt caching proxy address, do this before calling `imagebuilder.sh build minideb`:

```bash
export proxy_address="http://example.com:3142";
```

Note that an apt caching proxy is *required* for it to work. If you don't yet have one setup, I have a blog post about it here: [Cluster, Part 5: Staying current | Automating apt updates and using apt-cacher-ng](https://starbeamrainbowlabs.com/blog/article.php?article=posts/411-cluster-5-staying-current.html)


### I've found a security issue, how can I contact you?
Please use the contact details on my website and _privately_ get in touch (don't leave a public comment on my blog): <https://starbeamrainbowlabs.com/>


## Image Catalogue

Image		     	| Purpose
--------------------|----------------
`certbot`			| Dockerised certbot via certbot-auto - currently doesn't build anymore because "certbot doesn't support your OS anymore" or something like that, but the version pushed to our private registry works just fine until we can rectify the issue
`docker-registry-ui`| [docker registry ui](https://github.com/Joxit/docker-registry-ui), dockerised
`etherpad`			| Dockerised [etherpad](https://etherpad.org/) (currently faulty, see [this GitHub issue](https://github.com/ether/etherpad-lite/issues/4962))
`gossa`				| Dockerised [gossa](https://github.com/pldubouilh/gossa/)
`jellyfin`			| Dockerised [jellyfin](https://jellyfin.org/)
`minetest`			| Dockerised server for [minetest](https://github.com/minetest/minetest),, currently under construction
`minetest-mapserver`| Dockerised [minetest-mapserver](https://github.com/minetest-mapserver/mapserver)
`minideb`			| Our main base image for (most) other images. Built from [minideb](https://github.com/bitnami/minideb), but customised to use our local apt-cacher-ng instance.
`minideb-node`		| `minideb` with the latest Node.js installed via our [apt repository](https://apt.starbeamrainbowlabs.com/)
`mosquitto`			| Dockerised Mosquitto MQTT server
`node-serve`		| `minideb-node` with [serve](https://www.npmjs.com/package/serve) installed & set as the entrypoint
`paperless-ng`		| Dockerised [paperless-ng](https://github.com/jonaswinkler/paperless-ng) - currently under construction
`redis`				| Dockerised [redis](https://redis.io/), installs the latest stable version
`shiori`			| Dockerised [shiori](https://github.com/go-shiori/shiori), built from source
`imap-download`     | Fetchmail, procmail, inotifywait, and munpack working together to download emails and extract attachments. The provided fetchmailrc must be owned by `10000:10000`.
`archivebox`        | Dockerised [ArchiveBox](https://archivebox.io/)
`tinyproxy`         | Dockerised [tinyproxy](https://tinyproxy.github.io/) - config file should be mounted read-only to `/srv/tinyproxy.conf`
`json2collectdmqtt`	| Dockerised [json2collectdmqtt](https://github.com/sbrl/json2collectdmqtt) - `KEY=value` environment file should be mounted to `/srv/env`


## Docker container UID/GID map

UID     | GID	| Container				| Notes
--------|-------|-----------------------|-----------------
3		| 3		| docker-registry-ui	| 
60		| 60	| minetest-mapserver	| 
70		| 70	| etherpad				| 
80		| 80	| serve					| Static HTTP Server based on Node.js
85		| 85	| mosquitto				| 
90		| 90	| jellyfin				| 
95		| 95	| shiori				| Shiori bookmark system, built from source
999		| 994	| certbot				| The same user & group as fabio, because file permissions
2100	| 2100  | redis					| 
0		| 0		| imap-download			| Uses fetchmail to download emails and extract attachments. The provided fetchmailrc must be owned by `10000:10000`, and be chmodded to `0700`.
10200	| 10200	| archivebox			| 
10300   | 10300 | tinyproxy             | 
10400	| 10400	| json2collectdmqtt		| `chown` environment file to `10400:10400`, as it's loaded by a startup shell script inside the container
10500	| 10500	| n8n					| 

<!--
Container from the main public Docker registry:

(none documented yet, though we do use one for music.mooncarrot.space)

Note that user ids for these are set in the Nomad job spec file, and not in the Dockerfile for the container itself.
-->

## Development Notes

At present, 3 image types are present:

 - **`base`:** A base image - a script is called with an auto-generated output directory as the 1st and only argument. When the script exits the directory is checked for a `.tar.gz` file
 - **`base-nopush`:** A variant of the above that doesn't automatically push to the docker registry on completion.
 - **`docker`:** A `Dockerfile` is is built with `docker build` before being pushed to the docker registry.
 - **`import`:** Imports a Docker image from another Docker registry.

### Creating a new image
Each image should have it's own subdirectory inside the `image` directory. The following files should be present for a `docker` image type:

 - `type.txt`: Should contain the word `docker`
 - `Dockerfile`: The Dockerfile to build

Any other files are given to Docker as [build context](https://stackoverflow.com/a/44466648/1460422).

For `base` and `base-nopush` image types, the following files should be present:

 - `IMAGE_NAME.sh`: A `.sh` file named after the name of the parent directory. For example, the `minideb` image contains the script `minideb.sh`
 - `type.txt`: Should contain either the word `base` or `base-nopush`

For `import` image types, the file `imagetype.txt` should be present and contain the (fully qualified) name of the image to import.

Optionally, any image type can contain the following files:

 - `dependents.txt`: The names of images that depend on this image - 1 image name per file. This is read by my continuous integration system to queue rebuilds of dependent Docker containers once the current image has finished building & pushing automatically.
 - `nomad.txt`: Nomad jobs that should be restarted after the docker image is rebuilt. The format of `nomad.txt` is 1 definition per line in the form `jobname	taskname`.
   - In future, it will just be 1 jobname per line when [proer job restarting](https://github.com/hashicorp/nomad/issues/698) gets implemented


## Licence
The contents of this repository is licenced under the _Mozilla Public License 2.0_ (MPL-2.0). This license can be found in the _LICENSE_ file in this repository.