Host your own Mastodon Instance with Docker

Posted in Self Hosting

After the recent events on Twitter, the open source microblogging platform Mastodon seems to become more popular every day. The cool thing about Mastodon is, that you can run your own Mastodon instance. By doing so, you have full control over your data, and you can still interact with all the other users on any other Mastodon instance. In this blog post I’m setting up my own Mastodon instance with Docker.

Overview

A typical Mastodon instance consists of several services that all work together. Some of those services are required and others are optional. I’m going to set up an instance that consists of the following required services, each running in it’s own docker container:

  • The Mastodon web interface
  • A background processing service called Sidekiq
  • A database
  • A redis cache

And to complete the setup, I’m also going to add a container with a web proxy that automatically issues a Let’s Encrypt certificate for my Mastodon instance. As a reference I’m using the docker compose configuration from Mastodon’s GitHub repository.

If you want to follow along, all you need is a local docker installation. But if you want to be able to access your Mastodon instance from the web, you need to use a server that is accessible via a registered domain name. Otherwise you won’t be able to interact with other Mastodon instances.

The Database & Redis Containers

I’m starting with the easy part; the database. Easy because I had spun up MySQL and PostgreSQL databases with docker many times before, so I already knew how to do that. I’m using a PostgreSQL database in that case, just because that is what the reference docker compose configuration is using. Probably Mastodon could also work with other databases, but I haven’t checked that.

So to prepare the database container I’m creating the following docker compose file:

version: '3.8'

services:
  mastodon-db:
    image: 'postgres:alpine'
    volumes:
      - 'mastodon-db-volume:/var/lib/postgresql/data'
    environment:
      POSTGRES_DB: '${MASTODON_POSTGRES_DATABASE}'
      POSTGRES_USER: '${MASTODON_POSTGRES_USERNAME}'
      POSTGRES_PASSWORD: '${MASTODON_POSTGRES_PASSWORD}'

volumes:
  mastodon-db-volume:
    external: true

With that, I’m creating a services called mastodon-db which will be backed by a docker container from the image postgres:alpine, which is the official docker image of PostgreSQL. I’m also specifying a volume mount of an external volume to the path /var/lib/postgresql/data, which is the path were PostgreSQL will store the data. You could also just mount a directory from your local filesystem, but I found it easier and less error-prone in regards of permissions to use volumes, especially if I plan to run it on Windows during development. At last, I’m specifying some environment variables that are used by the container, in this case these are the name of the database as well as the username and password to access the database.

Before I can start that container, I need to do two additional things: I need to create the external volume that I specified and I need to create an environment file that contains the variables that I’m referencing in the docker compose file.

To create the external docker volume I run docker volume create mastodon-db-volume. And to specify the password I create a file called .env with the following content:

MASTODON_POSTGRES_DATABASE=mastodon
MASTODON_POSTGRES_USERNAME=mastodon-user
MASTODON_POSTGRES_PASSWORD=MySuperSecretPassword

That’s it. Now I’m ready to start the database container with docker-compose up. If you do this yourself, you should see the log output of the container in the console. I’m stopping the container with CTRL+C to continue with the next step.

Next up is the Redis container. The configuration is even easier than for the database container, because by default no special configuration is required. So the only thing I do is to add an additional service called mastodon-redis based on the redis:alpine docker image:

version: '3.8'

services:
  # ... existing mastodon-db configuration omitted ... #
  mastodon-redis:
    image: 'redis:alpine'

volumes:
  # ... existing mastodon-db-volume configuration omitted ... #

If I run docker-compose up again, I can see both containers spin up.

The Mastodon Web Interface

The next step is to set up the web interface of Mastodon. To do that, I’m adding the following service and another external docker volume to the docker compose file:

version: '3.8'

services:
  # ... existing services configuration omitted ... #
  mastodon-web:
    image: 'tootsuite/mastodon'
    command: 'bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"'
    volumes:
      - 'mastodon-volume:/mastodon/public/system'
    environment:
      RAILS_ENV: 'production'
      LOCAL_DOMAIN: '${MASTODON_DOMAIN}'
      REDIS_HOST: 'mastodon-redis' # name of the redis container
      DB_HOST: 'mastodon-db' # name of the database container
      DB_NAME: '${MASTODON_POSTGRES_DATABASE}'
      DB_USER: '${MASTODON_POSTGRES_USERNAME}'
      DB_PASS: '${MASTODON_POSTGRES_PASSWORD}'
      SECRET_KEY_BASE: '${MASTODON_SECRET_KEY_BASE}'
      OTP_SECRET: '${MASTODON_OTP_SECRET}'
    depends_on:
      - mastodon-db
      - mastodon-redis

volumes:
  # ... existing mastodon-db-volume configuration omitted ... #
  mastodon-volume:
    external: true

The docker image for the web interface is tootsuite/mastodon, which not only contains the web interface but also the background processing service and the streaming API. That’s why I tell docker to start the web interface by specifying the command to execute when the container is started. Additionally, I’m specifying that the service mastodon-web depends on the services mastodon-db and mastodon-redis to make sure they are started before the web interface. As before, I’m also mounting an external volume and configuring environment variables. Also as before, I need to create that external volume with docker volume create mastodon-volume before I can start the containers.

And you probably guessed that I also need to configure the additional environment variables in the .env file:

MASTODON_DOMAIN=social.foo.bar
# random values used for the sessions and two factor authentication tokens
# generate with cryptographically secure mechanism, for example `openssl rand -base64 48`
MASTODON_SECRET_KEY_BASE=RandomValue
MASTODON_OTP_SECRET=AnotherRandomValue

After that I’m ready to start the containers again with docker-compose up. Unfortunately this fails, because Mastodon is trying to load data from a database that is still empty. So I need to find a way to create the database schema for Mastodon.

Preparing the Database

According to the documentation I’m supposed to execute bundle exec rake mastodon:setup inside the container to do the initial configuration and prepare the database schema. But I have two problems with that approach: At first, I have already done the initial configuration by providing the environment variables via docker compose configuration. And second, that approach is interactive and requires me to either start an additional container to execute that command or to attach to the running container and execute the command inside it. But I prefer a non-interactive approach that does the required work automatically when the container starts.

So after digging through the source code of Mastodon I saw that the mastodon:setup command internally executes the db:setup command. I also saw that there is an additional db:migrate command that I can execute with bundle exec rake db:migrate. I decided to use the db:migrate command, as I can execute that multiple times. When I tried to execute the db:setup command multiple times, it failed after the first execution, warning me that I’m about to execute a destructive action. And resetting the database every time the container starts is not what I want.

So to prepare the database automatically when the container starts, I create a provision.sh file with the following content:

#!/bin/bash

bundle exec rake db:migrate

I then mount this file into the container and adjust the command that is executed when the container starts to execute the provision.sh file. The final configuration looks like this:

version: '3.8'

services:
  # ... existing services configuration omitted ... #
  mastodon-web:
    image: 'tootsuite/mastodon'
    command: 'bash -c "/provision.sh; rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"'
    volumes:
      - 'mastodon-volume:/mastodon/public/system'
      - './provision.sh:/provision.sh:ro'
    environment:
      RAILS_ENV: 'production'
      LOCAL_DOMAIN: '${MASTODON_DOMAIN}'
      REDIS_HOST: 'mastodon-redis' # name of the redis container
      DB_HOST: 'mastodon-db' # name of the database container
      DB_NAME: '${MASTODON_POSTGRES_DATABASE}'
      DB_USER: '${MASTODON_POSTGRES_USERNAME}'
      DB_PASS: '${MASTODON_POSTGRES_PASSWORD}'
      SECRET_KEY_BASE: '${MASTODON_SECRET_KEY_BASE}'
      OTP_SECRET: '${MASTODON_OTP_SECRET}'
    depends_on:
      - mastodon-db
      - mastodon-redis

volumes:
  # ... existing mastodon-db-volume configuration omitted ... #
  mastodon-volume:
    external: true

With that done I’m able to start all three containers with docker-compose up and I can see that Mastodon successfully starts:

mastodon-web_1    | => Booting Puma
mastodon-web_1    | => Rails 6.1.6 application starting in production
mastodon-web_1    | => Run `bin/rails server --help` for more startup options
mastodon-web_1    | [13] Puma starting in cluster mode...
mastodon-web_1    | [13] * Puma version: 5.6.4 (ruby 3.0.3-p157) ("Birdie's Version")
mastodon-web_1    | [13] *  Min threads: 5
mastodon-web_1    | [13] *  Max threads: 5
mastodon-web_1    | [13] *  Environment: production
mastodon-web_1    | [13] *   Master PID: 13
mastodon-web_1    | [13] *      Workers: 2
mastodon-web_1    | [13] *     Restarts: (✔) hot (✖) phased
mastodon-web_1    | [13] * Preloading application
mastodon-web_1    | [13] * Listening on http://0.0.0.0:3000
mastodon-web_1    | [13] Use Ctrl-C to stop
mastodon-web_1    | [13] - Worker 0 (PID: 16) booted in 0.01s, phase: 0
mastodon-web_1    | [13] - Worker 1 (PID: 19) booted in 0.0s, phase: 0

Unfortunately, I cannot connect to my Mastodon instance yet, because I did not publish any ports. But as initially mentioned, I want to put everything behind a proxy anyway, so I’m going to do that next.

Adding the Proxy

As proxy I’m going to use Caddy. I used Caddy before to automatically generate an SSL certificate for an Azure Container Instance as I described in an earlier blog post. Caddy comes in handy because it requires almost no configuration. All I need to do is to add the following configuration to my docker compose file:

version: '3.8'

services:
  # ... existing services configuration omitted ... #
  mastodon-proxy:
    image: 'caddy:latest'
    command: 'caddy reverse-proxy --from ${MASTODON_DOMAIN} --to mastodon-web:3000'
    ports:
      - '80:80'
      - '443:443'
    depends_on:
      - mastodon-web

volumes:
  # ... existing volumes configuration omitted ... #

With that I spin up a new container based on the caddy image and I tell it to map the domain name ${MASTODON_DOMAIN} (which I configured to point to social.raeffs.dev) to the Mastodon instance listening on the docker internal address mastodon-web:3000. Also I’m publishing the ports 80 and 443 to make the proxy accessible from the web.

With that done I can start all the containers again and access my own Mastodon instance:

The home page of my new Mastodon instance.

The next thing I want to do is to create an admin user. From the documentation I know that the only way to create an admin user is to use the Mastodon CLI from inside the docker container to do so. And because I don’t want to do any manual work, I’m going to extend my provisioning script to do so.

Creating an Admin User

To automatically create an admin user account when the Mastodon instance starts up, I’m extending my provisioning script as follows:

#!/bin/bash

echo "Migrating database..."
bundle exec rake db:migrate

CHECK=/mastodon/public/system/provisioned

if [ -f "$CHECK" ]; then
    echo "Provisioning not required"
else
    echo "Provisioning mastodon..."

    bin/tootctl accounts create $MASTODON_ADMIN_USERNAME --email $MASTODON_ADMIN_EMAIL --confirmed --role Owner

    echo "Provisioning done"
    touch "$CHECK"
fi

As you can see, I’m using a file to check whether the script was executed before, and if not I’m creating a new user with the username $MASTODON_ADMIN_USERNAME and the email $MASTODON_ADMIN_EMAIL. If I would not do this, the provisioning would fail if the user already exists and thus the container would not start. Of course I need to configure those values in the .env file:

MASTODON_ADMIN_USERNAME=foobar
MASTODON_ADMIN_EMAIL=foo@bar.ch

And map them in the docker compose file so that the container has access to the environment variables used in the script:

version: '3.8'

services:
  # ... existing services configuration omitted ... #
  mastodon-web:
    # ... existing configuration omitted ... #
    environment:
      # ... existing configuration omitted ... #
      MASTODON_ADMIN_USERNAME: '${MASTODON_ADMIN_USERNAME}'
      MASTODON_ADMIN_EMAIL: '${MASTODON_ADMIN_EMAIL}'

volumes:
  # ... existing volumes configuration omitted ... #

I also noticed that mastodon cannot access the filesystem inside the mastodon-volume that I created. To solve that I run docker run --rm -v mastodon-volume:/mastodon busybox /bin/sh -c 'chown -R 991:991 /mastodon', which starts a temporary docker container and sets the correct permission on the mounted mastodon-volume.

After that I can start the containers, and the password of the newly created admin user is printed to the console:

mastodon-web_1    | New password: bc4417b0953fb0b534fb87163218afd3
mastodon-web_1    | Provisioning done

If you missed the password, no problem. You can easily change it once the email delivery is set up, which is what I’m going to do next.

Configuring E-Mail Delivery

I initially tried to setup email delivery by simply configuring the SMTP settings mentioned in the documentation. But this didn’t work, because the emails are not sent by the web interface directly. Instead the web interface creates background jobs that take care of it. And this background jobs are processed by the background processing service called Sidekiq which is a part of the Mastodon docker image.

To startup Sidekiq alongside the other docker containers, I’m going to add an additional service to the docker compose file:

version: '3.8'

services:
  # ... existing services configuration omitted ... #
  mastodon-sidekiq:
    image: 'tootsuite/mastodon'
    command: 'bundle exec sidekiq'
    volumes:
      - 'mastodon-volume:/mastodon/public/system'
    environment:
      RAILS_ENV: 'production'
      LOCAL_DOMAIN: '${MASTODON_DOMAIN}'
      REDIS_HOST: 'mastodon-redis' # name of the redis container
      DB_HOST: 'mastodon-db' # name of the database container
      DB_NAME: '${MASTODON_POSTGRES_DATABASE}'
      DB_USER: '${MASTODON_POSTGRES_USERNAME}'
      DB_PASS: '${MASTODON_POSTGRES_PASSWORD}'
      SMTP_SERVER: '${SMTP_SERVER}'
      SMTP_PORT: '${SMTP_PORT}'
      SMTP_LOGIN: '${SMTP_LOGIN}'
      SMTP_PASSWORD: '${SMTP_PASSWORD}'
      SMTP_FROM_ADDRESS: '${SMTP_FROM_ADDRESS}'
    depends_on:
      - mastodon-db
      - mastodon-redis

volumes:
  # ... existing volumes configuration omitted ... #

As you can see the configuration of the newly added service mastodon-sidekiq is almost the same as the configuration of the service mastodon-web. The only differences are the command that now starts Sidekiq and the lack of the volume mount for the provisioning. And of course I added the additional environment variables for the SMTP settings, which I need to configure in the .env file:

SMTP_SERVER=MySmtpServer
SMTP_PORT=587
SMTP_LOGIN=MySmtpLogin
SMTP_PASSWORD=MySmtpPassword
SMTP_FROM_ADDRESS=foo@bar.ch

With that set up, I can finally startup all the containers, request a password change and log into my own Mastodon instance:

My first toot on my own Mastodon instance.

Conclusion

Everything is working. You check it yourself, and don’t forget to follow me! Well, at least it seems so. A look into the browsers dev tools shows a lot of errors regarding the streaming API that I did not set up yet. I thought that this is an optional part of Mastodon, but it seems not. So my Mastodon instance is usable for now, but the user experience is not that good because the user interface isn’t automatically updating.

I will add the streaming API as well as the optional Elasticsearch service another time. For know I leave you with the final docker compose configuration I used to spin up my own Mastodon instance:

version: '3.8'

services:
  mastodon-db:
    image: 'postgres:alpine'
    volumes:
      - 'mastodon-db-volume:/var/lib/postgresql/data'
    environment:
      POSTGRES_DB: '${MASTODON_POSTGRES_DATABASE}'
      POSTGRES_USER: '${MASTODON_POSTGRES_USERNAME}'
      POSTGRES_PASSWORD: '${MASTODON_POSTGRES_PASSWORD}'

  mastodon-redis:
    image: 'redis:alpine'

  mastodon-proxy:
    image: 'caddy:latest'
    command: 'caddy reverse-proxy --from ${MASTODON_DOMAIN} --to mastodon-web:3000'
    ports:
      - '80:80'
      - '443:443'
    depends_on:
      - mastodon-web

  mastodon-web:
    image: 'tootsuite/mastodon'
    command: 'bash -c "/provision.sh; rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"'
    volumes:
      - 'mastodon-volume:/mastodon/public/system'
      - './provision.sh:/provision.sh:ro'
    environment:
      RAILS_ENV: 'production'
      LOCAL_DOMAIN: '${MASTODON_DOMAIN}'
      REDIS_HOST: 'mastodon-redis' # name of the redis container
      DB_HOST: 'mastodon-db' # name of the database container
      DB_NAME: '${MASTODON_POSTGRES_DATABASE}'
      DB_USER: '${MASTODON_POSTGRES_USERNAME}'
      DB_PASS: '${MASTODON_POSTGRES_PASSWORD}'
      SECRET_KEY_BASE: '${MASTODON_SECRET_KEY_BASE}'
      OTP_SECRET: '${MASTODON_OTP_SECRET}'
      MASTODON_ADMIN_USERNAME: '${MASTODON_ADMIN_USERNAME}'
      MASTODON_ADMIN_EMAIL: '${MASTODON_ADMIN_EMAIL}'
    depends_on:
      - mastodon-db
      - mastodon-redis

  mastodon-sidekiq:
    image: 'tootsuite/mastodon'
    command: 'bundle exec sidekiq'
    volumes:
      - 'mastodon-volume:/mastodon/public/system'
    environment:
      RAILS_ENV: 'production'
      LOCAL_DOMAIN: '${MASTODON_DOMAIN}'
      REDIS_HOST: 'mastodon-redis' # name of the redis container
      DB_HOST: 'mastodon-db' # name of the database container
      DB_NAME: '${MASTODON_POSTGRES_DATABASE}'
      DB_USER: '${MASTODON_POSTGRES_USERNAME}'
      DB_PASS: '${MASTODON_POSTGRES_PASSWORD}'
      SECRET_KEY_BASE: '${MASTODON_SECRET_KEY_BASE}'
      OTP_SECRET: '${MASTODON_OTP_SECRET}'
      SMTP_SERVER: '${SMTP_SERVER}'
      SMTP_PORT: '${SMTP_PORT}'
      SMTP_LOGIN: '${SMTP_LOGIN}'
      SMTP_PASSWORD: '${SMTP_PASSWORD}'
      SMTP_FROM_ADDRESS: '${SMTP_FROM_ADDRESS}'
    depends_on:
      - mastodon-db
      - mastodon-redis

volumes:
  mastodon-volume:
    external: true
  mastodon-db-volume:
    external: true

As well as the .env file used to configure the environment variables:

MASTODON_DOMAIN=social.raeffs.dev
MASTODON_ADMIN_USERNAME=foobar
MASTODON_ADMIN_EMAIL=foo@bar.ch

MASTODON_POSTGRES_DATABASE=mastodon
MASTODON_POSTGRES_USERNAME=mastodon-user
MASTODON_POSTGRES_PASSWORD=MySuperSecretPassword

# random values used for the sessions and two factor authentication tokens
# generate with cryptographically secure mechanism, for example `openssl rand -base64 48`
MASTODON_SECRET_KEY_BASE=RandomValue
MASTODON_OTP_SECRET=AnotherRandomValue

SMTP_SERVER=MySmtpServer
SMTP_PORT=587
SMTP_LOGIN=MySmtpLogin
SMTP_PASSWORD=MySmtpPassword
SMTP_FROM_ADDRESS=foo@bar.ch

Also make sure you don’t forget to create the docker volumes and set the required permissions:

docker volume create mastodon-db-volume
docker volume create mastodon-volume
docker run --rm -v mastodon-volume:/mastodon busybox /bin/sh -c 'chown -R 991:991 /mastodon'

Update 03.12.2022

I capitalized the Admin role name in the provisioning script. With the latest major release of Mastodon 4.0.0 they added support for customizable roles and it looks like the default admin role is now called Admin and no longer admin. Thanks @alex@social.alexdobin.com for letting me know.

Update 02.01.2023

I updated the name of the role in the provisioning script again. It turns out the name of the super admin role since the introduction of customizable roles is Owner and not Admin. The Owner role by default has some more rights than the Admin role. For example it can view the Sidekiq dashboard.

I also added all the relevant files to a GitHub repository. There you can also find the extended configurations that I describe in Part 2 and Part 3.