Trying to Improve Docker Performance on macOS Using Mutagen

Testing if using Mutagen improves performance in comparison to NFS volume mounts. Comparing MacOS Docker performance with Linux.

TL;DR; Performance improvement by using Mutagen compared to NFS volume mounts: ~25%. Mutagen causes high CPU usage when using multiple syncs. Development experience not ideal due to delays until files are synced. Issues with intentional mass file changes being prevented by Mutagen safety mechanisms.

TL;DR; when using PHP / Drupal xDebug increased Drupal response times by about 5–6x. I’ve added an environment variable that allows disabling it on Docker run if I don’t need to debug.

Motivation

I’ve been using Docker for local development for some time now. The performance has been lacking, to say the least. Compared to other setups like using a VM provided by VirtualBox or running bare-metal directly on the machine. I investigated a bit and the culprit was volume mount performance (as expected). Running on Docker without volume mounts is multitudes faster.

However, Docker provides advantages as well. Especially if using Docker in production. Having the same - or a very similar - setup as in production can be very valuable.

Thus I looked for possible optimizations.

After reading some articles, I decided give Mutagen a try:

Mutagen setup

First off. What is Mutagen? What they say:

Mutagen provides real-time file synchronization and flexible network forwarding for developers, extending the reach of local development tools to cloud-based containers and infrastructure.

It allows to sync files from the host machine to Docker containers. An option to replace volume mounts.

Sounds great, no need for slow volume mounts. And still the same functionality and development workflow.

Using docker-compose

I first tried to configure it using Mutagen’s docker-compose feature.

Didn’t work though, I figured out afterward that I need to create all mount point directories in the Dockerfiles. And make the user that is running inside the container the owner of these directories(For example in the DB container the user with ID 1111). Otherwise, Mutagen cannot write to the mount points. Make sure the user’s home directory is also writable, see this issue for more info. Otherwise, it will fail to use the mutagen agent.

Here’s a simplified version, didn’t fix it though so it doesn’t work:

# Configuration for local development
version: "3.7"

services:
  # MySQL
  # -----
  db:
    image: bitnami/mysql:latest
    ports:
    - '127.0.0.1:3307:3306'
    env_file:
    - ./.env
    volumes:
    - db:/bitnami/mysql

  # PHP 7.4
  # -------
  drupal:
    image: php:7.4-fpm
    env_file:
    - ./.env
    depends_on:
    - db
    links:
    - db
    volumes:
    - appdata:/app/code
    - public-files:/app/public_files
    - private-files:/app/private_files

  # Nginx
  # -----
  nginx:
    image: nginx:latest
    env_file:
    - ./.env
    ports:
    - 80:8080
    - 443:8443
    depends_on:
    - drupal
    links:
    - drupal
    volumes:
    - appdata:/app/code
    - public-files:/app/public_files
    - private-files:/app/private_files

volumes:
  appdata:
  public-files:
  private-files:
  db:

x-mutagen:
  sync:
    defaults:
      mode: "two-way-resolved"
      symlink:
        mode: "posix-raw"
      configurationBeta:
        permissions:
          defaultOwner: "id:1111"
          defaultGroup: "id:1111"
          defaultFileMode: "0666"
          defaultDirectoryMode: "0666"
      ignore:
        vcs: true
    appdata:
      alpha: "."
      beta: "volume://appdata"
      ignore:
        paths:
          - "docker"
          - "frontend-build"
          - "Jenkins"
    public-files:
      alpha: "../public_files"
      beta: "volume://public-files"
    private-files:
      alpha: "../private_files"
      beta: "volume://private-files"
    db:
      alpha: "docker/db/mysql"
      beta: "volume://db"
      configurationBeta:
        permissions:
          defaultOwner: "id:1001"
          defaultGroup: "id:0"

Using Mutagen commands

I also ran into the issue of Mutagen preventing mass file changes because of its safety-mechanisms. Drupal has commands to export configuration, which writes a lot of files at once. So this is a problem for my case.

This made me scrap the docker-compose approach. So I used Mutagen’s shell commands to gain more flexibility:

#!/usr/bin/env bash

# Usage: See help
# $ bash mutagen.sh --help

COMPOSE_PROJECT_NAME=drupal

SYNC_APPDATA_SOURCE=.
SYNC_APPDATA=${COMPOSE_PROJECT_NAME}-appdata
SYNC_APPDATA_DRUPAL=${SYNC_APPDATA}-drupal
SYNC_APPDATA_NGINX=${SYNC_APPDATA}-nginx

SYNC_APPDATA_CONFIG_SOURCE=./config
SYNC_APPDATA_CONFIG_DRUPAL=${SYNC_APPDATA}-config-drupal

SYNC_PUBLIC_FILES_SOURCE=../public_files
SYNC_PUBLIC_FILES=${COMPOSE_PROJECT_NAME}-public-files
SYNC_PUBLIC_FILES_DRUPAL=${SYNC_PUBLIC_FILES}-drupal
SYNC_PUBLIC_FILES_NGINX=${SYNC_PUBLIC_FILES}-nginx

SYNC_PRIVATE_FLIES_SOURCE=../private_files
SYNC_PRIVATE_FLIES=${COMPOSE_PROJECT_NAME}-private-files
SYNC_PRIVATE_FILES_DRUPAL=${SYNC_PRIVATE_FLIES}-drupal
SYNC_PRIVATE_FILES_NGINX=${SYNC_PRIVATE_FLIES}-nginx

function terminate_syncs() {
  mutagen sync terminate --label-selector project="${COMPOSE_PROJECT_NAME}"
}

function pause_syncs() {
  mutagen sync pause --label-selector project="${COMPOSE_PROJECT_NAME}"
}

function resume_syncs() {
  mutagen sync resume --label-selector project="${COMPOSE_PROJECT_NAME}"
}

function flush_syncs() {
  mutagen sync flush --label-selector project="${COMPOSE_PROJECT_NAME}"
}

function list_syncs() {
  mutagen sync list --label-selector project="${COMPOSE_PROJECT_NAME}"
}

function syncs_exist() {
  if [[ -z $(list_syncs | grep 'No synchronization sessions found') ]]; then
    echo 1
  else
    echo 0
  fi
}

function monitor_syncs() {
  mutagen sync monitor --label-selector project="${COMPOSE_PROJECT_NAME}"
}

function do_containers_exist() {
  if [[ -z "$(docker-compose ps | grep "${COMPOSE_PROJECT_NAME}")" ]]; then
    echo 0
  else
    echo 1
  fi
}

function get_container_name() {
  local serviceName=${1}
  local existingContainerName=$(docker ps | grep ${COMPOSE_PROJECT_NAME}_${serviceName} | awk '{print $NF}')

  if [[ -n "${existingContainerName}" ]]; then
    echo "${existingContainerName}"
  else
    # Use default name schema assigned when a compose project is first run.
    echo ${COMPOSE_PROJECT_NAME}_${serviceName}_1
  fi
}

create_syncs_drupal() {
  local shouldOverwriteContainerData=${1}
  local containerToHostSyncMode
  if [[ "${shouldOverwriteContainerData}" == 1 ]]; then
    containerToHostSyncMode=one-way-safe
  else
    containerToHostSyncMode=two-way-resolved
  fi
  local containerName=drupal

  mutagen sync create \
    --name=${SYNC_APPDATA_DRUPAL} \
    --label project="${COMPOSE_PROJECT_NAME}" \
    --watch-polling-interval-alpha=10 \
    --watch-polling-interval-beta=120 \
    --default-owner-beta=drupal \
    --default-group-beta=drupal \
    --sync-mode=one-way-safe \
    --probe-mode=assume \
    --symlink-mode=posix-raw \
    --ignore-vcs \
    --ignore ./config,./docker,./frontend-build,./Jenkins,*.sql,*.tar.gz \
    ${SYNC_APPDATA_SOURCE} docker://drupal@$(get_container_name ${containerName})/app/code

    mutagen sync create \
    --name=${SYNC_APPDATA_CONFIG_DRUPAL} \
    --label project="${COMPOSE_PROJECT_NAME}" \
    --watch-polling-interval-alpha=30 \
    --watch-polling-interval-beta=30 \
    --default-owner-beta=drupal \
    --default-group-beta=drupal \
    --sync-mode="${containerToHostSyncMode}" \
    --probe-mode=assume \
    --symlink-mode=posix-raw \
    --ignore-vcs \
    ${SYNC_APPDATA_CONFIG_SOURCE} docker://drupal@$(get_container_name ${containerName})/app/code/config

  mutagen sync create \
    --name=${SYNC_PUBLIC_FILES_DRUPAL} \
    --label project="${COMPOSE_PROJECT_NAME}" \
    --watch-polling-interval=10 \
    --default-owner-beta=drupal \
    --default-group-beta=drupal \
    --sync-mode="${containerToHostSyncMode}" \
    --probe-mode=assume \
    --ignore-vcs \
    ${SYNC_PUBLIC_FILES_SOURCE} docker://drupal@$(get_container_name ${containerName})/app/public_files

  mutagen sync create \
    --name=${SYNC_PRIVATE_FILES_DRUPAL} \
    --label project="${COMPOSE_PROJECT_NAME}" \
    --watch-polling-interval=10 \
    --default-owner-beta=drupal \
    --default-group-beta=drupal \
    --sync-mode="${containerToHostSyncMode}" \
    --probe-mode=assume \
    --ignore-vcs \
    ${SYNC_PRIVATE_FLIES_SOURCE} docker://drupal@$(get_container_name ${containerName})/app/private_files
}

function create_syncs_nginx() {
  local containerName=nginx

  mutagen sync create \
    --name=${SYNC_APPDATA_NGINX} \
    --label project="${COMPOSE_PROJECT_NAME}" \
    --watch-polling-interval-alpha=30 \
    --watch-polling-interval-beta=120 \
    --default-owner-beta=drupal \
    --default-group-beta=drupal \
    --sync-mode=one-way-safe \
    --probe-mode=assume \
    --symlink-mode=posix-raw \
    --ignore-vcs \
    --ignore ./docker,./frontend-build,./Jenkins,./vendor,*.sql,*.tar.gz \
    ${SYNC_APPDATA_SOURCE} docker://drupal@$(get_container_name ${containerName})/app/code

  mutagen sync create \
    --name=${SYNC_PUBLIC_FILES_NGINX} \
    --label project="${COMPOSE_PROJECT_NAME}" \
    --watch-polling-interval=10 \
    --default-owner-beta=drupal \
    --default-group-beta=drupal \
    --sync-mode=one-way-safe \
    --probe-mode=assume \
    --ignore-vcs \
    ${SYNC_PUBLIC_FILES_SOURCE} docker://drupal@$(get_container_name ${containerName})/app/public_files

  mutagen sync create \
    --name=${SYNC_PRIVATE_FILES_NGINX} \
    --label project="${COMPOSE_PROJECT_NAME}" \
    --watch-polling-interval=10 \
    --default-owner-beta=drupal \
    --default-group-beta=drupal \
    --sync-mode=one-way-safe \
    --probe-mode=assume \
    --ignore-vcs \
    ${SYNC_PRIVATE_FLIES_SOURCE} docker://drupal@$(get_container_name ${containerName})/app/private_files
}

function create_syncs() {
  local shouldOverwriteContainerData=0
  if [[ $# -gt 0 ]]; then
    if [[ "$1" = 'overwite' ]]; then
      shouldOverwriteContainerData=1
    fi
  fi

  create_syncs_drupal ${shouldOverwriteContainerData}
  create_syncs_nginx
}

function create_overwrite_syncs() {
  create_syncs overwite
}

function overwrite_container_data() {
  create_overwrite_syncs
}

function stop() {
  printf '=> Pausing syncs\n'
  pause_syncs
}

function check_requirements() {
  if ! command -v mutagen &> /dev/null
  then
    printf 'Mutagen is not installed.\n'
    printf 'You can install it using Homebrew:\n'
    printf '$ brew install mutagen-io/mutagen/mutagen\n'
    printf 'Or using an alternative method, see: https://mutagen.io/documentation/introduction/installation\n'
    exit
  fi
}

check_requirements

if [[ $# -gt 0 ]]; then
  if [[ "$1" == 'up' ]]; then
    shift 1

    if [[ "${1}" == '--build' ]]; then
      printf '=> Building required Docker base images\n'
      build_required_docker_images
    fi

    if [[ $(do_containers_exist) == 0 ]]; then
      printf '=> No preexisting containers were found\n'
      printf '=> Starting Docker containers\n'
      docker-compose up -d "$@"
      printf '=> Overwriting container volumes with host data\n'
      overwrite_container_data
      flush_syncs
      terminate_syncs
    else
      printf '=> Starting Docker containers\n'
      docker-compose up -d "$@"
    fi

    if [[ $(syncs_exist) == 0 ]]; then
      printf '=> Creating volume syncs\n'
      create_syncs
    else
      printf '=> Resuming volume syncs\n'
      resume_syncs
    fi

    docker-compose up
  elif [[ "$1" == 'stop' ]]; then
    stop
  elif [[ "$1" == 'terminate-syncs' ]]; then
    terminate_syncs
  elif [[ "$1" == 'list-syncs' ]]; then
    list_syncs
  elif [[ "$1" == 'monitor-syncs' ]]; then
    monitor_syncs
  elif [[ "$1" == '--help' ]]; then
    printf 'Available commands:\n'
    printf '  up [options]\n'
    printf '    - Creates syncs and starts Docker containers\n'
    printf '      Checks if any preexisting containers exist.\n'
    printf '      If not it does a one-time sync from Host to containers that overwrites any data on the containers.\n'
    printf '    - Options:\n'
    printf '        All options that are available for the docker-compose command.\n'
    printf '        --build  Additionally builds all required base Docker images\n'
    printf '  stop\n'
    printf '    - Pauses syncs and stops the currently cunning Docker containers\n'
    printf '  terminate-syncs\n'
    printf '    - Terminates the syncs created for this project.\n'
    printf '  list-syncs\n'
    printf '    - Displays a list of the syncs created for this project.\n'
    printf '  monitor-syncs\n'
    printf '    - Monitors the syncs created for this project.\n'
  else
    printf 'Unknown command\n'
    printf 'Run "bash mutagen.sh --help" for usage\n'
  fi
fi

So now I have a working Mutagen setup.

Next step. See if it actually is any good.

Comparing MacOS + Mutagen vs Ubuntu + volume mounts

I thought volume mounts are fast on Linux systems. So let’s set one up to have a comparison. Using Ubuntu. And testing with a Drupal 9 project.

Response times are averages of 50 requests.

MacOS + NFS volume mounts

Average response time: 13.0558s This is the setup I was using for development & hoping to improve. As you can see, response times are not great 🙁.

Our reference system: Ubuntu + volume mounts

Average response time: 9.2684s I thought Docker on Linux was fast? Better than MacOS. But still, slow as a 🐢.

What’s going on here? 🤯🤔

MacOS + Mutagen

Average response time: 9.7394sBetter than the initial setup. Similar performance to Linux Docker. Still slow.

MacOS + NFS volume mounts, PHP xDebug extension disabled

Average response time: 1.6025s Aha! That’s the culprit.

Conclusion

Disable xDebug when you don’t need it. I added a flag via an environment variable in my case. Disabling it when I don’t need the debugger for the time being.

Mutagen's performance compared to using volume mounts was better in my case, using MacOS. I still decided to not use Mutagen. It had drawbacks that were not worth the marginal performance improvement to me:

The CPU usage was very high when using it. Often on 100%. (Using a MacBook Pro 2019, i7). Maybe this is because I used quite a few volumes.
The delay until the files are synced is a pain when developing. When I change some code, I want to be able to instantly see the result in my web browser. The polling intervals can be configured. It was not really an option for me to make them shorter. Since my CPU was already at its limit.

FYI: Docker for Mac is also adopting Mutagen. They first added it in version 2.3.10.

What are your experiences with Mutagen? Do you use other performance optimizations for your Docker development environment?

Trying to Improve Docker Performance on macOS Using Mutagen

Comments

More from this blog

How to move Terraform root module state inside other root module

Takeaways from SymfonyLive Online German Edition 2021

GitLab CI: Docker in Docker (DinD) on gitlab.com Shared Runners

GitLab CI: Security Scan using KICS

Create a private GCP Kubernetes cluster using Terraform

Motivation

Mutagen setup

Using docker-compose

Using Mutagen commands

Comparing MacOS + Mutagen vs Ubuntu + volume mounts

MacOS + NFS volume mounts

Our reference system: Ubuntu + volume mounts

MacOS + Mutagen

MacOS + NFS volume mounts, PHP xDebug extension disabled

Conclusion

Command Palette

Comments

More from this blog

Motivation

Mutagen setup

Using docker-compose

Using Mutagen commands

Comparing MacOS + Mutagen vs Ubuntu + volume mounts

MacOS + NFS volume mounts

Our reference system: Ubuntu + volume mounts

MacOS + Mutagen

MacOS + NFS volume mounts, PHP xDebug extension disabled

Conclusion