Docksal. Our way to unified development environment

Aug 28, 2017

 

Development environments infinite variety

path

My personal path to docksal was pretty long. From the first PHP trial, I was looking for the perfect development environment. I’ve started with using Denwer on Windows. Six years ago it was an extremely popular stack BTW. Then I moved to WampServer, later to XAMPP, spent few weeks on Acquia Dev Desktop. My latest environment was usual LAMP stack, manually installed with a HomeBrew on my Mac. More or less, everybody in our team followed the same path.

 

Unification. Why?

When projects are small enough and everything goes smoothly, developers are very suspicious of standardization attempts. Everybody likes to customize the stuff. And in case of emergency, you simply don’t have time to think about standardization, you just need to get rid of an issue and keep the machine running. Though, have you ever seen the eyes of front-end developer installing XDebug? Or, maybe you’ve heard the painful moan of the junior developer who tries to have few versions of MySQL or PHP running on the same machine (Windows especially)? Few times we tried Vagrant, which was meant to solve the problem. But, personally, I don’t like the idea of using separate virtual machine per each project.

 

Docksal Introduction

Docksal FFW

From the official site:

“Docksal is a tool for defining and managing development environments. It brings together common tools, minimizes configuration, and ensures environment consistency throughout your continuous integration workflow.”

It’s an open source tool mainly supported by guys from FFW. It totally based on bash, docker, and VirtualBox (in case of Windows or macOS). Initially, it was called Drude, but due to some trade marks conflicts, it was renamed to “Docksal”. To be precise, docksal is only a “fin” bash script with about 5K lines of code. It contains a list of commonly used commands and simplifies management of docker environment.

Docksalized projects contain .docksal directory in the root of the project. Usually, there are few files inside of it. 

  • docksal.env – some environment variables, like the virtual host, DB name, docker images for web, DB, and CLI containers, etc.
  • docksal.yml – docksal related config. Probably, it’s very important, but I’ve never changed it.
  • commands/init – (optional) initialization bash script, which is used when you start working with the project. It performs some preconfigured tasks in addition to containers deployment
  • etc/php/php.ini, etc/mysql/my.cnf – PHP, MySQL config files, project related. These files are included in GIT repo, so you can be sure that everyone is using the same settings. Which is crucial for big projects.

 

Docksal Commands

Docksal ships with a bunch of built-in commands. I won’t list everything, but a few most useful.

  • start/stop/restart/status/reset – no comment;
  • logs/config – easy way to access the project logs (Apache, MySQL, etc.) or config;
  • db [import, dump, list] – DB utilities, I think names are self-descriptive;
  • project [list, create] – allows to create and manage docksal projects. Makes it easier to kick-start the new project;
  • vm [start, stop, etc.] – (Windows/macOS only) – wrapper for VirtualBox vm management;
  • bash [cli, db, web] – logs you into the shell of the given container;
  • drush/drupal/wp – wrapped CLI tools for Drupal and WP.

Also, the good thing about docksal, that it already has a nice list of integrations. XDdebug, Memcached, MailHog, etc. (you might check the list on the official site).

My personal love is ngrok integration. You simply run the “fin share” command and you can share the link to your local site. This feature may save front-end developer from a suicide.

 

Docksal Pros

  • Simple enough to get it launched by a non-backend guy;
  • Portability of server config;
  • Extendibility of commands;
  • Separated environments;
  • It’s easy enough to set up CI deployments on the dev server;
  • Contains a handful of useful integrations;
  • Simple bash syntax. You can extend it to fit your project needs.

 

Docksal Cons

  • You pay with performance for flexibility (as usual, though);
  • Windows / macOS users pay even more for flexibility (they need VirtualBox running);
  • In case of any issues, developers need to understand the basic Docksal architecture.

Some docksal commands snippets:

Finally, I wanted to provide you our init command as an example, just to make it more clear how the stuff is working. Our init file on the Boardr.in project contains the following steps:

  • import_db;
  • composer_install;
  • drush_cr;
  • drush_updb;
  • drush_config_import;
  • drush_cr.
#!/usr/bin/env bash

## Initialize Docksal powered Drupal 8 site
##
## Usage: fin init

# Abort if anything fails
set -e

#-------------------------- Settings --------------------------------

# PROJECT_ROOT is passed from fin.
# The following variables are configured in the '.env' file: DOCROOT, SITE_DIRECTORY, SOURCE_ALIAS and others.

DOCROOT_PATH="${PROJECT_ROOT}/${DOCROOT}"
SITEDIR_PATH="${DOCROOT_PATH}/sites/${SITE_DIRECTORY}"

#-------------------------- END: Settings --------------------------------

#-------------------------- Helper functions --------------------------------

# Console colors
red='\033[0;31m'
green='\033[0;32m'
green_bg='\033[42m'
yellow='\033[1;33m'
NC='\033[0m'

echo-red () { echo -e "${red}$1${NC}"; }
echo-green () { echo -e "${green}$1${NC}"; }
echo-green-bg () { echo -e "${green_bg}$1${NC}"; }
echo-yellow () { echo -e "${yellow}$1${NC}"; }

if_failed ()
{
	if [ ! $? -eq 0 ]; then
		if [[ "$1" == "" ]]; then msg="an error occurred"; else msg="$1"; fi
		echo-red "$msg";
		exit 1;
	fi
}

is_windows ()
{
	local res=$(uname | grep 'CYGWIN_NT')
	if [[ "$res" != "" ]]; then
		return 0
	else
		return 1
	fi
}

# Copy a settings file.
# Skips if the destination file already exists.
# @param $1 source file
# @param $2 destination file
copy_settings_file()
{
	local source="$1"
	local dest="$2"

	if [[ ! -f $dest ]]; then
		echo "Copying ${dest}..."
		cp $source $dest
	else
		echo-yellow "${dest} already in place."
	fi
}

#-------------------------- END: Helper functions --------------------------------

#-------------------------- Functions --------------------------------

# Initialize local settings files
init_settings ()
{
	# Copy from settings templates
	copy_settings_file "${SITEDIR_PATH}/default.settings.local.php" "${SITEDIR_PATH}/settings.local.php"
}

# Set file/folder permissions
file_permissions ()
{
	echo-green "Resetting files directory permissions..."
	mkdir -p "${SITEDIR_PATH}/files"
	chmod -R 777 "${SITEDIR_PATH}/files"
}

# Install site
import_db ()
{
	if [[ -f "${PROJECT_ROOT}/db/rbrd.sql" ]]; then
		fin sqli "${PROJECT_ROOT}/db/rbrd.sql"
	else
		echo-red "db/rbrd.sql wasn't found. Skip database importing."
	fi

	# Revert site dir permissions.
	chmod 755 "${SITEDIR_PATH}"
}

composer_install ()
{
    cd ${DOCROOT_PATH}
    fin exec composer install
    cd ${PROJECT_ROOT}
}

drush_cr ()
{
    cd ${DOCROOT_PATH}
    fin drush cr
    cd ${PROJECT_ROOT}
}

drush_updb ()
{
    cd ${DOCROOT_PATH}
    fin drush updb -y
    cd ${PROJECT_ROOT}
}

drush_config_import ()
{
    cd ${DOCROOT_PATH}
    fin drush config-import -y
    cd ${PROJECT_ROOT}
}

#-------------------------- END: Functions --------------------------------

#-------------------------- Execution --------------------------------

if [[ "$PROJECT_ROOT" == "" ]]; then
	echo-red "\$PROJECT_ROOT is not set"
	exit 1
fi

#echo-green "Setting file/folder permissions..."
#file_permissions

echo -e "${green_bg} Step 1 ${NC}${green} Initializing local project configuration...${NC}"
init_settings


if [[ $DOCKER_RUNNING == "true" ]]; then
	echo -e "${green_bg} Step 2 ${NC}${green} Recreating services...${NC}"
	fin reset -f
else
	echo -e "${green_bg} Step 2 ${NC}${green} Creating services...${NC}"
	fin up
fi

echo "Waiting for MySQL to start...";
RETRIES=10
for i in $(seq 1 $RETRIES); do
	echo-yellow "Try $i. Trying to connect to MySQL server every 3 seconds..."
	sleep 3

	# Check socket.
	STAT_SOCKET=$(fin docker exec ${COMPOSE_PROJECT_NAME_SAFE}_db_1 stat --format="%F" /var/run/mysqld/mysqld.sock 2>&1) || true
	if [[ $STAT_SOCKET != *"socket"* ]]; then
		if [[ $i == "$RETRIES" ]]; then
			echo-red "Error message: $STAT_SOCKET"
			echo-red "Can't find socket of mysql server. Exit."
			exit 1
		fi
	fi

	# Check ping.
	RESULT=$(fin docker exec "${COMPOSE_PROJECT_NAME_SAFE}_db_1" mysqladmin ping --verbose --no-beep --user="root" --password="${MYSQL_ROOT_PASSWORD:-root}" 2>&1) || true
	STATUS="1"
	if [[ $RESULT == *"alive"* ]]; then
		STATUS="0"
	fi
	if [[ $STATUS == "1" ]]; then
		if [[ $i == "$RETRIES" ]]; then
			echo-red "Error message: $RESULT"
			echo-red "Can't connect to mysql server. Exit."
			exit 1
		fi
	else
		break
	fi
done

echo -e "${green_bg} Step 3 ${NC}${green} Importing database...${NC}"
import_db

echo -e "${green_bg} Step 4 ${NC}${green} Composer install...${NC}"
composer_install

echo -e "${green_bg} Step 5 ${NC}${green} Rebuilding cache...${NC}"
drush_cr

echo -e "${green_bg} Step 6 ${NC}${green} Apply Drupal updates...${NC}"
drush_updb

echo -e "${green_bg} Step 7 ${NC}${green} Importing config...${NC}"
drush_config_import

echo -e "${green_bg} Step 8 ${NC}${green} Rebuilding cache...${NC}"
drush_cr

if is_windows; then
	echo-green "Add ${VIRTUAL_HOST} to your hosts file (/etc/hosts), e.g.:"
	echo-green "192.168.64.100  ${VIRTUAL_HOST}"
	echo
fi

echo -en "${green_bg} DONE! ${NC} "
echo -e "Open ${yellow}http://${VIRTUAL_HOST}${NC} in your browser to verify the setup."

#-------------------------- END: Execution --------------------------------

The import_db step is better to be replaced by the profile installation step. However, this project does not have the profile, so we just importing the DB. Also, this snippet assumes that the dump is in the DB folder. So, you need include DB file in the repo (obviously, a bad practice), otherwise, you need to ensure putting the file in the proper folder prior to running the init script (bad way of project simplification).

Luckily, on DrupalCamp Kyiv, Taras Tsiuper put a great idea in my head. In his presentation, he mentioned that their init script pulls the database dump from the production server. So, we replaced database import by a short function, which downloads the latest DB dump from the backups server. I won't put this snippet here, anyhow it's different for each project. 

 

Docksal and dev deployments

Unfortunately, not all of our projects use CI in the process. However, it’s pretty easy to setup the build process using the GitLab / BitBucket pipelines. In a few words, you just need to ensure your server has a “fin” command and run the “fin init” command in the root folder of the deployed project.

For instance, below is .gitlab-ci.yml from our project (created by Mikhail Molchanov):

image: michaeltigr/pipelines-agent:latest


staging:
    variables:
        REPO_SSH: "git@gitlab.com"
        CI_GIT_USER_EMAIL: "michael.molchanov@zanzarra.com"
        CI_GIT_USER_NAME: "Michael Molchanov"
    type: deploy
    script:
        - export BRANCH="$CI_BUILD_REF_NAME"
        - export REPO_OWNER="$CI_PROJECT_NAMESPACE"
        - export REPO_SLUG="$(echo -n $CI_PROJECT_NAME | sed -e 's/[^A-Za-z0-9]/-/g' | awk '{print tolower($0)}')"
        - export COMMIT="$CI_BUILD_REF"
        # Initialize the agent configuration
        - source build-env
        # Initialize the remote sandbox environment
        - build-init
        # Disable xdebug and make project permanent.
        - ssh docker-host "cd $REMOTE_BUILD_DIR && echo XDEBUG_ENABLED=0 | tee -a .docksal/docksal-local.env"
        - ssh docker-host "cd $REMOTE_BUILD_DIR && echo DOCKSAL_PERMANENT=true | tee -a .docksal/docksal-local.env"
        - ssh docker-host "cd $REMOTE_BUILD_DIR && echo -e VIRTUAL_HOST='\${VIRTUAL_HOST},boardr.in' | tee -a .docksal/docksal-local.env"
        # Run fin init on the remote docker host to provision a sandbox
        - ssh docker-host "cd $REMOTE_BUILD_DIR && fin init"
    only:
        - master

Obviously, it won’t be a big deal to adopt any dev build system to play well with docksal init process.

 

Sort of conclusion

finish

School teachers always say that all texts need to be finished with a conclusion. Here is mine.

Docksal is not a cure for all diseases and obviously, it’s not a start of Vagrant / Docksal holy war. The main point of my blog post is to show our path to choosing Docksal as a unified development environment. I’m almost sure that it’s not a lifetime choice. It may be that next year, I’ll prepare another post about new Super Ultimate Development Environment or even write my own bicycle instead. But for now, I am acknowledging that Docksal is a good solution for us and want to wish guys from FFW to not lose their path and make the tool even better than it is right now.

Cheers.

 

Resources