Commit faa81955 authored by Jürgen Haas's avatar Jürgen Haas

Merge branch 'release/1.9.0'

parents 74b30950 48066a67
......@@ -27,5 +27,7 @@ RUN echo "Install dorgflow" && \
./gitflow-installer.sh install stable && \
rm gitflow-installer.sh && \
\
sed -i -e "s/readlink -e/echo/g" /usr/local/bin/git-flow && \
\
rm -rf /var/cache/* && \
rm -rf /root/.composer/cache
......@@ -6,7 +6,7 @@ Provides fully configured Docker images for local Drupal development where your
Nothing else than docker being installed on your host. You don't need any other tool - none!
## Quick start
## Quick start Linux
You can install the tool with the following command:
......@@ -14,9 +14,40 @@ You can install the tool with the following command:
docker run -v /usr/local/bin:/setup --rm registry.lakedrops.com/docker/l3d/setup:latest
```
This will install the script `l3d` into the given directory `/usr/local/bin` from where it is executable everywhere on your host (assuming this path being in your search path). If you want that script to be installed elsewhere then provide that alternative path in the install command above. Also, instead of `latest` you can also start with any othe specific version number of this tool. You can find all available version string in the [tag list](https://gitlab.lakedrops.com/docker/l3d/tags) of this project.
This will install the script `l3d` into the given directory `/usr/local/bin` from where it is executable everywhere on your host (assuming this path being in your search path). If you want that script to be installed elsewhere then provide that alternative path in the install command above. Instead of `latest` you can also start with any other specific version number of this tool. You can find all available version strings in the [tag list](https://gitlab.lakedrops.com/docker/l3d/tags) of this project.
NOTE: depending on your system setup it may well be impossible for Docker to mount the example directory `/usr/local/bin` from above. We've seen this happening especially on MacOS. In such cases, please use a different directory and make sure that it's part of the PATH environment variable so that it remains convenient to execute that script from everywhere.
## Quick start MacOS
Here we have to circumwent some limitations that come with Docker for Mac:
1. Docker for Mac is not able to mount certain directories and one of those is anything underneath `/usr`. Therefore you have to use a different directory where the L3D script and some additional files get stored.
2. Docker for Mac does not properly mount sockets such that SSH agent can not be used inside of Docker containers the same way this is being done on Linux platforms. There are many workarounds available and we have integrated the approach from [mariusgrigaitis](https://github.com/mariusgrigaitis/docker-mac-ssh-auth-sock) into this project.
This requires [socat](https://linux.die.net/man/1/socat). You can install that with `brew install socat` or `sudo port install socat` or any other method that is applicable to you specific Mac. After installation make sure that `socat` can be found in your search path. If it got installed in a directory that isn't in the path, you can create a symbolic link to it into `/usr/local/bin`.
Then use the following command to install L3D (replace `myname` with your correct username:
```bash
docker run -v /User/myname/bin:/setup --rm registry.lakedrops.com/docker/l3d/setup:latest
```
Then make sure that the directory `/User/myname/bin` is included in the PATH environment variable or create a symbolic link in `/usr/local/bin`. If you work with a symbolic link, MacOS also requires [coreutils](https://de.wikipedia.org/wiki/GNU_Core_Utilities) which can be installed with `brew install coreutils` or `sudo port install coreutils`.
## Verifying SSH
As you will require SSH functionality in most development environments, let's make sure that this really works. In order to find out if your host is properly configured to use (and forward) SSH authentication, you should call `ssh-add -l` and see if it lists your public key(s). if it doesn't, just call `ssh-add` to add you public keys to the ssh agent. If this doesn't persist after a reboot of you host on MacOS, then please follow [these instructions](https://apple.stackexchange.com/questions/253779/macos-10-12-sierra-will-not-forget-my-ssh-keyfile-passphrase/254619#254619).
To test SSH easily, you can do something like this:
```shell script
$> ssh -T git@github.com
# Hi yourusername! You've successfully authenticated, but GitHub does not provide shell access.
$> ssh -T git@gitlab.com
# Welcome to GitLab, @yourusername!
```
The same test also works with other hosts where your public key is configured as your authentication method. Please note, you can repeat these SSH tests inside the L3D container later on as well and they should produce the very same results.
## Usage
......@@ -45,6 +76,12 @@ If you have chosen the LakeDrops Drupal 8 project template, type 'a d4d up' at t
Just call `l3d selfupdate` and the start script will be updated from the original repository. This will also update all your existing **L3D** images and remove the outdated containers so that they will be recreated the next timne you're going to use them.
## Resetting containers
Sometimes the mounted volumes into existing containers will change, e.g. after a reboot on a Mac, the directory where the SSH-Agent stores their information will have gotten a different name. In such situations, starting one of the existing containers will throw error messages to the console. To resolve this, all L3D-related existing containers need to be rebuilt.
You chould call `l3d reset` which cleans up all the related containers and prepares the system such that next call to L3D will rebuild those required containers from scratch. Note that removing containers will not remove their volumes. This is a good thing, because it makes sure that all your data will remain intact and will be used by the newly created containers just as before.
## Working with Git credentials
In order to commit and push with git you have to at least configure your email address and username for the git client. If you configure those globally on your host, each of the **L3D** containers will inherit those settings and there is nothing that needs to be done inside the container.
......@@ -56,6 +93,18 @@ git config --global user.email "you@example.com"
git config --global user.name "Your Name"
```
## Configuring the shell
By default, L3D tries to use the same shell inside the containers that you also use on your host. If that's not possible, it falls back to [FISH](https://fishshell.com).
If you want to overwrite the shell deliberatly, then define an environment variable on your host, like e.g.
```shell script
export L3DSHELL=/bin/bash
```
Note that only new containers will get affected by this setting.
## Getting help
Calling `l3d help` displays information about **L3D** and provides a link to further details.
......
......@@ -6,5 +6,5 @@ echo ""
echo "Version: ${VERSION}"
echo "Support: https://gitlab.lakedrops.com/docker/l3d"
echo ""
echo "Usage: l3d [ help | version | selfupdate | update | PROJECTNAME ]"
echo "Usage: l3d [ help | version | selfupdate | update | reset | PROJECTNAME ]"
echo ""
#!/bin/bash
echo "Reset L3D container"
if [[ -z ${L3D_FORCE_UPDATE} ]]; then
export L3D_FORCE_UPDATE=1
fi
export PHP_VERSION=7.0
/usr/local/bin/update
export PHP_VERSION=7.1
/usr/local/bin/update
export PHP_VERSION=7.2
/usr/local/bin/update
echo "Cleaning old container"
docker kill l3drun >/dev/null
......@@ -21,12 +21,5 @@ fi
echo "Self update to version ${NEWVERSION} succeeded!"
export VERSION=${NEWVERSION}
export PHP_VERSION=7.0
/usr/local/bin/update
export PHP_VERSION=7.1
/usr/local/bin/update
export PHP_VERSION=7.2
/usr/local/bin/update
echo "Cleaning old container"
docker kill l3drun >/dev/null
export L3D_FORCE_UPDATE=0
/usr/local/bin/reset
......@@ -23,16 +23,21 @@ function startContainer {
if [[ -n ${ID} ]]; then
docker start ${COMPOSE_PROJECT_NAME}_l3d
else
mkdir -p ${HOMEDIR}/.composer
touch ${HOMEDIR}/.composer/auth.json
touch ${HOMEDIR}/.gitconfig
docker run --name ${COMPOSE_PROJECT_NAME}_l3d -dt \
--hostname ${COMPOSE_PROJECT_NAME}-l3d \
--env L3DSHELL=${L3DSHELL} \
--env COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME} \
--env PHP_VERSION=${PHP_VERSION} \
--env SSH_AUTH_SOCK=/ssh-agent \
--env SSH_AUTH_SOCK=${SSHAUTHSOCK} \
--volume /var/run/docker.sock:/var/run/docker.sock \
--volume ${WORKDIR}:/drupal \
--volume ${SSHAUTH}:/ssh-agent \
--volume ${SSHAUTH}:${SSHAUTHSOCK} \
--volume ${HOMEDIR}/.traefik:/root/.traefik \
--volume ${HOMEDIR}/.gitconfig:/root/.gitconfig \
--volume ${HOMEDIR}/.composer/auth.json:/root/.composer/auth.json \
--workdir /drupal \
--restart unless-stopped \
registry.lakedrops.com/docker/l3d/php-${PHP_VERSION}:${VERSION}
......
#!/bin/bash
if [[ -n ${PHP_VERSION} ]]; then
IMAGEID=$(docker image ls -q registry.lakedrops.com/docker/l3d/php-${PHP_VERSION} | head -1)
if [[ ! -n ${IMAGEID} ]]; then
exit
fi
echo "Updating the image ..."
STATUS=$(docker pull registry.lakedrops.com/docker/l3d/php-${PHP_VERSION}:${VERSION})
if [[ "$STATUS" == *"Status: Image is up to date"* ]]; then
echo "Already up to date"
function removecontainer {
if [[ "$1" == "selective" ]]; then
IMAGEID=$(docker image ls -q registry.lakedrops.com/docker/l3d/$2 | grep -v ${NEWIMAGEID})
else
echo "Image updated"
NEWIMAGEID=$(docker image ls -q registry.lakedrops.com/docker/l3d/php-${PHP_VERSION}:${VERSION} | head -1)
IMAGEID=$(docker image ls -q registry.lakedrops.com/docker/l3d/$2)
fi
if [[ -n ${IMAGEID} ]]; then
while true; do
IMAGEID=$(docker image ls -q registry.lakedrops.com/docker/l3d/php-${PHP_VERSION} | grep -v ${NEWIMAGEID})
if [[ -n ${IMAGEID} ]]; then
while true; do
ID=$(docker container ls --all -q -f ancestor=${IMAGEID})
if [[ -n ${ID} ]]; then
echo "Removing outdated container ..."
docker kill ${ID} >/dev/null
docker rm ${ID} >/dev/null
else
break
fi
done
echo "Removing outdated image ..."
docker rmi ${IMAGEID} >/dev/null
ID=$(docker container ls --all -q -f ancestor=${IMAGEID})
if [[ -n ${ID} ]]; then
echo "Removing outdated container ..."
docker kill ${ID} >/dev/null
docker rm ${ID} >/dev/null
else
break
fi
done
IMAGEID=$(docker image ls -q registry.lakedrops.com/docker/node:8-jessie-slim)
if [[ -n ${IMAGEID} ]]; then
while true; do
ID=$(docker container ls --all -q -f ancestor=${IMAGEID})
if [[ -n ${ID} ]]; then
echo "Removing outdated node container ..."
docker kill ${ID} >/dev/null
docker rm ${ID} >/dev/null
else
break
fi
done
if [[ "$1" == "selective" ]]; then
echo "Removing outdated image ..."
docker rmi ${IMAGEID} >/dev/null
fi
fi
}
function cleanup {
if [[ $L3D_FORCE_UPDATE -eq 1 ]]; then
removecontainer all php-${PHP_VERSION}
else
NEWIMAGEID=$(docker image ls -q registry.lakedrops.com/docker/l3d/php-${PHP_VERSION}:${VERSION} | head -1)
removecontainer selective php-${PHP_VERSION}
fi
removecontainer all node:8-jessie-slim
}
if [[ -n ${PHP_VERSION} ]]; then
if [[ $L3D_FORCE_UPDATE -eq 1 ]]; then
echo "Force update"
cleanup
else
IMAGEID=$(docker image ls -q registry.lakedrops.com/docker/l3d/php-${PHP_VERSION} | head -1)
if [[ -z ${IMAGEID} ]]; then
exit
fi
echo "Updating the image ..."
STATUS=$(docker pull registry.lakedrops.com/docker/l3d/php-${PHP_VERSION}:${VERSION})
if [[ "$STATUS" == *"Status: Image is up to date"* ]]; then
echo "Already up to date"
else
echo "Image updated"
cleanup
fi
fi
fi
#!/bin/bash
SSHAUTH=${SSH_AUTH_SOCK}
SSHAUTHSOCK=/ssh-agent
L3DHOSTOS="$(uname -s)"
case "${L3DHOSTOS}" in
Darwin*)
if ! which greadlink >/dev/null; then
# This only works if no symlinks are involved.
SCRIPTPATH="$(cd "$(dirname "$0")" && pwd -P)"
else
# This works on MacOS when coreutils is installed.
SCRIPTPATH="$(dirname "$(greadlink -f "$0")")"
fi
${SCRIPTPATH}/prepareMac4L3d
if [[ $? > 0 ]]; then
exit $?
fi
;;
*)
# This works in most cases, i.e. on Linux.
SCRIPTPATH="$(dirname "$(readlink -f "$0")")"
esac
ID=$(docker container ls --all -q -f name=^l3drun$)
if [[ ! -n ${ID} ]]; then
SCRIPTPATH="$( cd "$(dirname "$0")" ; pwd -P )"
docker run --name=l3drun -dt --rm \
--env SCRIPTPATH=${SCRIPTPATH} \
--volume /var/run/docker.sock:/var/run/docker.sock \
--volume ${SCRIPTPATH}:/setup \
registry.lakedrops.com/docker/l3d/run:'{{ version }}'
fi
if [[ -n ${L3DSHELL} ]]; then
# Nothing to do, we go with this value
L3DSHELL=${L3DSHELL}
elif [[ -n ${SHELL} ]]; then
L3DSHELL=${SHELL}
else
L3DSHELL="/usr/bin/fish"
fi
if [[ -f ".env" ]]; then
export $(cat .env | xargs) > /dev/null 2>&1
fi
docker exec -it \
--env HOMEDIR=${HOME} \
--env L3DSHELL=${L3DSHELL} \
--env WORKDIR=${PWD} \
--env SSHAUTH=${SSH_AUTH_SOCK} \
--env SSHAUTH=${SSHAUTH} \
--env SSHAUTHSOCK=${SSHAUTHSOCK} \
--env PHP_VERSION=${PHP_VERSION} \
--env COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME} \
l3drun \
......
#!/usr/bin/env bash
# Credit: https://github.com/mariusgrigaitis/docker-mac-ssh-auth-sock
SOCATID=$(docker container ls --all -q -f name=l3d_socat)
if [[ -n ${SOCATID} ]]; then
exit 0
fi
if ! which socat >/dev/null; then
echo "socat is missing. Install it and make sure the executable is in the local search path."
echo "For help see https://stackoverflow.com/questions/16808543/install-socat-on-mac"
exit 1
fi
if ! docker ps >/dev/null; then
echo "Docker for Mac is not running. Make sure it's running."
exit 1
fi
if [[ -z "${SSH_AUTH_SOCK}" ]]; then
echo "SSH_AUTH_SOCK is missing. Is ssh-agent running?"
exit 1
fi
if ! test -S ${SSH_AUTH_SOCK}; then
echo "$SSH_AUTH_SOCK is not a socket. Check agent?"
exit 1
fi
TTY_FILE=~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
TTY_FILE_NEW=~/Library/Containers/com.docker.docker/Data/vms/0/tty
if ! test -c $TTY_FILE; then
echo "$TTY_FILE is not available. Docker for Mac setup has changed? Trying newer file..."
if ! test -c $TTY_FILE_NEW; then
echo "$TTY_FILE_NEW is not available. Docker for Mac setup has changed? Giving up."
exit 1
else
TTY_FILE=$TTY_FILE_NEW
fi
fi
# This is where the UGLY hack starts
#
# Problem: if you do: docker run -v $SSH_AUTH_SOCK:$SSH_AUTH_SOCK container
# you get a socket file which is mounted over osxfs from Mac host.
# This socket file can't be reused or removed because it would make ssh commands on
# host machine to not work
#
# Solution:
# 1. connect to VM over special tty channel
# 2. create an empty directory
# 3. bind mount that empty directory over $SSH_AUTH_SOCK directory
# 4. Profit
#
# This makes other docker containers see the created directory instead of osxfs mounted one.
# It also allows to create socket file under same path that does not collide with host one.
# Command is sent over special tty channel to Docker for Mac VM and does not check for errors, etc
# meaning it could be very "unreliable"
COMMAND="mkdir -p /ssh-auth-sock-hack && mount -o bind /ssh-auth-sock-hack $(dirname $SSH_AUTH_SOCK) && rmdir $SSH_AUTH_SOCK"
echo ctr -n services.linuxkit tasks exec --exec-id 'ssh-$(hostname)-$$' '$(ctr -n services.linuxkit tasks ls -q | grep docker)' sh -c \"$COMMAND\" > $TTY_FILE
# give some time for command to execute.
sleep 1
echo "Docker for Mac is now prepared."
echo "Starting socket proxy."
# This is where the proxying magic happens
# On host machine it connects to $SSH_AUTH_SOCK socket and pipes output to stdout, takes input from stdin
# On docker VM it launches a container running socat, which creates a socket file under $SSH_AUTH_SOCK path, accepts
# input and forwards it to stdout/stdin
# socat running on host machine connects stdin/stdout between those two sockets can communicate over stdin/stdout
#
# This is not really reliable because forwarding input/output over stdin/stdout does not allow for multiple communications
# at the same time. It fails when doing multiple connections to $SSH_AUTH_SOCK at the same time.
exec socat "EXEC:\"docker run -i --rm --name l3d_socat -v $(dirname $SSH_AUTH_SOCK):$(dirname $SSH_AUTH_SOCK) alpine/socat UNIX-LISTEN:$SSH_AUTH_SOCK,reuseaddr,fork -\"" "EXEC:\"socat - UNIX:${SSH_AUTH_SOCK}\"" &
echo "Waiting ..."
sleep 2
SOCATID=$(docker container ls --all -q -f name=l3d_socat)
if [[ -n ${SOCATID} ]]; then
echo "All set!"
exit 0
fi
exit 1
#!/bin/sh
cp /usr/local/bin/l3d /setup/l3d
cp /usr/local/bin/prepareMac4L3d /setup/prepareMac4L3d
echo "L3D installed successfully!"
echo ""
......
#!/bin/bash
function truncateCurrentDirectory {
function readEnv {
if [[ -f ".env" ]]; then
# shellcheck disable=SC2046
# shellcheck disable=SC2002
export $(cat .env | xargs) > /dev/null 2>&1
fi
}
function truncateCurrentDirectory {
readEnv
rm .* > /dev/null 2>&1
}
......@@ -15,30 +21,99 @@ function restoreEnvFile {
echo "COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME}" >>.env
fi
if [[ -f ".env" ]]; then
# shellcheck disable=SC2046
# shellcheck disable=SC2094
# shellcheck disable=SC2002
env -i $(cat .env | xargs) >.env
fi
}
function create {
if [[ -f ".init_site.json" ]]; then
mv .init_site.json /tmp/init_site.json
fi
truncateCurrentDirectory
composer create-project ${PROJECT} . --no-interaction
if [[ -n ${REPOSITORY} ]]; then
composer create-project "${PROJECT}" . --no-interaction --repository "${REPOSITORY}"
else
composer create-project "${PROJECT}" . --no-interaction
fi
restoreEnvFile
if [[ -f "docker-compose.yml" ]]; then
if [[ -f "web/profiles/contrib/config_installer/config_installer.info.yml" ]]; then
# Start container
a d4d up
sleep 2
drush --no-interaction si config_installer
# Init site config values
if [[ -f "drush/Commands/dev_modules/dev_modules.info.yml" ]]; then
if [[ -f "/tmp/init_site.json" ]]; then
mv /tmp/init_site.json web/.init_site.json
drush site:init .init_site.json
rm web/.init_site.json
fi
fi
# Dump database
if [[ $L3D_DUMP_DB -eq 1 ]]; then
git ignore "/*.sql"
drush sql:dump --result-file ../db.sql
fi
drush -y cex
# shellcheck disable=SC2035
git add *
git add .*
git commit -am "After site install"
# Push to remote git repository
if [[ -n ${L3D_GIT_REMOTE} ]]; then
git remote add origin "$L3D_GIT_REMOTE"
git pull
git push
fi
# Cleanup
if [[ $L3D_CLEANUP -eq 1 ]]; then
docker-compose stop
docker-compose rm --force
fi
fi
fi
if [[ $L3D_EXIT -eq 1 ]]; then
exit
fi
}
EXISTING=$(ls -1)
if [[ ! -n "$EXISTING" ]]; then
function clone {
truncateCurrentDirectory
git clone "${REPOSITORY}" .
composer update
restoreEnvFile
}
function initialSetup {
echo "Lets start a new project here ..."
echo ""
echo "Options to start:"
echo " 0 none: start with an empty container"
echo " 1 LakeDrops Drupal 8 project template"
echo " 2 Drupal's community project template"
echo " 3 Existing git repository"
echo " 3 Custom project template"
echo " 4 Existing git repository"
echo ""
echo ""
while true; do
# shellcheck disable=SC2162
read -p "Choose an option: " OPTION
case ${OPTION} in
0 )
restoreEnvFile
break
;;
1 )
PROJECT="lakedrops/d8-project"
......@@ -53,12 +128,19 @@ if [[ ! -n "$EXISTING" ]]; then
;;
3 )
# shellcheck disable=SC2162
read -p "Custom project template: " PROJECT
# shellcheck disable=SC2162
read -p "Repository URL (optional): " REPOSITORY
create
break
;;
4 )
# shellcheck disable=SC2162
read -p "Repository URL: " REPOSITORY
if [[ -n "${REPOSITORY}" ]]; then
truncateCurrentDirectory
git clone ${REPOSITORY} .
composer update
restoreEnvFile
clone
break
fi
;;
......@@ -68,7 +150,22 @@ if [[ ! -n "$EXISTING" ]]; then
;;
esac
done
}
readEnv
EXISTING=$(ls -1)
if [[ -z "$EXISTING" ]]; then
if [[ -n ${PROJECT} ]]; then
create
elif [[ -n ${REPOSITORY} ]]; then
clone
else
initialSetup
fi
fi
restoreEnvFile
/usr/bin/fish
if [[ -x "${L3DSHELL}" ]]; then
${L3DSHELL}
else
/usr/bin/fish
fi
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment