Skip to content
Snippets Groups Projects

Develop

Merged Develop
develop into main
Merged danielspeicher requested to merge develop into main
Viewing commit 9625fe9e
Prev Next
Show latest version
1 file
+ 26
− 0
Compare changes
  • Side-by-side
  • Inline
docs/gitlab/drupal/index.md
+ 362
− 29
@@ -12,18 +12,373 @@ tags:
This project contains a number of pre-configured tasks for GitLab CI/CD that
allow to run very powerful pipelines which contain the following stages:
- validation
- .pre (predefined)
- validate
- execute
- build (predefined)
- buildprod
- build
- prepare
- test
- deploy
- test (predefined)
- deploy (predefined)
- release
- finalize
- postprocess
- .post (predefined)
We have a predefined `execute` job called `LakeDrops`, which looks like this:
- execute
- LakeDrops
- buildprod
- Build Prod Site
- Build Prod Site NG
- build
- Build Site
- Build Theme
- Check 4 Outdated Packages
- Check 4 Security Vulnerability Advisories
- Download DB
- Download DB NG
- prepare
- Import DB
- Import DB NG
- Import Local DB
- Update DB
- test
- Test Code Style
- Test PHPUnit
- Test Backstop
- Test Cypress E2E
- finalize
- Shut Down Docker Project
These pre-configured tasks can easily be included into your Drupal project.
A GitLab runner on any host can then be configured to run those pipelines. The
following chapters describe the necessary steps and also all the options
available for configuration.
## Stages
The following chapters describes all stages and their jobs. The files:
- [lakedrops.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/lakedrops.yml?ref_type=heads).
- [test_deploy.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/test-and-deploy.yml?ref_type=heads).
### .pre
To prepare and check, if the environment is good to go, we have a `.pre` stage, which contains the
following jobs:
#### Sanity Checks
The sanity check gets important when you are using [Crowdsec](https://www.crowdsec.net/).
It contains two little scripts, that check:
- if the `crowdsec.settings.yml` is in the config split and
- if it is not enabled in the `core.extensions.yml`.
Otherwise, it is conflicting with you local settings, since `CrowdSec`
is a tool for production sites.
Except:
- Commit message contains "Merge tag"
- Commit message contains "Merge branch"
- When the job is triggered by config auto export (CAE)
#### Validate Environment
This job extends the `.prerequisites` in
[test_deploy.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/test-and-deploy.yml?ref_type=heads).
This job executes a script, where several [variables](#variables) are checked and set accordingly.
Except:
- Commit message contains "Merge tag"
- Commit message contains "Merge branch"
- When the job is triggered by config auto export (CAE)
### Validate
This stage is currently not officially used.
### Execute
This stage executes the build itself.
### Build Prod
This stage is responsible for building the production website, which we want to finally deploy.
#### Build Prod Site
First we have to do some preparation. This is done in the `before_script` section.
- [.prepareaccess](#prepareaccess)
- [.preparecomposerplugins](#preparecomposerplugins)
We start a docker image for php with the specifies versions:
- `PHP_MAJOR_VERSION`
- `PHP_MINOR_VERSION`
In the script section we first update the environment by executing `/usr/local/bin/update-env`.
After that we install the production site with `composer install` **without** the tools for development. That is pretty
it. We also add `drupal/core-vendor-hardening`.
The entire installation is saved as a build artefact called `build-prod` where all `git` files are excluded.
#### Build Prod Site NG
First we have to do some preparation. This is done in the `before_script` section.
- [.prepareaccess](#prepareaccess)
- [.preparecomposerplugins](#preparecomposerplugins)
We start a docker image for php with the specifies versions:
- `PHP_MAJOR_VERSION`
- `PHP_MINOR_VERSION`
In the script part we install two of our LakeDrops modules:
- [lakedrops/drupal-environment](../../composer/plugin/drupal-environment)
- [lakedrops/docker4drupal](../../composer/plugin/d4d)
After that we execute:
- `composer lakedrops:config`
- `composer lakedrops:docker4drupal`
The entire installation is saved as a build artefact called `build-prod-ng` where all `git` files are excluded.
Rules, when the job should run:
- `$DISABLE_DEPLOYMENT_LOCAL` != "1"
### Build
Here, we define jobs, which build the Drupal website and does some additional checks. This stage is
responsible for the database downloads as well.
#### Build Site
We start a docker image for php with the specifies versions:
- `PHP_MAJOR_VERSION`
- `PHP_MINOR_VERSION`
After that, we start to build the Drupal site with [Composer](https://getcomposer.org/):
Now we execute a `composer install` twice to ensure that all patches get applied correctly. After
that we setup our own `LakeDrops` tools by `composer lakedrops:scaffold` and
`composer lakedrops:docker4drupal`.
The next step is to pull all required [Docker](https://www.docker.com/) images
with [Docker Compose](https://docs.docker.com/compose/) adn start the services.
After everything has started, we create some directories, e.g. for styles, logs and JavaScript.
The entire installation is saved as a build artefact called `build` where all `git` files are excluded.
Rules, when the job should run:
- the tests are enabled by `$TESTSDISABLED` == "no"
#### Build Theme
This job creates the required files for the styling of the website.
The following jobs must run before successfully:
- [Build Prod Site](#build-prod-site)
#### Download DB
This job executes a script, that removes the SQL file `$]PROJECT_NAME}.sql`, if it exists already.
After that an Ansible script is executed, which dumbs the current Drupal database of the live stage.
The file is saved as a build artefact, which contains `$]PROJECT_NAME}.sql`.
Rules, when the job must not run:
- `$DISABLE_DEPLOYMENT` != "0"
- `$DISABLE_DEPLOYMENT_LOCAL` != "1"
- `$DBREQUIRED` == "no"
- `$INITIALINSTALL` == "yes"
- `$TESTSDISABLED` == "yes"
- `$LOCALDBFILE` != "none"
Rules, when the job should run:
- the tests are enabled by `$DISABLE_CI_TESTS` != "1"
#### Download DB NG
This is very similar to [Download DB](#download-db), but does not use Ansible.
It uses Docker Compose instead to execute the `drush sql:dump` command
to dumb the current live database to the file `$]PROJECT_NAME}.sql`.
The file is saved as a build artefact, which contains `$]PROJECT_NAME}.sql`.
Rules, when the job must not run:
- `$DISABLE_DEPLOYMENT` != "0"
- `$DISABLE_DEPLOYMENT_LOCAL` != "0"
- `$DBREQUIRED` == "no"
- `$INITIALINSTALL` == "yes"
- `$TESTSDISABLED` == "yes"
- `$LOCALDBFILE` != "none"
Rules, when the job should run:
- the tests are enabled by `$DISABLE_CI_TESTS` != "1"
### Prepare
This stage is used to prepare the system for testing. This stage is also used for internal purpose
and for proprietary tools.
#### Import DB
This job extends `.importdb`, which itself defines the
anchor [import_db_default](#import-db-default-anchor).
The following jobs must run before successfully:
- [Build Site](#build-site)
- [Download DB](#download-db)
Rules, when the job must not run:
- `$DISABLE_DEPLOYMENT` != "0"
- `$DISABLE_DEPLOYMENT_LOCAL` != "1"
- `$DBREQUIRED` == "no"
- `$INITIALINSTALL` == "yes"
- `$TESTSDISABLED` == "yes"
- `$LOCALDBFILE` != "none"
Rules, when the job should run:
- the tests are enabled by `$DISABLE_CI_TESTS` != "1"
##### Import DB Default Anchor
We start a docker image for php with the specifies versions:
- `PHP_MAJOR_VERSION`
- `PHP_MINOR_VERSION`
A script copies the available SQL file into the PHP container, drops the current DB and imports the file.
If the variable `SKIPCONFIGIMPORT` is `no`, a `drush config-import` gets executed as well. After that the
`UPDATE_DB_COMMAND` gets called as well as a cache reset.
#### Import DB NG
This job extends `.importdbNG`, which itself defines the
anchor [import_db_default](#import-db-default-anchor).
The following jobs must run before successfully:
- [Build Site](#build-site)
- [Download DB NG](#download-db-ng)
Rules, when the job must not run:
- `$DISABLE_DEPLOYMENT` != "0"
- `$DISABLE_DEPLOYMENT_LOCAL` != "0"
- `$DBREQUIRED` == "no"
- `$INITIALINSTALL` == "yes"
- `$TESTSDISABLED` == "yes"
- `$LOCALDBFILE` != "none"
Rules, when the job should run:
- the tests are enabled by `$DISABLE_CI_TESTS` != "1"
#### Import Local DB
This job extends `.importdbLocal`, which itself defines the
anchor [import_db_default](#import-db-default-anchor).
The following jobs must run before successfully:
- [Build Site](#build-site)
A before script is executed, which copies the `${LOCALDBFILE}` to `${PROJECT_NAME}.sql.gz` and
unzips that file.
Rules, when the job must not run:
- `$DBREQUIRED` == "no"
- `$TESTSDISABLED` == "yes"
- `$LOCALDBFILE` != "none"
Rules, when the job should run:
- the tests are enabled by `$DISABLE_CI_TESTS` != "1"
#### Update DB
We start a docker image for php with the specifies versions:
- `PHP_MAJOR_VERSION`
- `PHP_MINOR_VERSION`
If the variable `SKIPCONFIGIMPORT` is `no`, a `drush config-import` gets executed as well.
After that the `UPDATE_DB_COMMAND` gets called as well as a cache reset.
The following jobs must run before successfully:
- [Build Site](#build-site)
Rules, when the job must not run:
- `$DBREQUIRED` == "yes"
- `$DBUPDREQUIRED` == "no"
- `$INITIALINSTALL` == "yes"
- `$TESTSDISABLED` == "yes"
Rules, when the job should run:
- the tests are enabled by `$DISABLE_CI_TESTS` != "1"
### Deploy
This stage handles the deployment of the produced build artefact.
#### Deploy
The following jobs must run before successfully:
- [Build Prod Site](#build-prod-site)
- [Build Theme](#build-theme)
This job executes an [Ansible](https://www.ansible.com/) script, which does the deployment.
Rules, when the job must not run:
- `$DISABLE_DEPLOYMENT` != "0"
- `$DISABLE_DEPLOYMENT_LOCAL` != "1"
#### Deploy NG
The following jobs must run before successfully:
- [Build Prod Site](#build-prod-site)
- [Build Prod Site NG](#build-prod-site-ng)
- [Build Theme](#build-theme)
### Internal Stages
Here we have some internal stages, which cannot be executed alone. The purpose is to
collect repeating execution.
#### .prepareaccess
#### .preparecomposerplugins
## Configuration
### GitLab Runner
@@ -187,7 +542,7 @@ you have to write the scripts for those two tasks yourselves.
- `ENVIRONMENT_NAME`: the environment name for the GitLab UI.
- `THEME_CSS_PATH`: the relative path to the theme's css path where the
generated artefact can be found.
- `PHP_MAJOR_VERSION`: the major PHP version, defaults to 7.
- `PHP_MAJOR_VERSION`: the major PHP version, defaults to 10.
- `PHP_MINOR_VERSION`: the minor PHP version, default to 4.
- `COMPOSER_DOWNGRADE`: by default, composer 2 is being used. To use composer 1
instead, set this variable to 1.
@@ -201,6 +556,8 @@ you have to write the scripts for those two tasks yourselves.
test stage will be skipped. This is useful e.g. in a development environment
where you push and run pipelines often but don't want to run the tests every
single time.
- `$DISABLE_GITLAB_CI_TESTS`: if this variable is set to "1", no GitLab Tests will be
executed, see: [GitLab Templates](https://www.drupal.org/project/gitlab_templates).
- `DISABLE_CI_TEST_BACKSTOP`: if this variable is set to any value, the visual
regression tests with Backstop will be skipped.
- `DISABLE_CI_TEST_BEHAT`: if this variable is set to any value, the Behat
@@ -217,30 +574,6 @@ you have to write the scripts for those two tasks yourselves.
completely and built from scratch. Can also be used as
`[RESET_LOCALE]` in the Git commit message.
#### Stages and Tasks
It's important to use exactly these task names as most of them are also being
used to define dependencies:
- `validation`
- `Validate Environment`
- `buildprod`
- `Build Prod Site`
- `build`
- `Build Site`
- `Download DB`
- `prepare`
- `Import DB`
- `Update DB`
- `Build Theme`
- `test`
- `Test Code Style`
- `Test PHPUnit`
- `Test Behat`
- `Test Backstop`
- `deploy`
- `Deploy`
## Usage
This chapter is incomplete so far and needs more attention.