diff --git a/docs/composer/project/d10/index.md b/docs/composer/project/d10/index.md index 16935ab023a738949e93759fba6e7c60c35686f8..57d71282de0cf348c1021cfc34dffd4d88b4eddd 100644 --- a/docs/composer/project/d10/index.md +++ b/docs/composer/project/d10/index.md @@ -5,26 +5,60 @@ issues: https://gitlab.lakedrops.com/composer/project/drupal-10/-/issues tags: - composer --- -# Usage +# Create new project for local development -## Preparation +There are two ways to use this Drupal 10 project template. + +## By Composer - [Install composer](https://getcomposer.org): `sudo wget https://getcomposer.org/composer.phar --output-document=/usr/local/bin/composer && sudo chmod +x /usr/local/bin/composer` - Disable XDebug for CLI - optional: `sudo phpdismod -s cli xdebug` - [Install Prestissimo](https://github.com/hirak/prestissimo) - optional: `sudo composer global require "hirak/prestissimo:^0.3"` -## Create new project for local development - ```bash -composer create-project lakedrops/d9-project [DIRNAME] --no-interaction +composer create-project lakedrops/d10-project [DIRNAME] --no-interaction --no-dev +composer update cd [DIRNAME] ``` +## By L3D + +We **strongly recommend** to it this way. Just install [L3D](../../../docker/l3d/index.md) and things are going much easier. + +- Create an empty directory, where you want to locate your project, like `mkdir drupal10` +- go into this directory and execute: + +```bash +l3d +Name your project (used as subdomain): [drupal10] +``` + +Enter a name or press `Enter` to use the name of the directory. + +```bash +Lets start a new project here ... + +Options to start: + 0 none: start with an empty container + 1 LakeDrops Drupal 11 project template + 2 LakeDrops Drupal 10 project template + 3 drupal.org recommended project template (Drupal 11) + 4 drupal.org recommended project template (Drupal 10) + 5 Drupal's community project template (Drupal 11) + 6 Drupal's community project template (Drupal 10) + 7 Custom project template + 8 Existing git repository + + +Choose an option: +``` +Just select the `LakeDrops Drupal 10 Project Template` and you are ready to go. `L3D` cares about the initial setup. + This will build the full structure and create a git repository which you can then use during the full live-cycle of that project. To install a new site with the included default configuration, use `drush site-install --existing-config` and you get a jump-start from that. -## Deploy project to upstream stages +# Deploy project to upstream stages For deployment, both initially and for later updates, follow this: @@ -35,25 +69,9 @@ git clone YOUR-GIT-REPOSITORY . composer install ``` -## Adding new and updating existing components - -Edit the composer.json file accodring to your requirements and then run `composer update`. - -## Adding JS packages - -### CKEditor Codesnippet - -```bash -composer config repositories.codesnippet '{"type": "package","package": {"name": "ckeditorplugin/codesnippet","type": "drupal-library","version": "master","dist": {"type": "zip","url": "https://download.ckeditor.com/codesnippet/releases/codesnippet_4.7.3.zip","reference": "master"}}}' -composer require ckeditorplugin/codesnippet:master drupal/codesnippet -``` - -### Chosen +# Adding new and updating existing components -```bash -composer config repositories.codesnippet '{"type": "package","package": {"name": "harvesthq/chosen","version": "1.8.2","type": "drupal-library","dist": {"url": "https://github.com/harvesthq/chosen/releases/download/v1.8.2/chosen_v1.8.2.zip","type": "zip"}}}' -composer require harvesthq/chosen -``` +Edit the composer.json file according to your requirements and then run `composer update`. # Links diff --git a/docs/files/screenshots/pipeline_check4updates.png b/docs/files/screenshots/pipeline_check4updates.png new file mode 100644 index 0000000000000000000000000000000000000000..65942b32e26e11e3c42d53ee7f46e0cfef750f73 --- /dev/null +++ b/docs/files/screenshots/pipeline_check4updates.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b8e37419bac006b304fe61e0cf554d686885407e3f53cfd958971604dc332bfd +size 68316 diff --git a/docs/files/screenshots/pipeline_deploy.png b/docs/files/screenshots/pipeline_deploy.png new file mode 100644 index 0000000000000000000000000000000000000000..31e53473428609e06b41ade75549432674fc75a6 --- /dev/null +++ b/docs/files/screenshots/pipeline_deploy.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:34dd0dd3667d348bca2d3fb468a4013f1555aa935d0bf5730cb1bc62a1974796 +size 31753 diff --git a/docs/files/screenshots/pipeline_executebuild.png b/docs/files/screenshots/pipeline_executebuild.png new file mode 100644 index 0000000000000000000000000000000000000000..d3f782e8da1cb030c699a30f2bb3b5c8b1059e62 --- /dev/null +++ b/docs/files/screenshots/pipeline_executebuild.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6c172f171905fd1663ac2a0fc215271f398c23fb86be99a0887060a0b96a38b7 +size 47997 diff --git a/docs/files/screenshots/pipeline_lakedrops_build.png b/docs/files/screenshots/pipeline_lakedrops_build.png new file mode 100644 index 0000000000000000000000000000000000000000..a883676db5f96a992c1288d94ac92f5e4e78ff15 --- /dev/null +++ b/docs/files/screenshots/pipeline_lakedrops_build.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ebea0c72b325180cabfd90896565a434e0bc0fb6351933694b18fb4684f4be1b +size 63193 diff --git a/docs/files/screenshots/pipeline_lakedrops_deploy.png b/docs/files/screenshots/pipeline_lakedrops_deploy.png new file mode 100644 index 0000000000000000000000000000000000000000..9f0bcff2774c0bd00906ba1d54627022038482d5 --- /dev/null +++ b/docs/files/screenshots/pipeline_lakedrops_deploy.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:13d1d81ff8c03cb8d9adc2c17a46aea831b3e7142fa4ca2b0f9b910afe5e024b +size 37884 diff --git a/docs/files/screenshots/pipeline_publish_release.png b/docs/files/screenshots/pipeline_publish_release.png new file mode 100644 index 0000000000000000000000000000000000000000..45606e594f1e1d115112c8edf3168b70ac61fb53 --- /dev/null +++ b/docs/files/screenshots/pipeline_publish_release.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d08c881aa5513a2be423cc3045e950e04dbfb0321fc965fae88fe169735b0704 +size 42078 diff --git a/docs/gitlab/drupal/index.md b/docs/gitlab/drupal/index.md index c1d1b96bce27164395f3c051746f9be1eb46dd82..a0681a81bb01eb60ce566dc39f21fe14f43409d4 100644 --- a/docs/gitlab/drupal/index.md +++ b/docs/gitlab/drupal/index.md @@ -7,776 +7,10 @@ tags: - ci/cd - drupal --- -# GitLab CI for Drupal pipelines -This project contains a number of pre-configured tasks for GitLab CI/CD that -allow to run very powerful pipelines which contain the following stage order: +# Configuration -- [.pre (predefined)](#pre) -- [execute](#execute) -- [buildprod](#build-prod) -- [build (predefined)](#build) -- [prepare](#prepare) -- [postprocess](#postprocess) -- [validate](#validate) -- [test (predefined)](#test) -- [deploy (predefined)](#deploy) -- [finalize](#finalize) -- [release](#release-stage) - -We have a predefined `execute` job called `LakeDrops`, which looks like this: - -- [execute](#execute) - - LakeDrops - - [buildprod](#build-prod) - - [Build Prod Site](#build-prod-site) - - [Build Prod Site NG](#build-prod-site-ng) - - [build (predefined)](#build) - - [Build Site](#build-site) - - [Build Theme](#build-theme) - - [Check 4 Outdated Packages](#check-4-outdated-packages) - - [Check 4 Security Vulnerability Advisories](#check-4-security-vulnerability-advisories) - - [Download DB](#download-db) - - [Download DB NG](#download-db-ng) - - [prepare](#prepare) - - [Import DB](#import-db) - - [Import DB NG](#import-db-ng) - - [Import Local DB](#import-local-db) - - [Update DB](#update-db) - - [test](#test) - - [Test Code Style](#test-code-style) - - [Test PHPUnit](#test-phpunit) - - [Test Backstop](#test-backstop) - - [Test Cypress E2E](#test-cypress-e2e) - - [finalize](#finalize) - - [Shut Down Docker Project](#shut-down-docker-project) - -These pre-configured tasks can easily be included into your Drupal project. -A GitLab runner on any host can then be configured to run those pipelines. The -following chapters describe the necessary steps and also all the options -available for configuration. - -## Stages -The following chapters describes all stages and their jobs. The files: - -- [lakedrops.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/lakedrops.yml?ref_type=heads). -- [tests/lakedrops.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/tests/lakedrops.yml?ref_type=heads). -- [test_deploy.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/test-and-deploy.yml?ref_type=heads). -- [check4updates.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/check4updates.yml?ref_type=heads). - -### .pre - -To prepare and check, if the environment is good to go, we have a `.pre` stage, which contains the -following jobs: - -#### Sanity Checks - -The sanity check gets important when you are using [Crowdsec](https://www.crowdsec.net/). -It contains two little scripts, that check: - -- if the `crowdsec.settings.yml` is in the config split and -- if it is not enabled in the `core.extensions.yml`. - -Otherwise, it is conflicting with you local settings, since `CrowdSec` -is a tool for production sites. - -Except: - -- Commit message contains "Merge tag" -- Commit message contains "Merge branch" -- When the job is triggered by config auto export (CAE) - -#### Validate Environment - -This job extends the `.prerequisites` in -[test_deploy.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/test-and-deploy.yml?ref_type=heads). - -This job executes a script, where several [variables](#variables) are checked and set accordingly. - -Except: - -- Commit message contains "Merge tag" -- Commit message contains "Merge branch" -- When the job is triggered by config auto export (CAE) - -### Validate - -This stage is currently not officially used. - -### Execute - -This stage executes the build itself. - -### Build Prod - -This stage is responsible for building the production website, which we want to finally deploy. - -#### Build Prod Site - -First we have to do some preparation. This is done in the `before_script` section. - -- [.prepareaccess](#prepareaccess) -- [.preparecomposerplugins](#preparecomposerplugins) - -We start a docker image for php with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -In the script section we first update the environment by executing `/usr/local/bin/update-env`. - -After that we install the production site with `composer install` **without** the tools for development. That is pretty -it. We also add `drupal/core-vendor-hardening`. - -The entire installation is saved as a build artefact called `build-prod` where all `git` files are excluded. - -#### Build Prod Site NG - -First we have to do some preparation. This is done in the `before_script` section. - -- [.prepareaccess](#prepareaccess) -- [.preparecomposerplugins](#preparecomposerplugins) - -We start a docker image for php with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -In the script part we install two of our LakeDrops modules: - -- [lakedrops/drupal-environment](../../composer/plugin/drupal-environment) -- [lakedrops/docker4drupal](../../composer/plugin/d4d) - -After that we execute: - -- `composer lakedrops:config` -- `composer lakedrops:docker4drupal` - -The entire installation is saved as a build artefact called `build-prod-ng` where all `git` files are excluded. - -Rules, when the job should run: - -- `$DISABLE_DEPLOYMENT_LOCAL` != "1" - -### Build - -Here, we define jobs, which build the Drupal website and does some additional checks. This stage is -responsible for the database downloads as well. - -#### Build Site - -We start a docker image for `php` with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -After that, we start to build the Drupal site with [Composer](https://getcomposer.org/): - -Now we execute a `composer install` twice to ensure that all patches get applied correctly. After -that we setup our own `LakeDrops` tools by `composer lakedrops:scaffold` and -`composer lakedrops:docker4drupal`. - -The next step is to pull all required [Docker](https://www.docker.com/) images -with [Docker Compose](https://docs.docker.com/compose/) and start the services. - -After everything has started, we create some directories, e.g. for styles, logs and JavaScript. - -The entire installation is saved as a build artefact called `build` where all `git` files are excluded. - -Rules, when the job should run: - -- the tests are enabled by `$TESTSDISABLED` == "no" - -#### Build Theme - -This job creates the required files for the styling of the website. - -The following jobs must run before successfully: - -- [Build Prod Site](#build-prod-site) - -#### Check 4 Updates - -e start a docker image for `php` with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -First we have to do some preparation. This is done in the `before_script` section. - -- [.prepareaccess](#prepareaccess) -- [.preparecomposerplugins](#preparecomposerplugins) - -The job stores the artefacts in `check4updates`. - - - -Rules, when the job should run: - -- `$CHECKUPDATES` is set -- we are on the `develop` branch - - -#### Check 4 Outdated Packages - -We start a docker image for `php` with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -The following jobs must run before successfully. Some are optional, which means, if they exist, this job waits for them -to complete: - -- [Check 4 Updates](#check-4-updates) - - uses its artefacts - - optional -- [Build Prod Site](#build-prod-site) - - uses its artefacts - - optional -- [Build Prod Site NG](#build-prod-site-ng) - - uses its artefacts - - optional -- [Build Site](#build-site) - - uses its artefacts - - optional - -First we have to do some preparation. This is done in the `before_script` section. - -- [.prepareaccess](#prepareaccess) -- [.preparecomposerplugins](#preparecomposerplugins) - -First we perform a `composer update` which all dependencies without the development part. - -Then we check for outdated packages: - -```yml -composer outdated --minor-only --strict --no-interaction $OUTDATED_EXTRAS -composer outdated --patch-only --strict --no-interaction $OUTDATED_EXTRAS -``` - -The variable `$OUTDATED_EXTRAS` contains the ignored package like: -`--ignore=drupal/somemodule --ignore=drupal/another` - -Rules, when the job must not run: - -- the commit message contains `SKIP_check4outdated` and `$CHECKUPDATES` != "yes" -- the commit message contains "Merge branch 'develop' into 'main'" -- the commit message contains "Merge branch 'release' into 'main'" - -Rules, when the job should run: - -- `$IGNORE_COMPOSER_EXTENDED_AUDIT` == "0" - -#### Check 4 Security Vulnerability Advisories - -We start a docker image for `php` with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -The following jobs must run before successfully. Some are optional, which means, if they exist, this job waits for them -to complete: - -- [Check 4 Updates](#check-4-updates) - - uses its artefacts - - optional -- [Build Prod Site](#build-prod-site) - - uses its artefacts - - optional -- [Build Prod Site NG](#build-prod-site-ng) - - uses its artefacts - - optional -- [Build Site](#build-site) - - uses its artefacts - - optional - -First we have to do some preparation. This is done in the `before_script` section. - -- [.prepareaccess](#prepareaccess) -- [.preparecomposerplugins](#preparecomposerplugins) - -First we perform a `composer update` which all dependencies without the development part. - -Then we check for outdated packages: - -```yml -composer diagnose --no-interaction || true -composer audit --no-dev --locked --format=table -``` - -Rules, when the job must not run: - -- the commit message contains `SKIP_check4security` and `$CHECKUPDATES` != "yes" -- the commit message contains "Merge branch 'develop' into 'main'" -- the commit message contains "Merge branch 'release' into 'main'" - -Rules, when the job should run: - -- `$IGNORE_COMPOSER_AUDIT` == "0" - -#### Download DB - -This job executes a script, that removes the SQL file `$]PROJECT_NAME}.sql`, if it exists already. -After that an Ansible script is executed, which dumbs the current Drupal database of the live stage. - -The file is saved as a build artefact, which contains `$]PROJECT_NAME}.sql`. - -Rules, when the job must not run: - -- `$DISABLE_DEPLOYMENT` != "0" -- `$DISABLE_DEPLOYMENT_LOCAL` != "1" -- `$DBREQUIRED` == "no" -- `$INITIALINSTALL` == "yes" -- `$TESTSDISABLED` == "yes" -- `$LOCALDBFILE` != "none" - -Rules, when the job should run: - -- the tests are enabled by `$DISABLE_CI_TESTS` != "1" - -#### Download DB NG - -This is very similar to [Download DB](#download-db), but does not use Ansible. -It uses Docker Compose instead to execute the `drush sql:dump` command -to dumb the current live database to the file `$]PROJECT_NAME}.sql`. - -The file is saved as a build artefact, which contains `$]PROJECT_NAME}.sql`. - -Rules, when the job must not run: - -- `$DISABLE_DEPLOYMENT` != "0" -- `$DISABLE_DEPLOYMENT_LOCAL` != "0" -- `$DBREQUIRED` == "no" -- `$INITIALINSTALL` == "yes" -- `$TESTSDISABLED` == "yes" -- `$LOCALDBFILE` != "none" - -Rules, when the job should run: - -- the tests are enabled by `$DISABLE_CI_TESTS` != "1" - -### Prepare - -This stage is used to prepare the system for testing. This stage is also used for internal purpose -and for proprietary tools. - -#### Import DB - -This job extends `.importdb`, which itself defines the -anchor [import_db_default](#import-db-default-anchor). - -The following jobs must run before successfully: - -- [Build Site](#build-site) -- [Download DB](#download-db) - -Rules, when the job must not run: - - - `$DISABLE_DEPLOYMENT` != "0" - - `$DISABLE_DEPLOYMENT_LOCAL` != "1" - - `$DBREQUIRED` == "no" - - `$INITIALINSTALL` == "yes" - - `$TESTSDISABLED` == "yes" - - `$LOCALDBFILE` != "none" - -Rules, when the job should run: - -- the tests are enabled by `$DISABLE_CI_TESTS` != "1" - -##### Import DB Default Anchor - -We start a docker image for `php` with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -A script copies the available SQL file into the PHP container, drops the current DB and imports the file. -If the variable `SKIPCONFIGIMPORT` is `no`, a `drush config-import` gets executed as well. After that the -`UPDATE_DB_COMMAND` gets called as well as a cache reset. - -#### Import DB NG - -This job extends `.importdbNG`, which itself defines the -anchor [import_db_default](#import-db-default-anchor). - -The following jobs must run before successfully: - -- [Build Site](#build-site) -- [Download DB NG](#download-db-ng) - -Rules, when the job must not run: - -- `$DISABLE_DEPLOYMENT` != "0" -- `$DISABLE_DEPLOYMENT_LOCAL` != "0" -- `$DBREQUIRED` == "no" -- `$INITIALINSTALL` == "yes" -- `$TESTSDISABLED` == "yes" -- `$LOCALDBFILE` != "none" - -Rules, when the job should run: - -- the tests are enabled by `$DISABLE_CI_TESTS` != "1" - -#### Import Local DB - -This job extends `.importdbLocal`, which itself defines the -anchor [import_db_default](#import-db-default-anchor). - -The following jobs must run before successfully: - -- [Build Site](#build-site) - -A before script is executed, which copies the `${LOCALDBFILE}` to `${PROJECT_NAME}.sql.gz` and -unzips that file. - -Rules, when the job must not run: - -- `$DBREQUIRED` == "no" -- `$TESTSDISABLED` == "yes" -- `$LOCALDBFILE` != "none" - -Rules, when the job should run: - -- the tests are enabled by `$DISABLE_CI_TESTS` != "1" - -#### Update DB - -We start a docker image for php with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -If the variable `SKIPCONFIGIMPORT` is `no`, a `drush config-import` gets executed as well. -After that the `UPDATE_DB_COMMAND` gets called as well as a cache reset. - -The following jobs must run before successfully: - -- [Build Site](#build-site) - -Rules, when the job must not run: - -- `$DBREQUIRED` == "yes" -- `$DBUPDREQUIRED` == "no" -- `$INITIALINSTALL` == "yes" -- `$TESTSDISABLED` == "yes" - -Rules, when the job should run: - -- the tests are enabled by `$DISABLE_CI_TESTS` != "1" - -### Postprocess - -#### Check 4 Updates Commit - -We start a docker image for php with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -The following jobs must run before successfully. Some are optional, which means, if they exist, this job waits for them -to complete: - -- [Check 4 Updates](#check-4-updates) - - uses its artefacts -- [Check 4 Security Vulnerability Advisories](#check-4-security-vulnerability-advisories) - - optional -- [Check 4 Outdated Packages](#check-4-outdated-packages) - - optional - -First we have to do some preparation. This is done in the `before_script` section. - -- [.prepareaccess](#prepareaccess) -- [.preparecomposerplugins](#preparecomposerplugins) - - - -Rules, when the job should run: - -- `$CHECKUPDATES` is set -- we are on the `develop` branch - -### Test - -After the preparation and the setup of a Drupal database, we can execute tests. This happens in this stage. - -All tests use the two [Internal Test Stages](#internal-test-stages). - -#### Test Code Style - -This job uses [PHP Code Sniffer](../../dev_tools/test.md#php-code-sniffer) and stores the artefacts in `phpcs`. - -Rules, when the job should run: - -- the code style tests are enabled by `$DISABLE_CI_TEST_CODESTYLE` != "1" - -#### Test PHPUnit - -This job uses [PHP Unit](../../dev_tools/test.md#unitkernelfunctional-testing) and stores the artefacts in `phpunit`. - -Rules, when the job should run: - -- the code style tests are enabled by `$DISABLE_CI_TEST_PHPUNIT` != "1" - - -#### Test Backstop - -This job uses [Backstop](../../dev_tools/test.md#unitkernelfunctional-testing) and stores the artefacts in `backstop`. - -The following jobs must run before successfully. Some are optional, which means, if they exist, this job waits for them -to complete: - -- [Build Site](#build-site) -- [Build Theme](#build-theme) - - uses its artefacts - - optional -- [Import DB](#import-db) - - optional -- [Import DB NG](#import-db-ng) - - optional -- [Import Local DB](#import-local-db) - - optional -- [Update DB](#update-db) - - optional - -Rules, when the job should run: - -- the backstop tests are enabled by `$DISABLE_CI_TEST_BACKSTOP` != "1" - - -#### Test Cypress E2E - -This job uses [Cypress](../../dev_tools/test.md#cypress) and stores the artefacts in `cypresse2e`. The tests -are executed in headless mode. - -The following jobs must run before successfully. Some are optional, which means, if they exist, this job waits for them -to complete: - -- [Build Site](#build-site) -- [Build Theme](#build-theme) - - uses its artefacts - - optional -- [Import DB](#import-db) - - optional -- [Import DB NG](#import-db-ng) - - optional -- [Import Local DB](#import-local-db) - - optional -- [Update DB](#update-db) - - optional -- [Test Backstop](#test-backstop) - - optional - -Rules, when the job should run: - -- the cypress tests are enabled by `$DISABLE_CI_TEST_CYPRESSE2E` != "1" - -#### Internal Test Stages - -Every test stage needs internal reoccurring definitions. - -**.test_lakedrops_default** - -We start a docker image for `php` with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -The following jobs must run before successfully: - -- [Build Site](#build-site) - -But this job can start running when the [Build Site](#build-site) has finished, so other -test jobs can run in parallel. - -After running, we copy the logs from the `php` container to the current directory. - - -**.test_lakedrops_theme** - -The following jobs must run before successfully: - -- [Build Site](#build-site) -- [Build Theme](#build-theme) - -In the `before_script` part, we copy the `$THEME_CSS_PATH` into the `php` container, if the -`$THEME_BUILD` variable ist set to "yes". - -In `after_script` we remove `$THEME_CSS_PATH` again. - -#### - -### Deploy - -This stage handles the deployment of the produced build artefact. - -#### Deploy - -The following jobs must run before successfully: - -- [Build Prod Site](#build-prod-site) -- [Build Theme](#build-theme) - -This job executes an [Ansible](https://www.ansible.com/) script, which does the deployment. - -Rules, when the job must not run: - -- `$DISABLE_DEPLOYMENT` != "0" -- `$DISABLE_DEPLOYMENT_LOCAL` != "1" - -#### Deploy NG - -The following jobs must run before successfully: - -- [Build Prod Site](#build-prod-site) -- [Build Prod Site NG](#build-prod-site-ng) -- [Build Theme](#build-theme) - -First we have to do some preparation. This is done in the `before_script` section. - -- [.prepareaccess](#prepareaccess) - -This job does not use [Ansible](https://www.ansible.com/). A lot of commands are executed in the script section. -Here is a summary: - -- create the directories, where the site gets deployed to: - - app - - db - - files - - redis - - backup -- sets the ownership and permissions of these files -- start all the docker service with [Docker Compose](https://docs.docker.com/compose/) -- execute several `drush` commands -- create config auto export directory -- restart cron jobs - -To see all the commands, visit -[test_deploy.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/test-and-deploy.yml?ref_type=heads). - -Rules, when the job must not run: - -- `$DISABLE_DEPLOYMENT` != "0" -- `$CI_COMMIT_BRANCH` != "main" && `$ENFORCE_DEPLOYMENT` != "1" - -### Finalize - -Here we can do cleanups and remove some garbage the build process produces. - -#### Shut Down Docker Project - -We start a docker image for php with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -The following jobs must run before successfully: - -- [Build Site](#build-site) - -This job simply stops and removes the docker container, which are started for the build by: - -`docker compose down` - -### Release Stage - -This stage is responsible for creating a new release. - -#### Release Job - -This job extends the internal job [.merge](#merge). - -Rules, when the job should run: - -- `$CI_COMMIT_MESSAGE` contains `CREATE_RELEASE` -- we are on the `develop` branch - -Rules, when the job must not run: - -- if the reference is a `tag` -- `$CI_COMMIT_MESSAGE` contains `Merge tag` -- if it is triggered by config auto export (CAE) -- `$CHECKUPDATES` is set -- `$TRIGGERTASK` is set - - -### Internal Stages - -Here we have some internal stages, which cannot be executed alone. The purpose is to -collect repeating execution. - -#### .prepareaccess - -This internal job, you find in [mixins.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/general/-/blob/main/mixins.yml?ref_type=heads). - -Mainly, we care about getting access to: - -- the host `$CI_SERVER_HOST` with `ssh` via `$SSH_PRIVATE_KEY` -- GitLab Login via `$GITLAB_ACCESS_TOKEN` and `$GITLAB_PRIVATE_TOKEN` -- Docker configuration via `$DOCKER_AUTH_CONFIG` - - -#### .preparecomposerplugins - -This internal job, you find in [mixins.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/general/-/blob/main/mixins.yml?ref_type=heads). - -The following points will be prepared for [Composer](https://getcomposer.org/): - -- the version >2, if `$DOWNGRADE_COMPOSER` is not set -- the configuration `allow-plugins` -- the configuration `audit.abandoned`, depend on `$IGNORE_COMPOSER_ABANDONED_AUDIT` -- unsupported module versions, which are allowed to use - -#### .merge - -First we have to do some preparation. This is done in the `before_script` section. - -- [.prepareaccess](#prepareaccess) - -We start a docker image for php with the specifies versions: - -- `PHP_MAJOR_VERSION` -- `PHP_MINOR_VERSION` - -We create a clean directory and clone the git project `$CI_PROJECT_PATH` -for branch `$SOURCE_BRANCH` from `$CI_SERVER_HOST` right into it. - -After that we execute the [merge](https://gitlab.lakedrops.com/docker/gitlab-drupal-ci/-/blob/main/bin/merge?ref_type=heads) -script, which does a few checks and use the `gitlab` tool to merge. - -Finally, we remove the directory we created above. - -#### .retry-for-system-issues - -This little job ste the amount of retries to "2", when: - -- runner_system_failure -- stuck_or_timeout_failure - -#### .cache_paths - -All internal jobs concerning cache will use this job here. It defines the paths: - -- .docker-init/ -- assets/ -- drush/ -- files/ -- keys/ -- settings/ -- vendor/ -- web/core/ -- web/libraries/ -- web/modules/contrib/ -- web/profiles/contrib/ -- web/sites/ -- web/themes/contrib/ -- .ahoy.yml -- .env -- docker-compose.yml - - -## Configuration - -### GitLab Runner +## GitLab Runner To setup a GitLab runner, they need to be [installed first](https://docs.gitlab.com/runner/install) - see also our @@ -813,7 +47,7 @@ different PHP versions a different GitLab runner should be configured or the image being overwritten for each task in the `.gitlab-ci.yml` of the respective project. -### GitLab Variables +## GitLab Variables Some variables are required for proper access control, and you have to provide them in the project configuration by going to **Settings / CI/CD / Variables** @@ -825,7 +59,7 @@ and adding these variables: repository, you have to create an access tokeb for the user who runs the pipeline and provide it in this variable. -### GitLab CI instructions in Drupal project +## GitLab CI instructions in Drupal project The Drupal project should be setup with the [Drupal Development Environment](/composer/plugin/drupal-dev-environment) @@ -834,93 +68,18 @@ configuration: ```yaml variables: - COMPOSE_PROJECT_NAME: myproject_$CI_COMMIT_REF_SLUG - ENVIRONMENT_NAME: myproject/$CI_COMMIT_REF_NAME - THEME_CSS_PATH: web/themes/custom/mytheme/css + PROJECT_NAME: <project_name> + HOST_NAME: <your_host_to_deploy> + THEME_BASE_PATH: web/themes/<base_path> + THEME_CSS_PATH: web/themes/<path>/css + PHP_VERSION: '8.3' + PHP_MAJOR_VERSION: 8 + PHP_MINOR_VERSION: 3 include: - project: 'gitlab-ci-cd/drupal' - ref: master - file: '/test-and-deploy.yml' - -Validate Environment: - extends: '.prerequisites' - -Build Prod Site: - extends: '.build-prod' - -Build Site: - extends: '.build' - -Download DB: - stage: build - tags: - - default - variables: - GIT_STRATEGY: none - script: | - if [[ "$DBREQUIRED" == "yes" ]]; then - echo "Scripts have not been developed yet" - fi - cache: {} - dependencies: - - 'Validate Environment' - artifacts: - name: dbdump - when: always - paths: - - ${CI_PROJECT_NAME}.sql - except: - refs: - - tags - variables: - - $CI_COMMIT_MESSAGE =~ /^Merge tag /i - - $CAE - - $DISABLE_CI_TESTS - -Import DB: - extends: '.importdb' - -Update DB: - extends: '.updatedb' - -Build Theme: - extends: '.theme' - before_script: - - cd web/themes/custom/mytheme - -Test Code Style: - extends: '.codestyle' - -Test PHPUnit: - extends: '.phpunit' - -Test Behat: - extends: '.behat' - -Test Backstop: - extends: '.backstop' - -Deploy: - stage: deploy - tags: - - default - variables: - GIT_STRATEGY: none - environment: - name: ${ENVIRONMENT_NAME} - script: - - echo "Scripts have not been developed yet" - cache: {} - dependencies: - - 'Build Prod Site' - - 'Build Theme' - except: - refs: - - tags - variables: - - $CI_COMMIT_MESSAGE =~ /^Merge tag /i - - $CAE + ref: main + file: '/lakedrops.yml' ``` The tasks for downloading the database dump and to deploy the site finally, @@ -930,15 +89,19 @@ using our Ansible environment too, then please refer to [Using Ansible for Deployment](#Using Ansible for Deployment) below. Otherwise, you have to write the scripts for those two tasks yourselves. -#### Variables +### Variables - `COMPOSE_PROJECT_NAME`: a string only unique project name which will be used to identify caches and Docker containers. +- `HOST_NAME`: the host name where you want to deploy your Drupal site - `ENVIRONMENT_NAME`: the environment name for the GitLab UI. +- `ENVIRONMENT_URL`: the environment url of your website +- `THEME_BASE_PATH`: the relative patch to the base theme, if you want to use one - `THEME_CSS_PATH`: the relative path to the theme's css path where the generated artefact can be found. -- `PHP_MAJOR_VERSION`: the major PHP version, defaults to 10. -- `PHP_MINOR_VERSION`: the minor PHP version, default to 4. +- `PHP_VERSION: '8.3'`: the string representation of the PHP version +- `PHP_MAJOR_VERSION`: the major PHP version, defaults to 8. +- `PHP_MINOR_VERSION`: the minor PHP version, default to 3. - `COMPOSER_DOWNGRADE`: by default, composer 2 is being used. To use composer 1 instead, set this variable to 1. - `INITIAL_INSTALL`: if set to `yes`, the initial Drupal site installation gets @@ -969,14 +132,14 @@ you have to write the scripts for those two tasks yourselves. completely and built from scratch. Can also be used as `[RESET_LOCALE]` in the Git commit message. -## Usage +# Usage This chapter is incomplete so far and needs more attention. -### Initial installation of a Drupal site +## Initial installation of a Drupal site This pipeline covers both, the initial installation and later updates of a -Drupal site. The latter is the default, because it happens regularely whereas +Drupal site. The latter is the default, because it happens regularly whereas the initial installation only happens once. Therefore, if you run the pipeline the first time to initially install the @@ -987,7 +150,7 @@ pipeline from the GitLab UI. When either of these conditions apply, no database will be downloaded, imported or updated and all tests will be skipped. -### Handling of the database +## Handling of the database To build and test your Drupal site prior to deployment, a database with content for this project is required and this is handled by the pipeline pretty smart. @@ -1012,16 +175,16 @@ task `Import DB` will then import that dump in the `prepare` stage. Otherwise, those two tasks will be skipped and the task `Update DB` will be executed in the `prepare` stage instead. -## Example +# Example We provide a [Demo Drupal 10 project](https://gitlab.lakedrops.com/gitlab-ci-cd/demos/drupal-10) which uses this GitLab CI framework. If you want to give it a try, login to our GitLab, go to the project and click on "Request access". Once we've accepted your request, you can clone the project, make some changes and push them back to the project. This will [trigger the pipeline](https://gitlab.lakedrops.com/gitlab-ci-cd/demos/drupal-9/-/pipelines) -thet you can watch running. +then you can watch running. -The deployed demo site is [available online](https://d9.demo.lakedrops.com). +The deployed demo site is [available online](https://d10.demo.lakedrops.com). The setup for this project is very simple. Because we wanted to protect the pipeline configuration, we removed the default `.gitlab-ci.yml` file from diff --git a/docs/gitlab/pipelines/index.md b/docs/gitlab/pipelines/index.md new file mode 100644 index 0000000000000000000000000000000000000000..9e92ad6a7bbab216c5884b591a7a3cc2f21e68e4 --- /dev/null +++ b/docs/gitlab/pipelines/index.md @@ -0,0 +1,775 @@ +--- +title: GitLab Drupal +repo: https://gitlab.lakedrops.com/gitlab-ci-cd/pipelines +issues: https://gitlab.lakedrops.com/gitlab-ci-cd/pipelines/-/issues +tags: +- gitlab +- ci/cd +- pipeline +- job +--- +# GitLab CI for Drupal pipelines + +This project contains a number of pre-configured tasks for GitLab CI/CD that +allow to run very powerful pipelines which contain the following stage order: + +- [.pre (predefined)](#pre) +- [execute](#execute) +- [buildprod](#build-prod) +- [build (predefined)](#build) +- [prepare](#prepare) +- [postprocess](#postprocess) +- [validate](#validate) +- [test (predefined)](#test) +- [deploy (predefined)](#deploy) +- [finalize](#finalize) +- [release](#release-stage) + +We have a predefined `execute` job called `LakeDrops`, which looks like this: + +- [execute](#execute) + - LakeDrops + - [buildprod](#build-prod) + - [Build Prod Site](#build-prod-site) + - [Build Prod Site NG](#build-prod-site-ng) + - [build (predefined)](#build) + - [Build Site](#build-site) + - [Build Theme](#build-theme) + - [Check 4 Outdated Packages](#check-4-outdated-packages) + - [Check 4 Security Vulnerability Advisories](#check-4-security-vulnerability-advisories) + - [Download DB](#download-db) + - [Download DB NG](#download-db-ng) + - [prepare](#prepare) + - [Import DB](#import-db) + - [Import DB NG](#import-db-ng) + - [Import Local DB](#import-local-db) + - [Update DB](#update-db) + - [test](#test) + - [Test Code Style](#test-code-style) + - [Test PHPUnit](#test-phpunit) + - [Test Backstop](#test-backstop) + - [Test Cypress E2E](#test-cypress-e2e) + - [finalize](#finalize) + - [Shut Down Docker Project](#shut-down-docker-project) + +These pre-configured tasks can easily be included into your Drupal project. +A GitLab runner on any host can then be configured to run those pipelines. The +following chapters describe the necessary steps and also all the options +available for configuration. + +## Stages +The following chapters describes all stages and their jobs. The files: + +- [lakedrops.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/lakedrops.yml?ref_type=heads). +- [tests/lakedrops.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/tests/lakedrops.yml?ref_type=heads). +- [test_deploy.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/test-and-deploy.yml?ref_type=heads). +- [check4updates.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/check4updates.yml?ref_type=heads). + +### .pre + +To prepare and check, if the environment is good to go, we have a `.pre` stage, which contains the +following jobs: + +#### Sanity Checks + +The sanity check gets important when you are using [Crowdsec](https://www.crowdsec.net/). +It contains two little scripts, that check: + +- if the `crowdsec.settings.yml` is in the config split and +- if it is not enabled in the `core.extensions.yml`. + +Otherwise, it is conflicting with you local settings, since `CrowdSec` +is a tool for production sites. + +Except: + +- Commit message contains "Merge tag" +- Commit message contains "Merge branch" +- When the job is triggered by config auto export (CAE) + +#### Validate Environment + +This job extends the `.prerequisites` in +[test_deploy.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/test-and-deploy.yml?ref_type=heads). + +This job executes a script, where several [variables](#variables) are checked and set accordingly. + +Except: + +- Commit message contains "Merge tag" +- Commit message contains "Merge branch" +- When the job is triggered by config auto export (CAE) + +### Validate + +This stage is currently not officially used. + +### Execute + +This stage executes the build itself. + +### Build Prod + +This stage is responsible for building the production website, which we want to finally deploy. + +#### Build Prod Site + +First we have to do some preparation. This is done in the `before_script` section. + +- [.prepareaccess](#prepareaccess) +- [.preparecomposerplugins](#preparecomposerplugins) + +We start a docker image for php with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +In the script section we first update the environment by executing `/usr/local/bin/update-env`. + +After that we install the production site with `composer install` **without** the tools for development. That is pretty +it. We also add `drupal/core-vendor-hardening`. + +The entire installation is saved as a build artefact called `build-prod` where all `git` files are excluded. + +#### Build Prod Site NG + +First we have to do some preparation. This is done in the `before_script` section. + +- [.prepareaccess](#prepareaccess) +- [.preparecomposerplugins](#preparecomposerplugins) + +We start a docker image for php with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +In the script part we install two of our LakeDrops modules: + +- [lakedrops/drupal-environment](../../composer/plugin/drupal-environment) +- [lakedrops/docker4drupal](../../composer/plugin/d4d) + +After that we execute: + +- `composer lakedrops:config` +- `composer lakedrops:docker4drupal` + +The entire installation is saved as a build artefact called `build-prod-ng` where all `git` files are excluded. + +Rules, when the job should run: + +- `$DISABLE_DEPLOYMENT_LOCAL` != "1" + +### Build + +Here, we define jobs, which build the Drupal website and does some additional checks. This stage is +responsible for the database downloads as well. + +#### Build Site + +We start a docker image for `php` with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +After that, we start to build the Drupal site with [Composer](https://getcomposer.org/): + +Now we execute a `composer install` twice to ensure that all patches get applied correctly. After +that we setup our own `LakeDrops` tools by `composer lakedrops:scaffold` and +`composer lakedrops:docker4drupal`. + +The next step is to pull all required [Docker](https://www.docker.com/) images +with [Docker Compose](https://docs.docker.com/compose/) and start the services. + +After everything has started, we create some directories, e.g. for styles, logs and JavaScript. + +The entire installation is saved as a build artefact called `build` where all `git` files are excluded. + +Rules, when the job should run: + +- the tests are enabled by `$TESTSDISABLED` == "no" + +#### Build Theme + +This job creates the required files for the styling of the website. + +The following jobs must run before successfully: + +- [Build Prod Site](#build-prod-site) + +#### Check 4 Updates + +e start a docker image for `php` with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +First we have to do some preparation. This is done in the `before_script` section. + +- [.prepareaccess](#prepareaccess) +- [.preparecomposerplugins](#preparecomposerplugins) + +The job stores the artefacts in `check4updates`. + + + +Rules, when the job should run: + +- `$CHECKUPDATES` is set +- we are on the `develop` branch + + +#### Check 4 Outdated Packages + +We start a docker image for `php` with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +The following jobs must run before successfully. Some are optional, which means, if they exist, this job waits for them +to complete: + +- [Check 4 Updates](#check-4-updates) + - uses its artefacts + - optional +- [Build Prod Site](#build-prod-site) + - uses its artefacts + - optional +- [Build Prod Site NG](#build-prod-site-ng) + - uses its artefacts + - optional +- [Build Site](#build-site) + - uses its artefacts + - optional + +First we have to do some preparation. This is done in the `before_script` section. + +- [.prepareaccess](#prepareaccess) +- [.preparecomposerplugins](#preparecomposerplugins) + +First we perform a `composer update` which all dependencies without the development part. + +Then we check for outdated packages: + +```yml +composer outdated --minor-only --strict --no-interaction $OUTDATED_EXTRAS +composer outdated --patch-only --strict --no-interaction $OUTDATED_EXTRAS +``` + +The variable `$OUTDATED_EXTRAS` contains the ignored package like: +`--ignore=drupal/somemodule --ignore=drupal/another` + +Rules, when the job must not run: + +- the commit message contains `SKIP_check4outdated` and `$CHECKUPDATES` != "yes" +- the commit message contains "Merge branch 'develop' into 'main'" +- the commit message contains "Merge branch 'release' into 'main'" + +Rules, when the job should run: + +- `$IGNORE_COMPOSER_EXTENDED_AUDIT` == "0" + +#### Check 4 Security Vulnerability Advisories + +We start a docker image for `php` with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +The following jobs must run before successfully. Some are optional, which means, if they exist, this job waits for them +to complete: + +- [Check 4 Updates](#check-4-updates) + - uses its artefacts + - optional +- [Build Prod Site](#build-prod-site) + - uses its artefacts + - optional +- [Build Prod Site NG](#build-prod-site-ng) + - uses its artefacts + - optional +- [Build Site](#build-site) + - uses its artefacts + - optional + +First we have to do some preparation. This is done in the `before_script` section. + +- [.prepareaccess](#prepareaccess) +- [.preparecomposerplugins](#preparecomposerplugins) + +First we perform a `composer update` which all dependencies without the development part. + +Then we check for outdated packages: + +```yml +composer diagnose --no-interaction || true +composer audit --no-dev --locked --format=table +``` + +Rules, when the job must not run: + +- the commit message contains `SKIP_check4security` and `$CHECKUPDATES` != "yes" +- the commit message contains "Merge branch 'develop' into 'main'" +- the commit message contains "Merge branch 'release' into 'main'" + +Rules, when the job should run: + +- `$IGNORE_COMPOSER_AUDIT` == "0" + +#### Download DB + +This job executes a script, that removes the SQL file `$]PROJECT_NAME}.sql`, if it exists already. +After that an Ansible script is executed, which dumbs the current Drupal database of the live stage. + +The file is saved as a build artefact, which contains `$]PROJECT_NAME}.sql`. + +Rules, when the job must not run: + +- `$DISABLE_DEPLOYMENT` != "0" +- `$DISABLE_DEPLOYMENT_LOCAL` != "1" +- `$DBREQUIRED` == "no" +- `$INITIALINSTALL` == "yes" +- `$TESTSDISABLED` == "yes" +- `$LOCALDBFILE` != "none" + +Rules, when the job should run: + +- the tests are enabled by `$DISABLE_CI_TESTS` != "1" + +#### Download DB NG + +This is very similar to [Download DB](#download-db), but does not use Ansible. +It uses Docker Compose instead to execute the `drush sql:dump` command +to dumb the current live database to the file `$]PROJECT_NAME}.sql`. + +The file is saved as a build artefact, which contains `$]PROJECT_NAME}.sql`. + +Rules, when the job must not run: + +- `$DISABLE_DEPLOYMENT` != "0" +- `$DISABLE_DEPLOYMENT_LOCAL` != "0" +- `$DBREQUIRED` == "no" +- `$INITIALINSTALL` == "yes" +- `$TESTSDISABLED` == "yes" +- `$LOCALDBFILE` != "none" + +Rules, when the job should run: + +- the tests are enabled by `$DISABLE_CI_TESTS` != "1" + +### Prepare + +This stage is used to prepare the system for testing. This stage is also used for internal purpose +and for proprietary tools. + +#### Import DB + +This job extends `.importdb`, which itself defines the +anchor [import_db_default](#import-db-default-anchor). + +The following jobs must run before successfully: + +- [Build Site](#build-site) +- [Download DB](#download-db) + +Rules, when the job must not run: + + - `$DISABLE_DEPLOYMENT` != "0" + - `$DISABLE_DEPLOYMENT_LOCAL` != "1" + - `$DBREQUIRED` == "no" + - `$INITIALINSTALL` == "yes" + - `$TESTSDISABLED` == "yes" + - `$LOCALDBFILE` != "none" + +Rules, when the job should run: + +- the tests are enabled by `$DISABLE_CI_TESTS` != "1" + +##### Import DB Default Anchor + +We start a docker image for `php` with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +A script copies the available SQL file into the PHP container, drops the current DB and imports the file. +If the variable `SKIPCONFIGIMPORT` is `no`, a `drush config-import` gets executed as well. After that the +`UPDATE_DB_COMMAND` gets called as well as a cache reset. + +#### Import DB NG + +This job extends `.importdbNG`, which itself defines the +anchor [import_db_default](#import-db-default-anchor). + +The following jobs must run before successfully: + +- [Build Site](#build-site) +- [Download DB NG](#download-db-ng) + +Rules, when the job must not run: + +- `$DISABLE_DEPLOYMENT` != "0" +- `$DISABLE_DEPLOYMENT_LOCAL` != "0" +- `$DBREQUIRED` == "no" +- `$INITIALINSTALL` == "yes" +- `$TESTSDISABLED` == "yes" +- `$LOCALDBFILE` != "none" + +Rules, when the job should run: + +- the tests are enabled by `$DISABLE_CI_TESTS` != "1" + +#### Import Local DB + +This job extends `.importdbLocal`, which itself defines the +anchor [import_db_default](#import-db-default-anchor). + +The following jobs must run before successfully: + +- [Build Site](#build-site) + +A before script is executed, which copies the `${LOCALDBFILE}` to `${PROJECT_NAME}.sql.gz` and +unzips that file. + +Rules, when the job must not run: + +- `$DBREQUIRED` == "no" +- `$TESTSDISABLED` == "yes" +- `$LOCALDBFILE` != "none" + +Rules, when the job should run: + +- the tests are enabled by `$DISABLE_CI_TESTS` != "1" + +#### Update DB + +We start a docker image for php with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +If the variable `SKIPCONFIGIMPORT` is `no`, a `drush config-import` gets executed as well. +After that the `UPDATE_DB_COMMAND` gets called as well as a cache reset. + +The following jobs must run before successfully: + +- [Build Site](#build-site) + +Rules, when the job must not run: + +- `$DBREQUIRED` == "yes" +- `$DBUPDREQUIRED` == "no" +- `$INITIALINSTALL` == "yes" +- `$TESTSDISABLED` == "yes" + +Rules, when the job should run: + +- the tests are enabled by `$DISABLE_CI_TESTS` != "1" + +### Postprocess + +#### Check 4 Updates Commit + +We start a docker image for php with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +The following jobs must run before successfully. Some are optional, which means, if they exist, this job waits for them +to complete: + +- [Check 4 Updates](#check-4-updates) + - uses its artefacts +- [Check 4 Security Vulnerability Advisories](#check-4-security-vulnerability-advisories) + - optional +- [Check 4 Outdated Packages](#check-4-outdated-packages) + - optional + +First we have to do some preparation. This is done in the `before_script` section. + +- [.prepareaccess](#prepareaccess) +- [.preparecomposerplugins](#preparecomposerplugins) + + + +Rules, when the job should run: + +- `$CHECKUPDATES` is set +- we are on the `develop` branch + +### Test + +After the preparation and the setup of a Drupal database, we can execute tests. This happens in this stage. + +All tests use the two [Internal Test Stages](#internal-test-stages). + +#### Test Code Style + +This job uses [PHP Code Sniffer](../../dev_tools/test.md#php-code-sniffer) and stores the artefacts in `phpcs`. + +Rules, when the job should run: + +- the code style tests are enabled by `$DISABLE_CI_TEST_CODESTYLE` != "1" + +#### Test PHPUnit + +This job uses [PHP Unit](../../dev_tools/test.md#unitkernelfunctional-testing) and stores the artefacts in `phpunit`. + +Rules, when the job should run: + +- the code style tests are enabled by `$DISABLE_CI_TEST_PHPUNIT` != "1" + + +#### Test Backstop + +This job uses [Backstop](../../dev_tools/test.md#unitkernelfunctional-testing) and stores the artefacts in `backstop`. + +The following jobs must run before successfully. Some are optional, which means, if they exist, this job waits for them +to complete: + +- [Build Site](#build-site) +- [Build Theme](#build-theme) + - uses its artefacts + - optional +- [Import DB](#import-db) + - optional +- [Import DB NG](#import-db-ng) + - optional +- [Import Local DB](#import-local-db) + - optional +- [Update DB](#update-db) + - optional + +Rules, when the job should run: + +- the backstop tests are enabled by `$DISABLE_CI_TEST_BACKSTOP` != "1" + + +#### Test Cypress E2E + +This job uses [Cypress](../../dev_tools/test.md#cypress) and stores the artefacts in `cypresse2e`. The tests +are executed in headless mode. + +The following jobs must run before successfully. Some are optional, which means, if they exist, this job waits for them +to complete: + +- [Build Site](#build-site) +- [Build Theme](#build-theme) + - uses its artefacts + - optional +- [Import DB](#import-db) + - optional +- [Import DB NG](#import-db-ng) + - optional +- [Import Local DB](#import-local-db) + - optional +- [Update DB](#update-db) + - optional +- [Test Backstop](#test-backstop) + - optional + +Rules, when the job should run: + +- the cypress tests are enabled by `$DISABLE_CI_TEST_CYPRESSE2E` != "1" + +#### Internal Test Stages + +Every test stage needs internal reoccurring definitions. + +**.test_lakedrops_default** + +We start a docker image for `php` with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +The following jobs must run before successfully: + +- [Build Site](#build-site) + +But this job can start running when the [Build Site](#build-site) has finished, so other +test jobs can run in parallel. + +After running, we copy the logs from the `php` container to the current directory. + + +**.test_lakedrops_theme** + +The following jobs must run before successfully: + +- [Build Site](#build-site) +- [Build Theme](#build-theme) + +In the `before_script` part, we copy the `$THEME_CSS_PATH` into the `php` container, if the +`$THEME_BUILD` variable ist set to "yes". + +In `after_script` we remove `$THEME_CSS_PATH` again. + +#### + +### Deploy + +This stage handles the deployment of the produced build artefact. + +#### Deploy + +The following jobs must run before successfully: + +- [Build Prod Site](#build-prod-site) +- [Build Theme](#build-theme) + +This job executes an [Ansible](https://www.ansible.com/) script, which does the deployment. + +Rules, when the job must not run: + +- `$DISABLE_DEPLOYMENT` != "0" +- `$DISABLE_DEPLOYMENT_LOCAL` != "1" + +#### Deploy NG + +The following jobs must run before successfully: + +- [Build Prod Site](#build-prod-site) +- [Build Prod Site NG](#build-prod-site-ng) +- [Build Theme](#build-theme) + +First we have to do some preparation. This is done in the `before_script` section. + +- [.prepareaccess](#prepareaccess) + +This job does not use [Ansible](https://www.ansible.com/). A lot of commands are executed in the script section. +Here is a summary: + +- create the directories, where the site gets deployed to: + - app + - db + - files + - redis + - backup +- sets the ownership and permissions of these files +- start all the docker service with [Docker Compose](https://docs.docker.com/compose/) +- execute several `drush` commands +- create config auto export directory +- restart cron jobs + +To see all the commands, visit +[test_deploy.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/test-and-deploy.yml?ref_type=heads). + +Rules, when the job must not run: + +- `$DISABLE_DEPLOYMENT` != "0" +- `$CI_COMMIT_BRANCH` != "main" && `$ENFORCE_DEPLOYMENT` != "1" + +### Finalize + +Here we can do cleanups and remove some garbage the build process produces. + +#### Shut Down Docker Project + +We start a docker image for php with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +The following jobs must run before successfully: + +- [Build Site](#build-site) + +This job simply stops and removes the docker container, which are started for the build by: + +`docker compose down` + +### Release Stage + +This stage is responsible for creating a new release. + +#### Release Job + +This job extends the internal job [.merge](#merge). + +Rules, when the job should run: + +- `$CI_COMMIT_MESSAGE` contains `CREATE_RELEASE` +- we are on the `develop` branch + +Rules, when the job must not run: + +- if the reference is a `tag` +- `$CI_COMMIT_MESSAGE` contains `Merge tag` +- if it is triggered by config auto export (CAE) +- `$CHECKUPDATES` is set +- `$TRIGGERTASK` is set + + +### Internal Stages + +Here we have some internal stages, which cannot be executed alone. The purpose is to +collect repeating execution. + +#### .prepareaccess + +This internal job, you find in [mixins.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/general/-/blob/main/mixins.yml?ref_type=heads). + +Mainly, we care about getting access to: + +- the host `$CI_SERVER_HOST` with `ssh` via `$SSH_PRIVATE_KEY` +- GitLab Login via `$GITLAB_ACCESS_TOKEN` and `$GITLAB_PRIVATE_TOKEN` +- Docker configuration via `$DOCKER_AUTH_CONFIG` + + +#### .preparecomposerplugins + +This internal job, you find in [mixins.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/general/-/blob/main/mixins.yml?ref_type=heads). + +The following points will be prepared for [Composer](https://getcomposer.org/): + +- the version >2, if `$DOWNGRADE_COMPOSER` is not set +- the configuration `allow-plugins` +- the configuration `audit.abandoned`, depend on `$IGNORE_COMPOSER_ABANDONED_AUDIT` +- unsupported module versions, which are allowed to use + +#### .merge + +First we have to do some preparation. This is done in the `before_script` section. + +- [.prepareaccess](#prepareaccess) + +We start a docker image for php with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +We create a clean directory and clone the git project `$CI_PROJECT_PATH` +for branch `$SOURCE_BRANCH` from `$CI_SERVER_HOST` right into it. + +After that we execute the [merge](https://gitlab.lakedrops.com/docker/gitlab-drupal-ci/-/blob/main/bin/merge?ref_type=heads) +script, which does a few checks and use the `gitlab` tool to merge. + +Finally, we remove the directory we created above. + +#### .retry-for-system-issues + +This little job ste the amount of retries to "2", when: + +- runner_system_failure +- stuck_or_timeout_failure + +#### .cache_paths + +All internal jobs concerning cache will use this job here. It defines the paths: + +- .docker-init/ +- assets/ +- drush/ +- files/ +- keys/ +- settings/ +- vendor/ +- web/core/ +- web/libraries/ +- web/modules/contrib/ +- web/profiles/contrib/ +- web/sites/ +- web/themes/contrib/ +- .ahoy.yml +- .env +- docker-compose.yml diff --git a/mkdocs.yml b/mkdocs.yml index 75aab87f74f2c6f24d49d6e9458ddb1ae5f9f9b7..2a3414b5dbeb76adf82e179d5b140c60626ad0e9 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -266,6 +266,7 @@ nav: - GitLab: - gitlab/index.md - Drupal: gitlab/drupal/index.md + - Pipelines: gitlab/pipelines/index.md - Ansible: gitlab/ansible/index.md - DevOps: - ALM: