From dc8a385e834487ac2aff8633531adad280ef084b Mon Sep 17 00:00:00 2001 From: Daniel Speicher <daniel.speicher@lakedrops.com> Date: Sun, 21 Jul 2024 17:48:29 +0200 Subject: [PATCH] Document stages and jobs. --- docs/gitlab/drupal/index.md | 214 ++++++++++++++++++++++++++++-------- 1 file changed, 169 insertions(+), 45 deletions(-) diff --git a/docs/gitlab/drupal/index.md b/docs/gitlab/drupal/index.md index 644d489..b0c8cff 100644 --- a/docs/gitlab/drupal/index.md +++ b/docs/gitlab/drupal/index.md @@ -12,34 +12,180 @@ tags: This project contains a number of pre-configured tasks for GitLab CI/CD that allow to run very powerful pipelines which contain the following stages: -- .pre - - Sanity Checks - - Validate Environment +- .pre (predefined) +- validate - execute - - LakeDrops - - buildprod - - Build Prod Site - - build - - Build Prod Site NG - - Build Site - - Build Theme - - Download DB NG - - prepare - - Import DB NG - - test - - Test Code Style - - Test PHPUnit - - Test Backstop - - Test Cypress E2E - - finalize - - Shut Down Docker Project +- build (predefined) +- buildprod +- prepare +- test (predefined) +- deploy (predefined) - release +- finalize +- postprocess +- .post (predefined) + +We have a predefined `execute` job called `LakeDrops`, which looks like this: + +- execute + - LakeDrops + - buildprod + - Build Prod Site + - Build Prod Site NG + - build + - Build Site + - Build Theme + - Check 4 Outdated Packages + - Check 4 Security Vulnerability Advisories + - Download DB + - Download DB NG + - prepare + - Import DB + - Import DB NG + - Import Local DB + - Update DB + - test + - Test Code Style + - Test PHPUnit + - Test Backstop + - Test Cypress E2E + - finalize + - Shut Down Docker Project These pre-configured tasks can easily be included into your Drupal project. A GitLab runner on any host can then be configured to run those pipelines. The following chapters describe the necessary steps and also all the options available for configuration. +## Stages +The following chapters describes all stages and their jobs. The files: + +- [lakedrops.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/lakedrops.yml?ref_type=heads). +- [test_deploy.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/test-and-deploy.yml?ref_type=heads). + +### .pre +To prepare and check, if the environment is good to go, we have a `.pre` stage, which contains the +following jobs: + +#### Sanity Checks + +The sanity check gets important when you are using [Crowdsec](https://www.crowdsec.net/). +It contains two little scripts, that check: + +- if the `crowdsec.settings.yml` is in the config split and +- if it is not enabled in the `core.extensions.yml`. + +Otherwise, it is conflicting with you local settings, since `CrowdSec` +is a tool for production sites. + +Except: + +- Commit message contains "Merge tag" +- Commit message contains "Merge branch" +- When the job is triggered by config auto export (CAE) + +#### Validate Environment + +This job extends the `.prerequisites` in +[test_deploy.yml](https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/blob/main/test-and-deploy.yml?ref_type=heads). + +This job executes a script, where several [variables](#variables) are checked and set accordingly. + +Except: + +- Commit message contains "Merge tag" +- Commit message contains "Merge branch" +- When the job is triggered by config auto export (CAE) + +### Validate + +This stage is currently not officially used. + +### Execute + +This stage executes the build itself. + +### Build + +Here, we define jobs, which build the Drupal website and does some additional checks. This stage is +responsible for the database downloads as well. + +#### Download DB + +This job executes a script, that removes the SQL file `$]PROJECT_NAME}.sql`, if it exists already. +After that an Ansible script is executed, which dumbs the current Drupal database of the live stage. + +The file is saved as a build artefact, which contains `$]PROJECT_NAME}.sql`. + +Rules, when the job must not run: + +- `$DISABLE_DEPLOYMENT` != "0" +- `$DISABLE_DEPLOYMENT_LOCAL` != "1" +- `$DBREQUIRED` == "no" +- `$INITIALINSTALL` == "yes" +- `$TESTSDISABLED` == "yes" +- `$LOCALDBFILE` != "none" + +Rules, when the job should run: + +- the tests are enabled by `$DISABLE_CI_TESTS` != "1" + +#### Download DB NG + +This is very similar to [Download DB](#download-db), but does not use Ansible. +It uses Docker Compose instead to execute the `drush sql:dump` command +to dumb the current live database to the file `$]PROJECT_NAME}.sql`. + +The file is saved as a build artefact, which contains `$]PROJECT_NAME}.sql`. + +The rules are equal to [Download DB](#download-db). + +### Prepare + +This stage is used to prepare the system for testing. This stage is also used for internal purpose +and for proprietary tools. + +#### Import DB + +This job extends `.importdb`, which itself defines the +anchor [import_db_default](#import-db-default-anchor). + +The following jobs must run before successfully: + +- [Build Site]() +- [Download DB](#download-db) + +Rules, when the job must not run: + + - `$DISABLE_DEPLOYMENT` != "0" + - `$DISABLE_DEPLOYMENT_LOCAL` != "1" + - `$DBREQUIRED` == "no" + - `$INITIALINSTALL` == "yes" + - `$TESTSDISABLED` == "yes" + - `$LOCALDBFILE` != "none" + +Rules, when the job should run: + +- the tests are enabled by `$DISABLE_CI_TESTS` != "1" + +##### Import DB Default Anchor +We start a docker image for php with the specifies versions: + +- `PHP_MAJOR_VERSION` +- `PHP_MINOR_VERSION` + +A script copies the available SQL file into the PHP container, drops the current DB and imports the file. +If the variable `SKIPCONFIGIMPORT` is `no`, a `drush config-import` gets executed as well. After that the +`UPDATE_DB_COMMAND` gets called as well as a cache reset. + +#### Import DB + + +#### Import Local DB + + + + ## Configuration ### GitLab Runner @@ -203,7 +349,7 @@ you have to write the scripts for those two tasks yourselves. - `ENVIRONMENT_NAME`: the environment name for the GitLab UI. - `THEME_CSS_PATH`: the relative path to the theme's css path where the generated artefact can be found. -- `PHP_MAJOR_VERSION`: the major PHP version, defaults to 7. +- `PHP_MAJOR_VERSION`: the major PHP version, defaults to 10. - `PHP_MINOR_VERSION`: the minor PHP version, default to 4. - `COMPOSER_DOWNGRADE`: by default, composer 2 is being used. To use composer 1 instead, set this variable to 1. @@ -217,6 +363,8 @@ you have to write the scripts for those two tasks yourselves. test stage will be skipped. This is useful e.g. in a development environment where you push and run pipelines often but don't want to run the tests every single time. +- `$DISABLE_GITLAB_CI_TESTS`: if this variable is set to "1", no GitLab Tests will be + executed, see: [GitLab Templates](https://www.drupal.org/project/gitlab_templates). - `DISABLE_CI_TEST_BACKSTOP`: if this variable is set to any value, the visual regression tests with Backstop will be skipped. - `DISABLE_CI_TEST_BEHAT`: if this variable is set to any value, the Behat @@ -233,30 +381,6 @@ you have to write the scripts for those two tasks yourselves. completely and built from scratch. Can also be used as `[RESET_LOCALE]` in the Git commit message. -#### Stages and Tasks - -It's important to use exactly these task names as most of them are also being -used to define dependencies: - -- `validation` - - `Validate Environment` -- `buildprod` - - `Build Prod Site` -- `build` - - `Build Site` - - `Download DB` -- `prepare` - - `Import DB` - - `Update DB` - - `Build Theme` -- `test` - - `Test Code Style` - - `Test PHPUnit` - - `Test Behat` - - `Test Backstop` -- `deploy` - - `Deploy` - ## Usage This chapter is incomplete so far and needs more attention. -- GitLab