diff --git a/docs/installation/docker/converting.md b/docs/installation/docker/converting.md index 5f8583766..fcaf03004 100644 --- a/docs/installation/docker/converting.md +++ b/docs/installation/docker/converting.md @@ -1,87 +1,327 @@ -# Converting ISLE-DC to Site Template +# Converting to modern ISLE Site Template -The following instructions explain how to convert your ISLE-DC site into the Site Template style of docker-compose.yml. +This guide is intended to help convert ISLE-DC sites or ISLE Site Template sites that use docker compose profiles to manage a `dev` and `prod` profile for docker services. -## Development Instructions +Converting your docker-compose is completely optional. If you are happy with your setup, you can ignore this conversion guide. However, if you'd like to align with the supported community version, the following instructions explain how to convert your site. -ISLE-DC and the Site Template both use [isle-builkit](https://github.com/Islandora-Devops/isle-buildkit) images to spin up an Islandora site. Since the images are the same, you can transfer an ISLE-DC site to a Site Template setup by doing the following: +## Major changes -1. Follow the instructions in the [Site Template README](https://github.com/Islandora-Devops/isle-site-template) to create a new site from the Islandora Starter Site +If you're trying to decide whether you want to convert, these are the major changes from older versions of ISLE Site Template and ISLE-DC. If any of these address issues you've had with your setup, you may want to consider converting. - !!! note - Use the same TAG in your Site Template .env that you used in ISLE-DC, so you have the same set of containers. +### docker-compose.yml -2. Replace the Starter Site's composer.json and composer.lock files with yours, and install the correct Drupal modules using Composer +The major change from isle-dc is `docker-compose.yml` is now tracked in version control, so both development and production have the same `docker-compose.yml` and any changes are tracked in git. - !!! note - You may need to make some modifications to your composer.json file. Specifically, the `scripts` and `drupal-scaffold` sections may have changed. - - You can look at the [most recent starter site composer.json](https://github.com/Islandora-Devops/islandora-starter-site/blob/main/composer.json) for comparison. +This means that in ISLE-DC you had separate docker-compose.yml files for development and production, but in site template there is one file for both. - If you have custom modules and/or themes that are not installed via Composer you will need to install those as well. +Older versions of ISLE Site Template used docker compose profiles to manage a `dev` and `prod` profile for docker services. This has been consolidated into a single set of services that can be managed with `make` commands. -3. Compare your old Dockerfile with [the one in the Site Template repository](https://github.com/Islandora-Devops/isle-site-template/blob/main/drupal/Dockerfile). If you have made customizations, you may want to replicate them in the new Dockerfile. +### codebase now in ./drupal/rootfs/var/www/drupal -4. Backup your ISLE-DC site's Fedora, Drupal database, and public/private files. +ISLE-DC had the Drupal codebase in `./codebase`. The new location is `./drupal/rootfs/var/www/drupal` -5. Import your backups to the new Site Template site. +### Default to HTTP -6. Delete your Solr core, and regenerate new configs. +The default dev environment uses `http` rather than `https`. Running `https` in development environments has proven difficult to support for various host setups. For example, in production some institutions have TLS terminated in front of ISLE (having ISLE act as a backend on a frontend reverse proxy) so having better support around switching from TLS to HTTP seemed warranted. To help with this there are some new `make` commands to allow switching between http, tls-mkcert, and tls-letsencrypt easier. - !!! note - Site Template uses a different core name than ISLE-DC did +### Default domain -7. Commit and push your git repository so it is ready for production. +Using `http` also forced moving away from `islandora.dev`... HSTS rules in Google Chrome in particular do not allow accessing `*.dev` domains over http. So the default development domain is now `islandora.traefik.me` +### traefik labels to YML -## Production Instructions +Traefik labels have moved from docker compose YML and into [dynamic traefik templates](https://doc.traefik.io/traefik/reference/install-configuration/providers/others/file/) to make configuring traefik easier and more centralized. The traefik configuration is found in `./conf/traefik` -Once you have converted the development instance of your site, moving it to production requires the following: +### smart port allocation -1. Clone the git repository that you set up in the development instructions above +In development environments (i.e. `DEVELOPMENT_ENVIRONMENT==true`) `make up` tries to bring up ISLE on port 80/443 but if those ports are bound by some other process than the compose project it tries to bind to another port. The URL is then automatically opened in the web browser. This is mostly useful in a development environment where other sites might be occupying those ports. -2. Prepare your images/containers as described in the Site Template README +### make status - 1. Run the generate secrets script included with the site template, or copy your secrets from your existing site if you want them to be the same - 2. Add custom modules or themes, if you have some that are not included in your composer.json - 3. Build your Drupal container with `docker compose --profile prod build` - 4. Pull in the remaining containers with `docker compose --profile prod pull --ignore-buildable --ignore-pull-failures` - 5. Start the containers with `docker compose --profile prod up -d` +Added a command to help identify misconfigured docker-compose.yml + .env combinations. -3. Import the backups from your ISLE-DC site that you made in the development instructions above - -4. Delete your Solr core, and regenerate new configs. +### URLs -## Converting Your docker-compose.yml +The Site Template handles the URLs for microservices differently than ISLE-DC did. You will need to follow the instructions in the Site Template README to allow access to your microservices as a subdomain. For example, `solr.mydomain.com`. -The main difference between ISLE-DC and the Site Template is the way the docker-compose.yml file is generated. In ISLE-DC we generate it based on your .env variables and a make command, but in Site Template it is ready to go out of the box. +!!! Note "Subdomains" + If you are already using a subdomain and can’t use URLs like `solr.mysubdomain.mydomain.com`, you can change this in the Traefik section of the docker-compose.yml file, by changing the aliases from `solr.{DOMAIN}` to something else, like `solr-{DOMAIN}`. You will also need to change this in a few other spots in your docker-compose.yml. You can find them by searching for `.{DOMAIN}` -This means that in ISLE-DC you had separate docker-compose.yml files for development and production, but in site template there is one file for both, which contains instructions for a “dev” and “prod” profile. The end result is the same set of containers, but instead of running `docker compose up` you would run `docker compose --profile dev up` +## Conversion process + +!!! note + This doc assummes you default git branch is called `main`. Adjust the commands below if that's not the case + +### 1. Bring in the new config + +There are two different options to convert docker-compose. If you're converting from ISLE-DC, choose 1.a. If you're converting from the older version of ISLE Site Template, follow 1.b + +#### 1.a ISLE-DC + +1. Bring in the latest isle-site-template: +``` +git clone https://github.com/islandora-devops/isle-site-template +cd isle-site-template +``` +2. Copy your drupal codebase into it +``` +rm -rf drupal/rootfs/var/www/drupal +cp -r /path/to/isle/dc/codebase drupal/rootfs/var/www/drupal +``` +3. Update `docker-compose.yml` doing a find replace. +Find: `drupal-public-files` +Replace: `drupal-sites-data` + +Find: `DRUPAL_DEFAULT_SOLR_CORE: default` +Replace: `DRUPAL_DEFAULT_SOLR_CORE=ISLANDORA` +4. Copy `.env` +``` +cp -r /path/to/isle/dc/codebase/.env . +``` +5. Update `.env` +Find: `TAG=` +Replace: `ISLANDORA_TAG=` + +These deprecated environment variables can be removed from your `.env` that was copied from your ISLE-DC install + +??? note + + ``` + ACME_KEY_TYPE + ACTIVEMQ_MEMORY_LIMIT + ACTIVEMQ_PORT + ALPACA_FITS_TIMEOUT + ALPACA_HOMARUS_TIMEOUT + ALPACA_HOUDINI_TIMEOUT + ALPACA_MEMORY_LIMIT + ALPACA_OCR_TIMEOUT + BLAZEGRAPH_MEMORY_LIMIT + BLAZEGRAPH_PORT + CANTALOUPE_DELEGATE_SCRIPT_ENABLED + CANTALOUPE_DELEGATE_SCRIPT_PATHNAME + CANTALOUPE_HTTPSOURCE_LOOKUP_STRATEGY + CANTALOUPE_MEMORY_LIMIT + CODEBASE_PACKAGE + COMPOSE_HTTP_TIMEOUT + CRAYFITS_MEMORY_LIMIT + CUSTOM_IMAGE_NAME + CUSTOM_IMAGE_NAMESPACE + CUSTOM_IMAGE_TAG + DISABLE_SYN + DRUPAL_DATABASE_SERVICE + DRUPAL_INSTALL_PROFILE + DRUPAL_MEMORY_LIMIT + ENVIRONMENT + EXPOSE_ACTIVEMQ + EXPOSE_BLAZEGRAPH + EXPOSE_CANTALOUPE + EXPOSE_DRUPAL + EXPOSE_FEDORA + EXPOSE_MYSQL + EXPOSE_POSTGRES + EXPOSE_SOLR + EXPOSE_TRAEFIK_DASHBOARD + FCREPO_DATABASE_SERVICE + FCREPO_MEMORY_LIMIT + FEDORA_6 + FEDORA_PORT + FITS_MEMORY_LIMIT + HOMARUS_MEMORY_LIMIT + HOUDINI_MEMORY_LIMIT + HYPERCUBE_MEMORY_LIMIT + IDE_MEMORY_LIMIT + INCLUDE_ETCD_SERVICE + INCLUDE_TRAEFIK_SERVICE + INCLUDE_WATCHTOWER_SERVICE + INSTALL_EXISTING_CONFIG + MARIADB_MEMORY_LIMIT + MILLINER_MEMORY_LIMIT + MYSQL_PORT + PHP_MAJOR_VERSION + PHP_MAX_EXECUTION_TIME + PHP_MEMORY_LIMIT + PHP_MINOR_VERSION + PHP_POST_MAX_SIZE + PHP_UPLOAD_MAX_FILESIZE + POSTGRES_PORT + PROJECT_DRUPAL_DOCKERFILE + PUBLIC_FILES_TAR_DUMP_PATH + RECAST_MEMORY_LIMIT + RESTART_POLICY + SITE + SOLR_MEMORY_LIMIT + SOLR_PORT + TAG + TRAEFIK_DASHBOARD_PORT + TRAEFIK_LOG_LEVEL + TRAEFIK_MEMORY_LIMIT + USE_ACME + USE_SECRETS + WATCHTOWER_MEMORY_LIMIT + ``` +6. Bring in the changes to your ISLE-DC git repo + +``` +rm -rf .git +cp -r path/to/isle/dc/.git . +git checkout -B convert-site-template +``` + +#### 1.b ISLE Site Template + +In your dev environment, bring in the new config from ISLE Site Template + +``` +cd path/to/your/site/template +git checkout -B convert-site-template +git remote add ist https://github.com/islandora-devops/isle-site-template +git fetch ist +git checkout ist/main -- conf Makefile sample.env scripts docker-compose.dev.yml +docker compose --profile dev down +make down +``` + +### 2 - Remove compose profiles (ISLE Site Template only) + +If you're converting from ISLE-DC you can skip this step. + +You have two options to convert docker-compose. Choose either 2.a or 2.b + +#### 2.a - Vanilla overwrite + +If you've made no changes to docker-compose.yml since you made your own copy of isle-site-template you can simply overwrite the docker-compose.yml with the latest version + +``` +git checkout ist/main -- docker-compose.yml +``` + +#### 2.b - Attempt to merge + +If you've made changes to docker-compose.yml since copying the repo, you can attempt to convert it using `sitectl`. Install sitectl and run the `sitectl isle migrate merge-compose-profiles` command + +``` +brew tap libops/homebrew https://github.com/libops/homebrew +brew install libops/homebrew/sitectl-isle +sitectl isle migrate merge-compose-profiles -i docker-compose.yml -o docker-compose.yml -f +``` + +This will overwrite your `docker-compose.yml` with the new profiles-less version. + +### 3 - Review + +Very carefully review your changes locally + +``` +git diff +``` + +Or if you prefer viewing the diff in your git provider + +``` +git add . +git commit -m "Removing profiles" +git push origin convert-site-template +``` + +Ensure your services have the same environment variables, secrets, volumes your old services had. + +Familiarize yourself with the new make command helpers + +``` +make help +``` + +#### Environment Variables + +Ensure the new values you see in `sample.env` are set in your `.env`. Namely -### Environment Variables +``` +URI_SCHEME=http +TLS_PROVIDER="self-managed" +ACME_URL=https://acme-v02.api.letsencrypt.org/directory +ACME_EMAIL=postmaster@example.com +``` -You should compare the `environment` sections of your docker-compose.yml files. You may have some variables set in your ISLE-DC docker-compose.yml that will need to be added to the docker-compose.yml in the new Site Template site. +In the git diff, compare the `environment` sections of your docker-compose.yml files. You may have some variables set in your ISLE-DC docker-compose.yml that will need to be added to the docker-compose.yml in the new Site Template site. For example, the [ISLE Buildkit Nginx README](https://github.com/Islandora-Devops/isle-buildkit/blob/main/nginx/README.md) lists the available variables for Nginx that you may have changed on your Drupal containers to do things like increase the timeout time for PHP or the max POST size PHP will accept. -### URLs - -The Site Template also handles the URLs for microservices differently. You will need to follow the instructions in the Site Template README to allow access to your microservices as a subdomain. For example, `solr.mydomain.com`. - -!!! Note "Subdomains" - If you are already using a subdomain and can’t use URLs like `solr.mysubdomain.mydomain.com`, you can change this in the Traefik section of the docker-compose.yml file, by changing the aliases from `solr.{DOMAIN}` to something else, like `solr-{DOMAIN}`. You will also need to change this in a few other spots in your docker-compose.yml. You can find them by searching for `.{DOMAIN}` +And then adjust sample.env as needed. -## Converting the Makefile +#### Converting the Makefile Because ISLE-DC and Site Template use the same containers, much of what is in an ISLE-DC Makefile will work within the site template environment. The main differences that need to be addressed are with the way new sites are built, and the name of the containers. Commands to build a site, like `make starter` and `make production` are not necessary with the Site Template, so they can be removed from your Makefile. -Many of the other commands specify which containers to run commands in, for example `docker compose exec drupal`, but in the Site Template environment you need to specify -dev or -prod, so that would be changed to `docker compose exec drupal-dev` or `docker compose exec drupal-prod`. +Many of the other commands specify which containers to run commands in, for example `docker compose exec drupal`. + +#### Other Customizations + +If you have modified your repository, those modifications will need to be evaluated on a case-by-case basis. This includes editing the Dockerfile and environment variables mentioned above, as well as any other customizations you may have made, such as to settings.php, robots.txt, nginx configs, etc. + +### 4 - Test + +If you want to keep running TLS in your dev environment you can run this make helper to switch to `https`: + +``` +make traefik-https-mkcert +``` + +Otherwise, be sure your `DOMAIN` value in `.env` is `islandora.traefik.me` and be aware your dev site is available at http://islandora.traefik.me + +Now, bring up your services. In a development environment, after running `make up` your ISLE site should be available at `${URI_SCHEME}://${DOMAIN}` (the actual value will print after running make up depending on your settings). + +!!! warning + + If you don't like the new setup, it's not working, or the changes didn't apply cleanly, you can [reach out for help](https://www.islandora.ca/contact-us#comms-channels) (in the #isle slack channel) or simply delete the git branch that was created to revert the changes. + + This will delete any config settings made in previous steps + ``` + make down + git checkout main + git branch -D convert-site-template + ``` + +### 5 - Save changes and cleanup + +If you like this new setup and all looks well, merge the `convert-site-template` git branch into main + +``` +git checkout main +git merge convert-site-template +git push origin main +``` + +### 6 - Deploy to production + +If you committed the changes, deploy to production by: + +Pull down the changes + +``` +git pull origin main +``` + +If you're using letsencrypt run this make helper to switch to `https` in your prod environment: + +``` +make traefik-https-letsencrypt +``` -## Other Customizations +If you're not using letsencrypt but instead are using your own TLS certs, make sure `.env` has -If you have modified your ISLE-DC repository, those modifications will need to be evaluated on a case-by-case basis. This includes editing the Dockerfile and environment variables mentioned above, as well as any other customizations you may have made, such as to settings.php, robots.txt, nginx configs, etc. +``` +URI_SCHEME=https +TLS_PROVIDER="self-managed" +``` +Next, bring down the site, pull in the changes, and bring everything back up +``` +make down +docker compose --profile prod down +make up +``` +Your site should be online now using the new setup! diff --git a/docs/installation/docker/docker-introduction.md b/docs/installation/docker/docker-introduction.md index 0b269fe68..839c248d9 100644 --- a/docs/installation/docker/docker-introduction.md +++ b/docs/installation/docker/docker-introduction.md @@ -12,4 +12,4 @@ ISLE's architecture using [Docker](https://www.docker.com/) separates out the "s ISLE is a suite of Docker containers that run the various components of Islandora: drupal, fedora, solr, alpaca, crayfish, matomo, etc. The individual containers are created (and automatically pushed to [Docker Hub](https://hub.docker.com/u/islandora)) by [ISLE BuildKit](https://github.com/Islandora-Devops/isle-buildkit). -In order to deploy the containers, however, you need to use a container orchestration tool. The ISLE project provides tools for running and maintaining the containers using docker-compose with [ISLE Docker Compose](https://github.com/Islandora-Devops/isle-dc) or [ISLE Site Template](https://github.com/Islandora-Devops/isle-site-template). +In order to deploy the containers, however, you need to use a container orchestration tool. The ISLE project provides tools for running and maintaining the containers using docker-compose with [ISLE Site Template](https://github.com/Islandora-Devops/isle-site-template). diff --git a/docs/installation/docker/docker-prereq.md b/docs/installation/docker/docker-prereq.md index 6e3f6d3aa..d80d42164 100644 --- a/docs/installation/docker/docker-prereq.md +++ b/docs/installation/docker/docker-prereq.md @@ -38,7 +38,7 @@ Docker Compose is a tool to simplify the process of running multiple Docker cont ### GNU Make -Make allows us to define commands that simplify installing and maintaining our Islandora site. For a complete list of available commands see the Makefile included with ISLE-DC. +Make allows us to define commands that simplify installing and maintaining our Islandora site. For a complete list of available commands see the Makefile included with ISLE Site Template. ### Composer diff --git a/docs/installation/docker/isle-dc/docker-available-commands.md b/docs/installation/docker/isle-dc/docker-available-commands.md index d037f0a1a..772219dad 100644 --- a/docs/installation/docker/isle-dc/docker-available-commands.md +++ b/docs/installation/docker/isle-dc/docker-available-commands.md @@ -1,5 +1,8 @@ # Available Commands +!!! warning + isle-dc is deprecated in favor of [isle-site-template](/documentation/installation/docker/site-template/site-template/). + There's a lot of useful commands available to you from within the `isle-dc` folder. ## Rebuild docker-compose.yml diff --git a/docs/installation/docker/isle-dc/docker-available-configuration.md b/docs/installation/docker/isle-dc/docker-available-configuration.md index c9dda6d7b..adc9df1b8 100644 --- a/docs/installation/docker/isle-dc/docker-available-configuration.md +++ b/docs/installation/docker/isle-dc/docker-available-configuration.md @@ -1,5 +1,8 @@ # Available Configuration +!!! warning + isle-dc is deprecated in favor of [isle-site-template](/documentation/installation/docker/site-template/site-template/). + ISLE Docker Compose has a single configuration file, `.env`. Within it, here is all the values you can set and their effects. This list is subjectively sorted in order of importance and usefulness. diff --git a/docs/installation/docker/isle-dc/docker-basic-usage.md b/docs/installation/docker/isle-dc/docker-basic-usage.md index 112716fa1..8c2e091ec 100644 --- a/docs/installation/docker/isle-dc/docker-basic-usage.md +++ b/docs/installation/docker/isle-dc/docker-basic-usage.md @@ -1,5 +1,8 @@ # Basic Usage +!!! warning + isle-dc is deprecated in favor of [isle-site-template](/documentation/installation/docker/site-template/site-template/). + After you've finished installing Islandora using ISLE, here's some useful information to keep close at hand about running your site. diff --git a/docs/installation/docker/isle-dc/docker-custom.md b/docs/installation/docker/isle-dc/docker-custom.md index 737f7022b..b83e44214 100644 --- a/docs/installation/docker/isle-dc/docker-custom.md +++ b/docs/installation/docker/isle-dc/docker-custom.md @@ -1,5 +1,8 @@ # Installing a Production/Staging Server +!!! warning + isle-dc is deprecated in favor of [isle-site-template](/documentation/installation/docker/site-template/site-template/). + If you are deploying Islandora on a server that is publicly accessible, there are extra precautions you should take for performance and security reasons. diff --git a/docs/installation/docker/isle-dc/docker-local.md b/docs/installation/docker/isle-dc/docker-local.md index 64f55cd37..7670a6cd0 100644 --- a/docs/installation/docker/isle-dc/docker-local.md +++ b/docs/installation/docker/isle-dc/docker-local.md @@ -1,5 +1,8 @@ # Installing a Development or Demo Server +!!! warning + isle-dc is deprecated in favor of [isle-site-template](/documentation/installation/docker/site-template/site-template/). + When developing locally, your Drupal site resides in your `isle-dc/codebase` folder and is bind-mounted into your Drupal container. This lets you update code using the IDE of your choice on your host machine, and the changes are automatically reflected on the Drupal container. diff --git a/docs/installation/docker/isle-dc/docker-maintain-drupal.md b/docs/installation/docker/isle-dc/docker-maintain-drupal.md index c4deba8a5..bc925b63b 100644 --- a/docs/installation/docker/isle-dc/docker-maintain-drupal.md +++ b/docs/installation/docker/isle-dc/docker-maintain-drupal.md @@ -1,5 +1,8 @@ # Maintaining Your Drupal Site +!!! warning + isle-dc is deprecated in favor of [isle-site-template](/documentation/installation/docker/site-template/site-template/). + Drupal has a dedicated security team, and regularly produces updates to address security issues that are discovered. You should always keep your Drupal site up to date so that you are protected against known vulnerabilities. Updating Drupal can be intimidating, but we have diff --git a/docs/installation/docker/isle-dc/docker-maintain-isle.md b/docs/installation/docker/isle-dc/docker-maintain-isle.md index 675b626f2..b14f40abb 100644 --- a/docs/installation/docker/isle-dc/docker-maintain-isle.md +++ b/docs/installation/docker/isle-dc/docker-maintain-isle.md @@ -1,5 +1,8 @@ # Maintaining Your ISLE Infrastructure +!!! warning + isle-dc is deprecated in favor of [isle-site-template](/documentation/installation/docker/site-template/site-template/). + You will regularly be updating your Drupal site as security patches and module updates are released. Less often, you will also need to update the rest of your Islandora installation. ISLE makes this easy. In fact, it was specifically designed to streamline this process. diff --git a/docs/installation/docker/isle-dc/docker-troubleshooting.md b/docs/installation/docker/isle-dc/docker-troubleshooting.md index a38c83c39..b0d7ca2f0 100644 --- a/docs/installation/docker/isle-dc/docker-troubleshooting.md +++ b/docs/installation/docker/isle-dc/docker-troubleshooting.md @@ -1,5 +1,8 @@ # Troubleshooting +!!! warning + isle-dc is deprecated in favor of [isle-site-template](/documentation/installation/docker/site-template/site-template/). + ### Make command not found If you get the error `bash: make: command not found` then you need to install GNU make. Run `sudo apt update` and `sudo apt install make` to install. diff --git a/docs/installation/docker/site-template/backup.md b/docs/installation/docker/site-template/backup.md index 51d4686b1..884bb951e 100644 --- a/docs/installation/docker/site-template/backup.md +++ b/docs/installation/docker/site-template/backup.md @@ -1,23 +1,23 @@ # Backups & Transferring Data Between Sites -The following instructions describe how to back up and restore an islandora site. This is also used to transfer data between a development and production site, or to sync a staging site with a production site. +The following instructions describe how to back up and restore an Islandora site. This is also used to transfer data between a development and production site, or to sync a staging site with a production site. -Note that containers are named with a suffix of -dev or -prod depending on your environment. For example, backing up your development site’s Drupal database requires running the command on the drupal-dev container, but backing up the production site’s database requires running the command on the drupal-prod container. The following instructions are mostly written for development sites, but the same commands will work on production containers. +Container names are based on the service names defined in your `docker-compose.yml` file. For example, the Drupal container is typically named `drupal`, the MariaDB container is `mariadb`, etc. The following instructions use these standard service names. Before attempting the following commands, you should familiarize yourself with [running commands inside a container](/documentation/installation/docker/site-template/containers.md) and with [docker compose cp](https://docs.docker.com/reference/cli/docker/compose/cp/) ## Drupal Configuration -The typical use case of this is to export your drupal configuration files out of your development site’s Drupal database and onto the host machine. This process lets you check your configuration into your git repository so you can import it into your production site. +The typical use case of this is to export your Drupal configuration files out of your development site's Drupal database and onto the host machine. This process lets you check your configuration into your git repository so you can import it into your production site. ### Export -To export your development config run: +To export your config run: ``` -docker compose exec -T drupal-dev drush -l default config:export -y +docker compose exec drupal drush config:export -y ``` Then commit and push your git repo so the new config files are included. @@ -29,15 +29,15 @@ To import your config to a production or staging site you first need to `git pul ``` git pull -docker compose --profile prod build -docker compose --profile prod down -docker compose --profile prod up -d +docker compose build +docker compose down drupal +docker compose up -d ``` Once this is done, your config files will be included in the container. You can then import them by running ``` -docker compose exec -T drupal-prod drush -l default config:import -y +docker compose exec drupal drush config:import -y ``` @@ -53,13 +53,17 @@ These instructions can also be used to move your content from production to stag Backing up the Drupal database involves doing a database dump in the drupal container: ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'mysqldump -u ${DRUPAL_DEFAULT_DB_USER} -p${DRUPAL_DEFAULT_DB_PASSWORD} -h mariadb ${DRUPAL_DEFAULT_DB_NAME} > /tmp/dump.sql' +docker compose exec drupal \ + bash -lc "drush sql-dump -y \ + --skip-tables-list=cache,cache_*,watchdog \ + --structure-tables-list=cache,cache_*,watchdog \ + --result-file=/tmp/drupal.sql" ``` and then copying that dump from the container to the host machine: ``` -docker compose cp drupal-dev:/tmp/dump.sql [path/on/host/dump.sql] +docker compose cp drupal:/tmp/drupal.sql [path/on/host/drupal.sql] ``` @@ -68,19 +72,19 @@ docker compose cp drupal-dev:/tmp/dump.sql [path/on/host/dump.sql] Restoring from a database dump requires copying the dump file into the drupal container: ``` -docker compose cp [path-to/dump.sql] drupal-dev:/tmp/dump.sql +docker compose cp [path-to/drupal.sql] drupal:/tmp/drupal.sql ``` Then replacing the database with the dump: ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'chown root:root /tmp/dump.sql && mysql -u ${DRUPAL_DEFAULT_DB_USER} -p${DRUPAL_DEFAULT_DB_PASSWORD} -h mariadb ${DRUPAL_DEFAULT_DB_NAME} < /tmp/dump.sql' +docker compose exec drupal drush sqlq --debug --file /tmp/drupal.sql' ``` And finally, rebuilding the Drupal cache: ``` -docker compose exec drupal-dev drush cr +docker compose exec drupal drush cr ``` @@ -96,13 +100,16 @@ These instructions can also be used to move your content from production to stag Drupal public files can be compressed to a tgz file in the Drupal container: ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'tar zcvf /tmp/public-files.tgz -C /var/www/drupal/web/sites/default/files .' +docker compose exec drupal \ + drush archive:dump \ + --files \ + --destination=/tmp/files.tar.gz ``` Then copied to the host machine: ``` -docker compose cp drupal-dev:/tmp/public-files.tgz [path/on/host/public-files.tgz] +docker compose cp drupal:/tmp/files.tar.gz [path/on/host/files.tar.gz] ``` @@ -111,20 +118,22 @@ docker compose cp drupal-dev:/tmp/public-files.tgz [path/on/host/public-files.t Drupal files can be restored from a tgz file by copying them into the Drupal container: ``` -docker compose cp public-files.tgz drupal-dev:/tmp/public-files.tgz +docker compose cp files.tar.gz drupal:/tmp/files.tar.gz ``` Then placed in the correct directory inside the Drupal volume: ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'tar zxvf /tmp/public-files.tgz -C /var/www/drupal/web/sites/default/files && chown -R nginx:nginx /var/www/drupal/web/sites/default/files && rm /tmp/public-files.tgz' +docker compose exec drupal tar zxvf /tmp/files.tar.gz -C /var/www/drupal/web/sites/default/files +docker compose exec drupal chown -R nginx:nginx /var/www/drupal/web/sites/default/files +docker compose exec drupal rm /tmp/files.tar.gz ``` !!! note This will overwrite existing files if they have the same filename, but does not remove existing files otherwise. If you want to make sure that the public files directory does not contain anything but the newly imported files, you will want to empty the directory before copying the new files in. - + ``` - docker compose exec -T drupal-dev with-contenv bash -lc 'rm -r /var/www/drupal/web/sites/default/files/*' + docker compose exec drupal with-contenv bash -lc 'rm -r /var/www/drupal/web/sites/default/files/*' ``` @@ -133,19 +142,18 @@ docker compose exec -T drupal-dev with-contenv bash -lc 'tar zxvf /tmp/public-fi These instructions can also be used to move your content from production to staging or development. - ### Back Up -Drupal public files can be compressed to a tgz file in the Drupal container: +Drupal private files can be compressed to a tgz file in the Drupal container: ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'tar zcvf /tmp/private-files.tgz -C /var/www/drupal/private .' +docker compose exec drupal tar zcvf /tmp/private-files.tgz -C /var/www/drupal/private . ``` Then copied to the host machine: ``` -docker compose cp drupal-dev:/tmp/private-files.tgz [path/on/host/private-files.tgz] +docker compose cp drupal:/tmp/private-files.tgz [path/on/host/private-files.tgz] ``` @@ -154,20 +162,22 @@ docker compose cp drupal-dev:/tmp/private-files.tgz [path/on/host/private-files. Drupal files can be restored from a tgz file by copying them into the Drupal container: ``` -docker compose cp private-files.tgz drupal-dev:/tmp/private-files.tgz +docker compose cp private-files.tgz drupal:/tmp/private-files.tgz ``` Then placed in the correct directory inside the Drupal volume: ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'tar zxvf /tmp/private-files.tgz -C /var/www/drupal/private && chown -R nginx:nginx /var/www/drupal/private && rm /tmp/private-files.tgz' +docker compose exec drupal tar zxvf /tmp/private-files.tgz -C /var/www/drupal/private +docker compose exec drupal chown -R nginx:nginx /var/www/drupal/private +docker compose exec drupal rm /tmp/private-files.tgz ``` !!! note This will overwrite existing files if they have the same filename, but does not remove existing files otherwise. If you want to make sure that the private files directory does not contain anything but the newly imported files, you will want to empty the directory before copying the new files in. - + ``` - docker compose exec -T drupal-dev with-contenv bash -lc 'rm -r /var/www/drupal/private/*' + docker compose exec drupal with-contenv bash -lc 'rm -r /var/www/drupal/private/*' ``` @@ -183,13 +193,13 @@ Fedora 6 uses a file structure called [OCFL](https://ocfl.io/) to store files an To back up Fedora we create a .tgz file from the ocfl-root directory in the Fedora container: ``` -docker compose exec -T fcrepo-dev with-contenv bash -lc 'tar zcvf /tmp/fcrepo-export.tgz -C /data/home/data/ocfl-root/ .' +docker compose exec fcrepo tar zcvf /tmp/fcrepo-export.tgz -C /data/home/data/ocfl-root/ . ``` Then we copy that file to the host machine: ``` -docker compose cp fcrepo-dev:/tmp/fcrepo-export.tgz [path/on/host/fcrepo-export.tgz] +docker compose cp fcrepo:/tmp/fcrepo-export.tgz [path/on/host/fcrepo-export.tgz] ``` @@ -198,30 +208,32 @@ docker compose cp fcrepo-dev:/tmp/fcrepo-export.tgz [path/on/host/fcrepo-export. To restore our fedora database we need to copy the backup into the Fedora container: ``` -docker compose cp [path-to/fcrepo-export.tgz] fcrepo-dev:/tmp/fcrepo-export.tgz +docker compose cp [path-to/fcrepo-export.tgz] fcrepo:/tmp/fcrepo-export.tgz ``` Empty the existing ocfl-root directory: ``` -docker compose exec -T fcrepo-dev bash -lc 'rm -r /data/home/data/ocfl-root/*' +docker compose exec fcrepo bash -lc 'rm -r /data/home/data/ocfl-root/*' ``` Put our backup files in the ocfl-root directory: ``` -docker compose exec -T fcrepo-dev with-contenv bash -lc 'tar zxvf /tmp/fcrepo-export.tgz -C /data/home/data/ocfl-root/ && chown -R tomcat:tomcat /data/home/data/ocfl-root/ && rm /tmp/fcrepo-export.tgz' +docker compose exec fcrepo tar zxvf /tmp/fcrepo-export.tgz -C /data/home/data/ocfl-root/ +docker compose exec fcrepo chown -R tomcat:tomcat /data/home/data/ocfl-root/ +docker compose exec fcrepo rm /tmp/fcrepo-export.tgz ``` Drop the existing Fedora database: ``` -docker compose exec -T mariadb-dev bash -lc 'mysql -e "drop database fcrepo;"' +docker compose exec mariadb mariadb -e "drop database fcrepo;" ``` Restart Fedora to reinitialize the database from the ocfl-root directory: ``` -docker compose --profile dev restart fcrepo-dev +docker compose restart fcrepo ``` !!! note diff --git a/docs/installation/docker/site-template/containers.md b/docs/installation/docker/site-template/containers.md index b449dad69..25f0c357d 100644 --- a/docs/installation/docker/site-template/containers.md +++ b/docs/installation/docker/site-template/containers.md @@ -4,22 +4,29 @@ When using Docker, we often have to run commands inside the Docker containers. F ## Running Commands in the Containers -If you want to work inside a container, you can use Docker Compose to run bash from inside the container. You can get a bash shell inside the Drupal container by running +If you want to work inside a container, you can use Docker Compose to run bash from inside the container. You can be dropped into a bash shell inside the Drupal container by running -`docker compose exec drupal-dev with-contenv bash` +`docker compose exec -it drupal with-contenv bash` Or you can run your command from the host machine by running -`docker compose exec -T drupal-dev with-contenv bash -lc 'YOUR COMMAND'` +`docker compose exec drupal YOUR COMMAND` For example, if you want to perform a Drush cache rebuild you could run -`docker compose exec -T drupal-dev with-contenv bash -lc 'drush cr'` +`docker compose exec drupal drush cr` + +If your commands rely on bash expression like wildcard expandsions or referencing variables defined inside the container you'll need to wrap you command in `bash -lc ""` + +``` +docker compose exec drupal bash -lc "ls **/*.php" +``` + !!! Note "Container Names" - The container name drupal-dev is the ‘services’ name as set in your docker-compose.yml. To access the drupal container in production you would replace drupal-dev with drupal-prod. + The container name `drupal` is the 'service' name as set in your docker-compose.yml. - You can also access other containers in this way, by replacing drupal-dev with their service name. For example, mariadb-dev or solr-prod. + You can also access other containers in this way, by replacing `drupal` with their service name. For example, `mariadb`, `solr`, `fcrepo`, etc. For more information, see the documentation on [docker compose exec](https://docs.docker.com/reference/cli/docker/compose/exec/) @@ -27,20 +34,20 @@ For more information, see the documentation on [docker compose exec](https://doc In some of the commands throughout the ISLE Site Template documentation you will see references to variables like `${DRUPAL_DEFAULT_SITE_URL}`. These are variables that are provided in the containers by the s6-overlay in [Islandora Buildkit](https://github.com/Islandora-Devops/isle-buildkit). -If you want to see their values, you can run +If you want to see their values, you can run ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'echo ${DRUPAL_DEFAULT_SITE_URL}' +docker compose exec drupal echo ${DRUPAL_DEFAULT_SITE_URL}' ``` !!! Note "with-contenv" - Adding `with-contenv` to your commands gives you access to the s6-overlay environment variables. If you don’t need to use these variables, the commands above can be simplified to - - `docker compose exec drupal-dev bash` + Adding `with-contenv` to your commands gives you access to the s6-overlay environment variables. If you don't need to use these variables, the commands above can be simplified to + + `docker compose exec drupal bash` and - `docker compose exec -T drupal-dev -lc 'YOUR COMMAND'` + `docker compose exec -T drupal bash -lc 'YOUR COMMAND'` ## Helpful Commands @@ -53,7 +60,7 @@ You can read logs for a container with: docker compose logs service_name ``` -For example, to read nginx logs for Drupal, use `docker compose logs drupal-dev`. +For example, to read nginx logs for Drupal, use `docker compose logs drupal`. If you don't know what you're looking for exactly, you can turn on the fire hose and look through all logs by dropping the service name and simply using: @@ -66,13 +73,13 @@ docker compose logs You can reindex Solr through the Drupal admin page, or via drush commands by queuing for reindex: ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} search-api-reindex' +docker compose exec drupal drush search-api-reindex ``` then triggering the reindex: ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} search-api-index' +docker compose exec drupal drush search-api-index ``` ### Reindex Fedora @@ -84,19 +91,27 @@ docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/d You can reindex all of your Drupal data into Fedora, by running the following commands: ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} vbo-exec non_fedora_files emit_file_event --configuration="queue=islandora-indexing-fcrepo-file-external&event=Update"' +docker compose exec drupal \ + drush vbo-exec non_fedora_files emit_file_event \ + --configuration="queue=islandora-indexing-fcrepo-file-external&event=Update" ``` ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} vbo-exec all_taxonomy_terms emit_term_event --configuration="queue=islandora-indexing-fcrepo-content&event=Update"' +docker compose exec drupal \ + drush vbo-exec all_taxonomy_terms emit_term_event \ + --configuration="queue=islandora-indexing-fcrepo-content&event=Update" ``` ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} vbo-exec content emit_node_event --configuration="queue=islandora-indexing-fcrepo-content&event=Update"' +docker compose exec drupal \ + drush vbo-exec content emit_node_event \ + --configuration="queue=islandora-indexing-fcrepo-content&event=Update" ``` ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} vbo-exec media emit_media_event --configuration="queue=islandora-indexing-fcrepo-media&event=Update"' +docker compose exec drupal \ + drush vbo-exec media emit_media_event \ + --configuration="queue=islandora-indexing-fcrepo-media&event=Update" ``` ### Reindex Blazegraph @@ -108,13 +123,19 @@ docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/d You can reindex all of your Drupal data into Blazegraph, by running the following commands: ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} vbo-exec all_taxonomy_terms emit_term_event --configuration="queue=islandora-indexing-triplestore-index&event=Update"' +docker compose exec drupal \ + drush vbo-exec all_taxonomy_terms emit_term_event \ + --configuration="queue=islandora-indexing-triplestore-index&event=Update" ``` ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} vbo-exec content emit_node_event --configuration="queue=islandora-indexing-triplestore-index&event=Update"' +docker compose exec drupal \ + drush vbo-exec content emit_node_event \ + --configuration="queue=islandora-indexing-triplestore-index&event=Update" ``` ``` -docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} vbo-exec media emit_media_event --configuration="queue=islandora-indexing-triplestore-index&event=Update"' +docker compose exec drupal \ + drush vbo-exec media emit_media_event \ + --configuration="queue=islandora-indexing-triplestore-index&event=Update" ``` diff --git a/docs/installation/docker/site-template/customizations.md b/docs/installation/docker/site-template/customizations.md index 6e6ecb85f..31bca6cea 100644 --- a/docs/installation/docker/site-template/customizations.md +++ b/docs/installation/docker/site-template/customizations.md @@ -16,25 +16,41 @@ If your server uses iptables, you can block a range of IP addresses using the fo ### TLS Settings -Traefik allows you to modify security settings by doing things like setting a minimum TLS version or specifying cipher suites. Isle Site Template ships with two versions of the TLS settings file, one [for development](https://github.com/Islandora-Devops/isle-site-template/blob/main/dev-tls.yml), and one [for production](https://github.com/Islandora-Devops/isle-site-template/blob/main/prod-tls.yml). +Traefik allows you to modify security settings by doing things like setting a minimum TLS version or specifying cipher suites. Isle Site Template includes Traefik configuration files in the `traefik/dynamic` directory. -These files are mounted as volumes in the Traefik containers, so you can modify them as desired. +These configuration files control routing rules and TLS settings. You can modify them to customize your Traefik setup. More information is available in the [Traefik documentation](https://doc.traefik.io/traefik/https/tls/#tls-options) +### Switching between HTTP and HTTPS + +The template provides make commands to easily switch between different modes: + +- `make traefik-http` - Switch to HTTP mode (default for local development) +- `make traefik-https-mkcert` - Switch to HTTPS using mkcert self-signed certificates +- `make traefik-https-letsencrypt` - Switch to HTTPS using Let's Encrypt ACME for production + +After switching modes, restart Traefik with `make down-traefik up` + ### Hiding Fedora from the public -By default, your Fedora repository will be exposed to the public at fcrepo.${DOMAIN}. If you don't want the public to be able to access your repository you can add the `traefik-disable` label to the fcrepo-prod service in your docker-compose.yml. +By default, your Fedora repository will be exposed to the public at `fcrepo.${DOMAIN}`. If you don't want the public to be able to access your repository, you can modify the Traefik routing configuration in the `traefik/dynamic` directory to disable external access while keeping internal access for other services. -``` - labels: - <<: [*traefik-disable, *fcrepo-labels] -``` +If you do this, you can remove the fcrepo subdomain from your DNS records. + +Finally, ensure that Drupal is configured to access Fedora using the internal Docker network hostname: -If you do this, you can remove the fcrepo domain from your DNS records. +```yaml +DRUPAL_DEFAULT_FCREPO_URL: "http://fcrepo:8080/fcrepo/rest/" +``` ### Exposing ActiveMQ, Blazegraph, and Solr -By default, these services are hidden by Traefik. If you want to allow access to them, you need to remove the `traefik-disable` label from your docker-compose.yml. +By default in production environments, these services are hidden from public access. The visibility is controlled through Traefik configuration in the `traefik/dynamic` directory. + +To expose these services: +1. Modify the Traefik dynamic configuration to enable routing for these services +2. Add their URLs to your DNS records (e.g., `activemq.${DOMAIN}`, `blazegraph.${DOMAIN}`, `solr.${DOMAIN}`) +3. Consider adding authentication middleware to protect these administrative interfaces -You will also need to add their URL to your DNS record. The URLs are defined in the docker-compose.yml, and by default will be activemq.{DOMAIN}, blazegraph.{DOMAIN}, and solr.{DOMAIN} +For local development with `DEVELOPMENT_ENVIRONMENT=true`, these services are typically accessible at subdomain URLs for convenience. diff --git a/docs/installation/docker/site-template/docker-modifications.md b/docs/installation/docker/site-template/docker-modifications.md index d9dcd8ef9..b98e51bf4 100644 --- a/docs/installation/docker/site-template/docker-modifications.md +++ b/docs/installation/docker/site-template/docker-modifications.md @@ -24,23 +24,25 @@ Please see the README for the different buildkit images to see what is available - [Milliner](https://github.com/Islandora-Devops/isle-buildkit/tree/main/milliner) - [Solr](https://github.com/Islandora-Devops/isle-buildkit/tree/main/solr) -You can add these environment variables to your docker-compose.yml in order to change their values. For example, if you want to increase the PHP memory limit in your production Drupal container, you can do so like this: +You can add these environment variables to your docker-compose.yml in order to change their values. For example, if you want to increase the PHP memory limit in your Drupal container, you can do so like this: +```yaml +services: + drupal: + environment: + PHP_MEMORY_LIMIT: 1G ``` - drupal-prod: - <<: [*prod, *drupal] - Environment: - PHP_MEMORY_LIMIT: 1G -``` + +Or use a `docker-compose.override.yml` file to keep your customizations separate from the main configuration. ## Removing Services -You may not want to use all the images that are included in the Site Template’s `docker-compose.yml`. You can remove containers by deleting their sections in the docker-compose.yml file. +You may not want to use all the images that are included in the Site Template's `docker-compose.yml`. You can remove containers by deleting their service definitions in the docker-compose.yml file. -For example, to remove Fedora, you would delete the services called fcrepo-dev and fcrepo-prod. +For example, to remove Fedora, you would delete the `fcrepo` service. -Depending on the container you are removing, you may need to delete references to it as well. For example, some containers are referenced by others in the `depends_on` field. You will need to also delete these references, so if you delete the `fedora-dev` service, you will need to remove the rule that `traefik-dev` depends on it. +Depending on the container you are removing, you may need to delete references to it as well. For example, some containers are referenced by others in the `depends_on` field. You will need to also delete these references. If you are removing a container which is referenced by Drupal, ensure that you update Drupal as well (e.g. if removing Fedora, ensure your Media's files are not writing to the Fedora filesystem). @@ -48,22 +50,16 @@ After doing `docker compose down`, run `docker compose up -d --remove-orphans` t ## Hiding Fedora From the Public -By default, your Fedora repo will be available to the public at `fcrepo.${DOMAIN}`. If you do not want to expose your Fedora, you can stop this URL from working by disabling it via Traefik in your `docker-compose.yml`. To do this, you need to add the `traefik-disable` label to `fcrepo-prod` like this, +By default, your Fedora repo will be available to the public at `fcrepo.${DOMAIN}`. If you do not want to expose your Fedora, you can stop this URL from working by disabling it via Traefik in your `docker-compose.yml` or `docker-compose.override.yml`. -```yaml - fcrepo-prod: - <<: [*prod, *fcrepo] - environment: - <<: [*fcrepo-environment] - FCREPO_ALLOW_EXTERNAL_DRUPAL: "https://${DOMAIN}/" - labels: - <<: [*traefik-disable, *fcrepo-labels] -``` +You can modify the Traefik routing configuration in the `traefik/dynamic` directory to disable external access to Fedora while keeping it accessible internally to other services. -If you have done this, you can also remove the DNS records that point this URL to your production server. +If you have done this, you can also remove the DNS records that point this URL to your server. -Finally, you will have to change the URL that Drupal uses to access the Fedora repo. This can be found in your `docker-compose.yml` in the `environment` section for `drupal-prod`, and should be changed to: +Finally, ensure that Drupal is configured to access Fedora using the internal Docker network hostname: ```yaml DRUPAL_DEFAULT_FCREPO_URL: "http://fcrepo:8080/fcrepo/rest/" ``` + +This is typically the default configuration, allowing Drupal to access Fedora internally even when external access is disabled. diff --git a/docs/installation/docker/site-template/setup.md b/docs/installation/docker/site-template/setup.md index 192651a51..81500541b 100644 --- a/docs/installation/docker/site-template/setup.md +++ b/docs/installation/docker/site-template/setup.md @@ -4,21 +4,33 @@ Instructions for creating your site can be found in the [project's README file]( This page lists a few things to note about the process, but is not meant to be a replacement for the instructions in the README file. +## Quick Start + +Getting started is now simplified with the `make` commands: + +1. Clone your repository (created from the template) +2. Run `make up` to initialize and start all services + +The `make up` command handles: +- Preparing your host machine +- Creating the `.env` file from `sample.env` if needed +- Generating necessary secrets and certificates +- Building the custom Drupal Docker image +- Starting all services with smart port allocation + +By default, your site will be available at `http://islandora.traefik.me` (which resolves to 127.0.0.1). + ## Custom Drupal Image ISLE Site Template uses a custom Drupal image that you build on top of the provided Islandora Drupal image. This means you will not be running the islandora/drupal image directly, but the provided Dockerfile will use it to build your image. !!! Note "Note for those coming from ISLE-DC" - In ISLE-DC, we only use a custom image in production, but in the ISLE Site Template, we use it for both. + In ISLE-DC, we only use a custom image in production, but in the ISLE Site Template, we use it for both development and production environments. -Building your custom Drupal image is done by running +Building your custom Drupal image is done by running: -`docker compose --profile dev build` for your development image - -or - -`docker compose --profile prod build` for your production image +`make build` or `docker compose build` This builds the docker image based on the Dockerfile in the `drupal` directory, which uses your composer files to pull the Drupal modules it needs into the image. Because the Dockerfile and composer files are part of the git repository, you can build your Drupal image locally, or on your production server. @@ -31,39 +43,9 @@ This documentation assumes you will be building your production image on the pro ## Adding a Staging Site -The process for setting up a staging site is the same as production, but you will need to use a different URL. You have two options: - -1. You can override the `DOMAIN` environment variable on your staging server by editing its `.env` file -2. Use `docker-compose.override.yml` for this. In `docker-compose.override.yml` you will need to override anywhere the `DOMAIN` variable is used, for example: +The process for setting up a staging site is the same as for your production environment. The simplest approach is to override the `DOMAIN` environment variable on your staging server by editing its `.env` file to use a different URL (e.g., `staging.example.com`). -``` -services: - cantaloupe-prod: - labels: - traefik.http.routers.cantaloupe_http.rule: &traefik-host-cantaloupe-prod Host(`staging-url.com`) && PathPrefix(`/cantaloupe`) - traefik.http.routers.cantaloupe_https.rule: *traefik-host-cantaloupe-prod - drupal-prod: - environment: - DRUPAL_DEFAULT_CANTALOUPE_URL: "https://staging-url.com/cantaloupe/iiif/2" - DRUPAL_DEFAULT_SITE_URL: "staging-url.com" - DRUPAL_DRUSH_URI: "https://staging-url.com" - labels: - traefik.http.routers.drupal_http.rule: &traefik-host-drupal-prod Host(`staging-url.com`) - traefik.http.routers.drupal_https.rule: *traefik-host-drupal-prod - fcrepo-prod: - environment: - FCREPO_ALLOW_EXTERNAL_DRUPAL: "https://staging-url.com" - labels: - traefik.http.routers.fcrepo_http.rule: &traefik-host-fcrepo-prod Host(`fcrepo.staging-url.com`) - traefik.http.routers.fcrepo_https.rule: *traefik-host-fcrepo-prod - traefik-prod: - networks: - default: - aliases: - # Allow services to connect on the same name/port as the outside. - - "staging-url.com" # Drupal is at the root domain. - - "fcrepo.staging-url.com" -``` +If you need more fine-grained control, you can create a `docker-compose.override.yml` file with site-specific customizations. However, with the new unified docker-compose setup, most staging configurations can be handled by simply changing environment variables in the `.env` file. !!! note "Restricting Access to Staging Servers" @@ -77,9 +59,13 @@ If you are spinning up a new site for testing, you can add some demo content to [ -d "islandora_workbench/islandora_demo_objects" ] || git clone https://github.com/Islandora-Devops/islandora_demo_objects.git islandora_workbench/islandora_demo_objects -cd islandora_workbench && docker build --build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g) -t workbench-docker .; cd .. +docker build \ + --build-arg USER_ID=$(id -u) \ + --build-arg GROUP_ID=$(id -g) \ + -t workbench-docker:latest \ + islandora_workbench -perl -i -pe 's#^host.*#host: "https://islandora.dev"#g' islandora_workbench/islandora_demo_objects/create_islandora_objects.yml +perl -i -pe 's#^host.*#host: "http://islandora.traefik.me"#g' islandora_workbench/islandora_demo_objects/create_islandora_objects.yml perl -i -pe 's#^input_dir.*#input_dir: "islandora_demo_objects"#g' islandora_workbench/islandora_demo_objects/create_islandora_objects.yml @@ -87,7 +73,14 @@ perl -i -pe 's#^input_csv.*#input_csv: "create_islandora_objects.csv"#g' islando grep secure_ssl_only islandora_workbench/islandora_demo_objects/create_islandora_objects.yml || echo 'secure_ssl_only: false' >> islandora_workbench/islandora_demo_objects/create_islandora_objects.yml -cd islandora_workbench && docker run -it --rm --network="host" -v .:/workbench --name my-running-workbench workbench-docker bash -lc "./workbench --config islandora_demo_objects/create_islandora_objects.yml"; cd .. +pushd islandora_workbench +docker run -it --rm \ + --network="host" \ + -v .:/workbench \ + --name my-running-workbench \ + workbench-docker:latest \ + bash -lc "./workbench --config islandora_demo_objects/create_islandora_objects.yml" +popd ``` ## Custom Themes & Modules diff --git a/docs/installation/docker/site-template/site-template.md b/docs/installation/docker/site-template/site-template.md index 04be9c163..335bd81bd 100644 --- a/docs/installation/docker/site-template/site-template.md +++ b/docs/installation/docker/site-template/site-template.md @@ -11,31 +11,30 @@ kinds of commands that will be a key part of running and maintaining Islandora. ## Usage -1. **Do not clone the Isle Site Template!** - * Unlike most other repositories we provide, the Isle Site Template is not - meant to be cloned or forked. Rather, it can be downloaded using `curl` -and installed either manually or automatically. - -2. Instead, follow the instructions in the ISLE Site Template's `README.md` and -`README.template.md` files. - - * Instructions are provided both for `dev` and `prod` environments, with - different services available on each. - -3. During installation, you will install a copy of the [Islandora Starter -Site][Islandora Starter Site]. - * Though, if you select the manual installation option, you can change that - out for a different base composer project. This will form the basis of -your Drupal site. If you don't have a custom version, we recommend using the -Islandora Starter Site (and it's installed automatically during the automatic -install). - -4. Customizing your site can be persisted to your own repo. - * In the process of setting you the ISLE Site Template, you are encouraged - to create a custom Git repository for this project. When you do, you can -save your changes to several components of your own site, for example the site -name in Docker, which services you have running, and all changes made to your -entire Drupal site configuration. +1. **Use the GitHub template feature** + * Click the green "Use this template" button on the [ISLE Site Template][ISLE Site Template] + repository to create your own copy. This creates a new repository in your GitHub account + with the same directory structure and files, including a pre-installed copy of the + [Islandora Starter Site][Islandora Starter Site]. + +2. **Clone your new repository** + * After creating your repository from the template, clone it to your local machine or server. + +3. **Quick start with make commands** + * The template now uses a `Makefile` to simplify setup. Just run `make up` to initialize + and start your Islandora site. This handles generating secrets, certificates, + and bringing up all services. + +4. **Default configuration** + * By default, the environment runs over HTTP at `islandora.traefik.me` (which resolves to + 127.0.0.1) for easier local development. You can switch to HTTPS using make commands + for development (`make traefik-https-mkcert`) or production (`make traefik-https-letsencrypt`). + +5. **Customizing your site** + * All customizations you make can be committed to your repository, including Docker + configuration, service settings, Drupal modules, themes, and site configuration. Your + custom Drupal codebase is part of the repository and gets built into a custom Docker + image. [ISLE Site Template]: https://github.com/Islandora-Devops/isle-site-template [Docker Compose]: https://docs.docker.com/compose/ diff --git a/docs/installation/docker/site-template/updating.md b/docs/installation/docker/site-template/updating.md index c5814a41d..a8aaaed06 100644 --- a/docs/installation/docker/site-template/updating.md +++ b/docs/installation/docker/site-template/updating.md @@ -12,25 +12,21 @@ For example, to upgrade from Buildkit 2.0.0 to 3.0.0 you would do the following 2. Stop your Docker containers - `docker compose --profile dev down` + `docker compose down` 3. Pull the new Docker images (except the Drupal image) - `docker compose --profile dev pull --ignore-buildable --ignore-pull-failures` - -4. Build your custom Drupal image + `docker compose pull --ignore-buildable --ignore-pull-failures` - `docker compose --profile dev build` - -5. Start your containers from the new images +4. Build your custom Drupal image - `docker compose --profile dev up -d` + `docker compose build` or `make build` -Once you have upgraded your images, you may need to perform extra steps for Solr and Mariadb, depending on whether these have new versions in the new images. +5. Start your containers from the new images -!!! note "Production Sites" + `docker compose up -d` or `make up` - Upgrading a production site works the same way, just replace `dev` with `prod` in the above instructions. +Once you have upgraded your images, you may need to perform extra steps for Solr and MariaDB, depending on whether these have new versions in the new images. ### Solr @@ -40,16 +36,16 @@ To generate new configs perform the following steps: 1. Remove existing solr configs - `docker compose exec -T solr-dev with-contenv bash -lc 'rm -r server/solr/default/*'` - + `docker compose exec -T solr with-contenv bash -lc 'rm -r server/solr/default/*'` + 2. Restart the Solr container - `docker compose --profile dev restart solr-dev` + `docker compose restart solr` 3. Recreate solr configs for new solr versions - `docker compose exec -T drupal-dev with-contenv bash -lc "for_all_sites create_solr_core_with_default_config"` - + `docker compose exec -T drupal with-contenv bash -lc "for_all_sites create_solr_core_with_default_config"` + 4. Reindex Solr through the admin page or via Drush ### MariaDB @@ -58,7 +54,7 @@ After updating MariaDB, you may need to run [mariadb-upgrade](https://mariadb.co You can run this with -`docker compose exec mariadb-dev mariadb-upgrade` +`docker compose exec mariadb mariadb-upgrade` ## Updating Traefik @@ -66,21 +62,21 @@ Traefik is not updated with the other buildkit images. It is recommended that yo ## Updating Drupal Modules -Drupal updates are performed through composer on your development site. Once the modules have been added/removed/updated, your `composer.json` and `composer.lock` files can be checked into your git repository and you can rebuild your production Drupal container with the new files. +Drupal updates are performed through composer in your Drupal container. Once the modules have been added/removed/updated, your `composer.json` and `composer.lock` files can be checked into your git repository and you can rebuild your Drupal container with the new files. -### Development +### Local Development Composer commands need to [run in your Drupal container](../docker-prereq.md#containers). For example: -​​`docker compose exec drupal-dev composer update -W` +`docker compose exec drupal composer update -W` -Or +Or -`docker compose exec drupal-dev composer require 'drupal/islandora:^2.11'` +`docker compose exec drupal composer require 'drupal/islandora:^2.11'` Running database updates is also done in the container like this: -`docker compose exec drupal-dev drush updb` +`docker compose exec drupal drush updb` !!! note @@ -90,20 +86,20 @@ If you are enabling or uninstalling modules, you will also need to export your D Once you have finished your composer changes you can commit and push your repository with the new `composer.json` and `composer.lock` changes. -### Production +### Deploying to Production First you will `git pull` to get the `composer.json` and `composer.lock` changes you made in development. Next you build your Drupal image again, which will install the modules specified in those files: -`docker compose --profile prod build` +`docker compose build` or `make build` Then you stop and start your containers to get the new image: -`docker compose --profile prod down` +`docker compose down` -`docker compose --profile prod up -d` +`docker compose up -d` or `make up` And if necessary, run database updates: -`docker compose exec drupal-prod drush updb` +`docker compose exec drupal drush updb` diff --git a/mkdocs.yml b/mkdocs.yml index 6f958d0ca..51299c7a0 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -27,6 +27,7 @@ markdown_extensions: - footnotes - toc: permalink: True + - pymdownx.details - pymdownx.highlight: anchor_linenums: true line_spans: __span @@ -126,7 +127,16 @@ nav: - 'Docker': - 'Introduction to ISLE': 'installation/docker/docker-introduction.md' - 'Prerequisites': 'installation/docker/docker-prereq.md' - - 'ISLE-DC': + - 'ISLE Site Template': + - 'Introduction to Site Template': 'installation/docker/site-template/site-template.md' + - 'Creating Your Site': 'installation/docker/site-template/setup.md' + - 'Docker Modifications': 'installation/docker/site-template/docker-modifications.md' + - 'Working in Your Containers': 'installation/docker/site-template/containers.md' + - 'Updating': 'installation/docker/site-template/updating.md' + - 'Back Up and Restore': 'installation/docker/site-template/backup.md' + - 'Customizations' : 'installation/docker/site-template/customizations.md' + - 'Converting to modern Site Template': 'installation/docker/converting.md' + - 'ISLE-DC (deprecated)': - Installing: - 'Installing a Development Server': 'installation/docker/isle-dc/docker-local.md' - 'Installing a Staging/Production Server': 'installation/docker/isle-dc/docker-custom.md' @@ -137,15 +147,6 @@ nav: - 'Available Commands': 'installation/docker/isle-dc/docker-available-commands.md' - 'Available Configuration': 'installation/docker/isle-dc/docker-available-configuration.md' - 'Troubleshooting' : 'installation/docker/isle-dc/docker-troubleshooting.md' - - 'ISLE Site Template': - - 'Introduction to Site Template': 'installation/docker/site-template/site-template.md' - - 'Creating Your Site': 'installation/docker/site-template/setup.md' - - 'Docker Modifications': 'installation/docker/site-template/docker-modifications.md' - - 'Working in Your Containers': 'installation/docker/site-template/containers.md' - - 'Updating': 'installation/docker/site-template/updating.md' - - 'Back Up and Restore': 'installation/docker/site-template/backup.md' - - 'Customizations' : 'installation/docker/site-template/customizations.md' - - 'Converting ISLE-DC to Site Template': 'installation/docker/converting.md' - 'Ansible Playbook': 'installation/playbook.md' - Manual Installation: