Merge pull request #169 from ChlodAlejandro/new-backend-docker

Update Docker files (post-Symfony migration)

.dockerignore

@@ -0,0 +1,27 @@
+# Copied from .gitignore
+.idea/
+
+###> symfony/framework-bundle ###
+/.env.local
+/.env.local.php
+/.env.*.local
+/config/secrets/prod/prod.decrypt.private.php
+/public/bundles/
+/var/
+/vendor/
+###< symfony/framework-bundle ###
+
+###> squizlabs/php_codesniffer ###
+/.phpcs-cache
+###< squizlabs/php_codesniffer ###
+
+###> symfony/webpack-encore-bundle ###
+/node_modules/
+npm-debug.log
+yarn-error.log
+###< symfony/webpack-encore-bundle ###
+
+###> symfony/phpunit-bridge ###
+.phpunit.result.cache
+/phpunit.xml
+###< symfony/phpunit-bridge ###

.github/workflows/ci.yml

@@ -45,17 +45,18 @@ jobs:
                 uses: actions/checkout@v2
 
             -   name: Set up QEMU
-                uses: docker/setup-qemu-action@v1
+                uses: docker/setup-qemu-action@v2
 
             -   name: Set up Docker Buildx
                 id: buildx
-                uses: docker/setup-buildx-action@v1
+                uses: docker/setup-buildx-action@v2
 
             - name: Build image
               id: docker_build
-              uses: docker/build-push-action@v2
+              uses: docker/build-push-action@v4
               with:
                   context: .
+                  file: docker/Dockerfile
                   target: ${{ matrix.targets.name }}
                   tags: ${{ matrix.targets.tag }}:latest
                   outputs: type=docker,dest=/tmp/copypatrol-${{ matrix.targets.name }}.image.tar

README.md

@@ -43,54 +43,76 @@ the [ToolforgeBundle](https://github.com/wikimedia/ToolforgeBundle).
 4. Start the Symfony web server with `symfony serve`
 
 ## Installing using Docker
+Development through Docker is suggested if you have a different version of PHP locally
+installed, or if you wish to keep an isolated installation of PHP 7.4 for CopyPatrol.
+The base Docker image used is the `toolforge-php74-sssd-base` image from the Toolforge
+Docker image registry, to ensure an environment as close to Toolforge as possible.
 
-_TODO: Update any of these instructions as necessary following the Symfony migration._
-
-1. Build the Docker image for CopyPatrol.
-   ```bash
-   docker-compose -f docker-compose.dev.yml build
-   ```
-2. Install Composer packages using the PHP version used by the Docker image. Be sure that your current/present working directory is this repository.
-   ```bash
-   # The `wikimedia/copypatrol-development` image is generated in the first step.
-   docker run --rm -it -v $(pwd):/app wikimedia/copypatrol-development composer install
-   ```
-   On Windows, use the following command instead:
-   ```bash
-   docker run --rm -it -v %CD%:/app wikimedia/copypatrol-development composer install
-   ```
-3. Edit the `.env.local` file that was created by composer.
-    1. Use the credentials in your `replica.my.cnf` file in the home directory of your
+1. Copy [.env](.env) to [.env.local](.env.local) and fill in the appropriate details.
+    1. Set `REPLICAS_HOST_*` and `TOOLSDB_HOST` to `127.0.0.1`
+    2. Use the credentials in your `replica.my.cnf` file in the home directory of your
        Toolforge account for `REPLICAS_USERNAME`, `REPLICAS_PASSWORD`, as well as
        `TOOLSDB_USERNAME` and `TOOLSDB_PASSWORD`.
-    2. If you need to test (un)reviewing CopyPatrol cases, `TOOLSDB_USERNAME` and `TOOLSDB_PASSWORD`
+    3. If you need to test (un)reviewing CopyPatrol cases, `TOOLSDB_USERNAME` and `TOOLSDB_PASSWORD`
        need to be set to a user with an installation of the CopyPatrol database (`COPYPATROL_DB_NAME`).
-    3. If you need to test OAuth, obtain tokens by registering a new consumer on Meta at
+    4. If you need to test OAuth, obtain tokens by registering a new consumer on Meta at
        [Special:OAuthConsumerRegistration](https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration).
        Alternatively, you can set `LOGGED_IN_USER` to any value to simulate being that user
        after clicking on 'Login'.
-    4. `ITHENTICATE_USERNAME` and `ITHENTICATE_PASSWORD` are not necessary unless you need
+    5. `ITHENTICATE_USERNAME` and `ITHENTICATE_PASSWORD` are not necessary unless you need
        to test the "iThenticate report" functionality.
-4. Open up an SSH tunnel to access the databases on Toolforge. This assumes you have left
-   the `REPLICAS_HOST_*` and `REPLICAS_PORT_*` variables at their defaults.
+2. Build the development image once and install Composer dependencies with the following
    ```bash
-   symfony console toolforge:ssh --bind-address=0.0.0.0 --toolsdb
+   # (optional) Prevent double-downloading when the build occurs.
+   docker image pull docker-registry.tools.wmflabs.org/toolforge-php74-sssd-base:latest
+   docker compose build
+   # On Windows, use `%CD%` instead of `$(pwd)`.
+   docker run --rm -ti -v $(pwd):/app wikimedia/copypatrol:development composer install
+   ```
+3. (*Windows only*) Set the `HOME` environment variable to your user profile directory.
+   ```cmd
+   setx HOME "%UserProfile%"
+   set HOME "%UserProfile%"
    ```
-5. Run the Docker Compose file. CopyPatrol will be accessible on http://localhost:80
+   The first command sets `HOME` for future shells. The second command sets `HOME` for the current shell.
+4. Open a new terminal and start the development container with
    ```bash
-   docker-compose -f docker-compose.dev.yml up
+   docker compose up
    ```
+   Starting the local development server will be delayed until the next
+   step is finished.
+5. Open up an SSH tunnel to access the databases on Toolforge.
+   ```bash
+   # Your SSH config at $HOME/.ssh will be used.
+   # Your passphrase will be requested if your private key is protected.
+   # If your Toolforge shell name is different from the default, append
+   # your shell name after "ssh". (e.g. `... start ssh exampleuser`)
+   docker compose exec copypatrol start ssh
+   ```
+   This terminal will stay open as long as SSH is connected. No successful
+   connection message is shown, but Symfony will start immediately once the
+   ports are open.
 
 Changes to this folder will automatically be applied to the running Docker container. This includes
-changes to `src` files, `.env.local`, etc. XDebug is set up to connect to the host machine (the computer
-running the Docker container) on request, see the `Dockerfile` for the specific configuration values.
-
-If you wish to use testing databases instead of the Plagiabot databases live on Toolforge, change `COPYPATROL_DB_NAME`,
-and all related connection options. You will still need to connect to the Replica DBs for revision
-information, so leave `REPLICAS_HOST_*` untouched and keep tunneling the port for the Replica DB in step 4.
-
-To make a **production-level** build, run `docker build --target production -t wikimedia/copypatrol:latest`.
-XDebug and other related components will be disabled.
+changes to `src` files, `.env.local`, etc. XDebug is set up to connect to the host machine
+(the computer running the Docker container) on port 9003 upon request ([more info](https://xdebug.org/docs/step_debug)).
+
+If you wish to use testing databases instead of the Plagiabot databases live on Toolforge, change
+`COPYPATROL_DB_NAME`, and all related connection options. You will still need to connect to
+the Replica DBs for revision information, so leave `REPLICAS_HOST_*` untouched and keep
+tunneling the port for the Replica DB in step 5.
+
+To make a **production-level** (Toolforge-like) build, run the following. XDebug and
+other related components will be disabled.
+```bash
+docker build -f docker/Dockerfile . --target production -t wikimedia/copypatrol:latest
+```
+When using this image, bind an `.env.local` file to `/app/.env.local` for configuration.
+This configuration file must also point to proper hosts. When using local ports, use
+`host.docker.internal` for Windows, or `172.17.0.1` for other platforms.
+```bash
+docker run -ti -p 80:80 -v $(pwd)/.env.local:/app/.env.local wikimedia/copypatrol:latest
+```
 
 ## Adding new languages
 

docker-compose.yml

@@ -0,0 +1,43 @@
+version: '3.8'
+
+# =========================================================
+#
+# This file allows anyone to start the CopyPatrol web interface
+# locally. Additional configuration on the `.env.local` file is still
+# required to bring this up.
+#
+# Ensure the following before bringing services from this file up (in order):
+#  - Images must be built (see README)
+#  - Composer packages must be installed (see README)
+#  - `.env.local` contains appropriate credentials (see README)
+#    - `REPLICAS_USERNAME` and `REPLICAS_PASSWORD` must be supplied
+#    - `TOOLSDB_USERNAME` and `TOOLSDB_PASSWORD` must be supplied
+#    - `TOOLSDB_HOST` and `REPLICAS_HOST_*` must be `toolforge-gateway.internal`
+#
+# =========================================================
+
+services:
+    copypatrol:
+        build:
+            context: .
+            dockerfile: docker/Dockerfile
+            target: development
+        image: wikimedia/copypatrol:development
+        command: serve
+        stdin_open: true
+        tty: true
+        extra_hosts:
+            - host.docker.internal:host-gateway
+        ports:
+            - "8000:8000"
+        volumes:
+            # This binds your SSH configuration into the container.
+            -   type: bind
+                source: "$HOME/.ssh"
+                target: "/ssh"
+                read_only: true
+            # This will bind the files in the development directory to the app.
+            -   type: bind
+                source: "."
+                target: "/app"
+        stop_signal: SIGINT

Dockerfile → docker/Dockerfile

@@ -1,69 +1,80 @@
-FROM docker-registry.tools.wmflabs.org/toolforge-php74-sssd-web:latest AS dependencies
+FROM docker-registry.tools.wmflabs.org/toolforge-php74-sssd-web:latest AS vendor
 # ===============================================
 #  COMPOSER INSTALL
+#  Post-install scripts are run in a later stage.
 # ===============================================
 ENV COPYPATROL_ROOT=/app
 WORKDIR ${COPYPATROL_ROOT}
 
 # Install unzip for safety
 RUN apt update && apt install -y unzip
 
-# Install dependencies
+# Copy composer lock file, Symfony config, and bin/ folder
 COPY composer.* ${COPYPATROL_ROOT}
-RUN composer install
 
-FROM docker-registry.tools.wmflabs.org/toolforge-php74-sssd-web:latest AS base
+RUN composer install --no-scripts
+
+# :~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:
+
+FROM docker-registry.tools.wmflabs.org/toolforge-php74-sssd-web:latest as production
 # ===============================================
-#  BASE IMAGE
+#  PRODUCTION IMAGE
 # ===============================================
 ENV COPYPATROL_ROOT=/app
 WORKDIR ${COPYPATROL_ROOT}
 
-# == WORK ==
-
 # Disable file error logging for Lighttpd (enables error logging to stderr)
 RUN sed -i 's!server.errorlog!# server.errorlog!g' /etc/lighttpd/lighttpd.conf
 
 # Enable required Lighttpd modules (rewrite, php)
 RUN lighty-enable-mod fastcgi-php
 RUN lighty-enable-mod rewrite
 
-# add XDebug (if needed)
-RUN apt-get clean && \
-    apt-get update && \
-    DEBIAN_FRONTEND=noninteractive && \
-    apt-get install --yes php7.4-xdebug && \
-    apt-get clean && \
-    rm -rf /var/lib/apt/lists/*
-
 # Add rewrite rules
 RUN echo 'url.rewrite-if-not-file += ( "^(/.*)" => "/index.php$0" )' >> /etc/lighttpd/conf-enabled/90-copypatrol.conf
 
-## Only these two copy statements below actually matter. Everything before this was
-## just to set up a Toolforge-like environment for local development.
+## Everything before this was to set up a Toolforge-like environment
+## for local development.
+
+# Symlink CopyPatrol public to document root
+RUN rm -rf /var/www/html
+RUN ln -s ${COPYPATROL_ROOT}/public /var/www/html
+
 # Copy vendor files
-COPY --from=dependencies ${COPYPATROL_ROOT}/vendor ${COPYPATROL_ROOT}/vendor
+COPY --from=vendor ${COPYPATROL_ROOT}/vendor ${COPYPATROL_ROOT}/vendor
 
 # Copy files
 COPY . ${COPYPATROL_ROOT}
 
-# Symlink CopyPatrol public to document root
-RUN rm -rf /var/www/html
-RUN ln -s ${COPYPATROL_ROOT}/public /var/www/html
+# Run post-install scripts (which we skipped in the vendor stages)
+RUN composer run-script post-install-cmd
 
+EXPOSE 80
 # Set start command (enable FastCGI and start lighttpd)
 CMD [ "lighttpd", "-D", "-f", "/etc/lighttpd/lighttpd.conf" ]
 
-FROM base as production
-# ===============================================
-#  PRODUCTION IMAGE
-# ===============================================
-RUN phpdismod xdebug
+# :~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:
 
-FROM base as development
+FROM docker-registry.tools.wmflabs.org/toolforge-php74-sssd-base:latest as development
 # ===============================================
 #  DEVELOPMENT IMAGE
 # ===============================================
+ENV COPYPATROL_ROOT=/app
+WORKDIR ${COPYPATROL_ROOT}
+
+# Add OpenSSH, XDebug, and Symfony CLI
+RUN curl -1sLf 'https://dl.cloudsmith.io/public/symfony/stable/setup.deb.sh' | bash && \
+    apt-get clean && \
+    apt-get update && \
+    DEBIAN_FRONTEND=noninteractive && \
+    apt-get install --yes openssh-client php7.4-xdebug symfony-cli && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+# Add Composer
+RUN curl https://getcomposer.org/installer | php && \
+    mv composer.phar /usr/local/bin/composer
+
 RUN echo -e "error_reporting=E_ALL\\n\
 \\n\
 [xdebug]\\n\
@@ -72,6 +83,23 @@ xdebug.mode=develop,coverage,debug,profile\\n\
 xdebug.start_with_request=yes\\n\
 xdebug.log=/tmp/xdebug.log\\n\
 xdebug.log_level=0\\n\
-xdebug.remote_host=host.docker.internal\n\
-# XDebug 3\\n\
+xdebug.remote_host=host.docker.internal\\n\
 xdebug.client_host=host.docker.internal\\n" >> /etc/php/7.4/mods-available/xdebug.ini
+
+# Copy vendor files
+COPY --from=vendor ${COPYPATROL_ROOT}/vendor ${COPYPATROL_ROOT}/vendor
+
+# Copy files
+COPY . ${COPYPATROL_ROOT}
+
+# Run post-install scripts (which we skipped in the vendor stages)
+RUN composer run-script post-install-cmd
+
+# Copy the entrypoint file, convert from CRLF to LF (if not
+# already LF), set permissions, and link
+RUN tr -d '\015' </app/docker/docker-entrypoint.sh >/docker-entrypoint.sh && \
+    chmod 700 /docker-entrypoint.sh && \
+    ln -s /docker-entrypoint.sh /usr/local/bin/start
+
+# Set start command (symfony serve)
+ENTRYPOINT [ "/docker-entrypoint.sh" ]