Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Testsuite updates and some tests #3

Merged
merged 43 commits into from
Sep 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
e8c56f0
:package: Updates dev reqs
jefftriplett Sep 20, 2023
9bac050
:gear: Updates test settings
jefftriplett Sep 20, 2023
b8eb114
:tractor: Refactor test settings
jefftriplett Sep 20, 2023
07dc661
:green_heart: Adds a simple model test
jefftriplett Sep 20, 2023
d1063c0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 20, 2023
7383915
:green_heart: Adds router tests
jefftriplett Sep 25, 2023
62391d1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 25, 2023
f197c73
:gear: :hammer: mypy fixes?
jefftriplett Sep 25, 2023
9ecbbd9
:pencil: commits blank file to make mypy happy
jefftriplett Sep 25, 2023
6140dde
:tractor: Adds/updates lint+mypy tasks
jefftriplett Sep 25, 2023
426f7cd
:gear: Fixes coverage ignore migrations
jefftriplett Sep 25, 2023
b872c1a
:gear: Sets coverage to 33% (we will increase this)
jefftriplett Sep 25, 2023
7ff31c5
:gear: :arrow_down: Drop coverage
jefftriplett Sep 25, 2023
7187cec
:gear: Configures pytest-coverage support
jefftriplett Sep 25, 2023
1dcd1bd
:green_heart: Adds test_runrelay cmd
jefftriplett Sep 25, 2023
73a4e27
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 25, 2023
f8b785a
add intro to README
joshuadavidthomas Sep 20, 2023
3edfaf3
adjust formatting
joshuadavidthomas Sep 20, 2023
f74db20
adjust wording
joshuadavidthomas Sep 20, 2023
af7484a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 20, 2023
f06d50b
add docker publish job to release workflow
joshuadavidthomas Sep 21, 2023
fcfb62b
add if checks to release workflow
joshuadavidthomas Sep 21, 2023
edb3e9b
add correct DATABASES settings for PgBouncer (#7)
joshuadavidthomas Sep 21, 2023
d5b01c8
add default email settings to service (#9)
joshuadavidthomas Sep 22, 2023
4e7beec
fix router name in README
joshuadavidthomas Sep 22, 2023
ef300a2
move version to pyproject.toml
joshuadavidthomas Sep 22, 2023
5c983a9
add changelog
joshuadavidthomas Sep 22, 2023
8639a19
add py.typed file, aspirationally
joshuadavidthomas Sep 22, 2023
fff47a0
fix backend name in README
joshuadavidthomas Sep 22, 2023
355f853
change test workflow trigger (#14)
joshuadavidthomas Sep 22, 2023
14bbdac
move version back to package (#13)
joshuadavidthomas Sep 22, 2023
c28779b
remove Django 4.0 (#12)
joshuadavidthomas Sep 22, 2023
d99f6ec
add support for user settings from environ (#15)
joshuadavidthomas Sep 25, 2023
b39f1eb
add more logging to relay (#19)
joshuadavidthomas Sep 25, 2023
13f32ce
:robot: Bump docker/build-push-action from 4 to 5 (#16)
dependabot[bot] Sep 25, 2023
479f233
:robot: Bump docker/login-action from 2 to 3 (#17)
dependabot[bot] Sep 25, 2023
0dbb574
:robot: Bump docker/setup-buildx-action from 2 to 3 (#18)
dependabot[bot] Sep 25, 2023
ee6a3e5
:robot: [pre-commit.ci] pre-commit autoupdate (#20)
pre-commit-ci[bot] Sep 25, 2023
f0b95d3
:tractor: Run pytest-cov
jefftriplett Sep 25, 2023
92f981c
:arrow_up: Bumps coverage to check to >50%
jefftriplett Sep 25, 2023
674cdf8
:gear: Run normal pytest
jefftriplett Sep 25, 2023
2e5b499
:tractor: Adds python -m to all the things
jefftriplett Sep 25, 2023
3a28784
:tractor: Updates nox entrypoint too
jefftriplett Sep 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 56 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,21 @@ on:
release:
types: [created]
workflow_dispatch:
inputs:
pypi:
description: "Publish to PyPI"
required: false
default: true
type: boolean
docker:
description: "Publish to GHCR"
required: false
default: true
type: boolean

jobs:
publish:
pypi:
if: ${{ github.event_name == 'release' || inputs.pypi }}
runs-on: ubuntu-latest
environment: release
permissions:
Expand All @@ -32,3 +44,46 @@ jobs:
- if: ${{ github.event_name != 'workflow_dispatch' }}
name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1

docker:
if: ${{ github.event_name == 'release' || inputs.docker }}
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
ghcr.io/${{ github.repository }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=ref,event=pr
type=pep440,pattern={{version}}
type=pep440,pattern={{major}}.{{minor}}
type=pep440,pattern={{major}}
type=sha,prefix=sha-
type=raw,value=latest,enable={{is_default_branch}}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and publish Docker image
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
10 changes: 5 additions & 5 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: test
on:
push:
branches: main
pull_request_target:
pull_request:

concurrency:
group: test-${{ github.head_ref }}
Expand All @@ -21,7 +21,7 @@ jobs:
fail-fast: false
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
django-version: ['3.2', '4.0', '4.1', '4.2', 'main']
django-version: ['3.2', '4.1', '4.2', 'main']
steps:
- uses: actions/checkout@v4

Expand All @@ -37,7 +37,7 @@ jobs:

- name: Run tests
run: |
nox --session "tests-${{ matrix.python-version }}(django='${{ matrix.django-version }}')"
python -m nox --session "tests-${{ matrix.python-version }}(django='${{ matrix.django-version }}')"

tests:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -88,7 +88,7 @@ jobs:
# https://hynek.me/articles/ditch-codecov-python/
- name: Run tests
run: |
coverage run -m pytest
python -m pytest
python -m coverage html --skip-covered --skip-empty
python -m coverage report | sed 's/^/ /' >> $GITHUB_STEP_SUMMARY
python -m coverage report --fail-under=100
python -m coverage report --fail-under=50
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@ repos:
- id: check-yaml

- repo: https://github.com/adamchainz/django-upgrade
rev: 1.14.1
rev: 1.15.0
hooks:
- id: django-upgrade
args: [--target-version, "4.2"]

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.0.290
rev: v0.0.291
hooks:
- id: ruff
alias: autoformat
Expand Down
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project attempts to adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- An email backend that stores emails in a database ala a Message model rather than sending them via SMTP or other means
- A database backend that routes writes to the Message model to a separate database
- A Message model that stores the contents of an email
- A relay service intended to be run separately, either as a standalone Docker image or as a management command within a Django project
- Initial documentation (README.md)
- Initial tests
- Initial CI/CD (GitHub Actions)

[unreleased]: https://github.com/westerveltco/django-email-relay/compare/HEAD...HEAD
9 changes: 5 additions & 4 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,7 @@ test:
python -m nox --reuse-existing-virtualenvs

coverage:
rm -rf htmlcov
python -m coverage run -m pytest
python -m coverage html --skip-covered --skip-empty
python -m nox --reuse-existing-virtualenvs --session "coverage"

types:
python -m mypy .
Expand Down Expand Up @@ -235,4 +233,7 @@ envsync:
##################

lint:
pre-commit run --all-files
python -m nox --reuse-existing-virtualenvs --session "lint"

mypy:
python -m nox --reuse-existing-virtualenvs --session "mypy"
22 changes: 16 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,25 @@

[![PyPI](https://img.shields.io/pypi/v/django-email-relay)](https://pypi.org/project/django-email-relay/)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/django-email-relay)
![Django Version](https://img.shields.io/badge/django-3.2%20%7C%204.0%20%7C%204.1%20%7C%204.2-%2344B78B?labelColor=%23092E20)
![Django Version](https://img.shields.io/badge/django-3.2%20%7C%204.1%20%7C%204.2-%2344B78B?labelColor=%23092E20)
<!-- https://shields.io/badges -->
<!-- django-3.2 | 4.0 | 4.1 | 4.2-#44B78B-->
<!-- django-3.2 | 4.1 | 4.2-#44B78B -->
<!-- labelColor=%23092E20 -->

`django-email-relay` enables Django projects without direct access to a preferred SMTP server to use that server for email dispatch.

It consists of two parts: a Django app with a custom email backend that stores emails in a central database queue, and a relay service that reads from this queue to orchestrate email sending, available as either a standalone Docker image or a management command to be used within a Django project that does have access to the preferred SMTP server.

Why opt for this setup?

- The potential for emails sent through an external Email Service Provider (ESP) to be marked as spam or filtered, a common issue when routing transactional emails from internal applications to internal users via an ESP.
- It eliminates the necessity to open firewall ports or the need to utilize services like Tailscale for SMTP server access.
- It decouples the emailing process from the main web application, much in the same way as using a task queue like Celery or Django-Q2 would.

## Requirements

- Python 3.8, 3.9, 3.10, 3.11, or 3.12
- Django 3.2, 4.0, 4.1, or 4.2
- Django 3.2, 4.1, or 4.2
- PostgreSQL (for provided Docker image)

## Installation
Expand All @@ -35,7 +45,7 @@ INSTALLED_APPS = [
]
```

3. Add the `EmailRelayBackend` to your `EMAIL_BACKEND` setting:
3. Add the `RelayDatabaseEmailBackend` to your `EMAIL_BACKEND` setting:

```python
EMAIL_BACKEND = 'email_relay.backend.RelayDatabaseEmailBackend'
Expand Down Expand Up @@ -81,12 +91,12 @@ DJANGO_EMAIL_RELAY = {
}
```

4. Add the `EmailRelayDatabaseRouter` to your `DATABASE_ROUTERS` setting:
4. Add the `EmailDatabaseRouter` to your `DATABASE_ROUTERS` setting:

```python
DATABASE_ROUTERS = [
...
'email_relay.db.EmailRelayDatabaseRouter',
'email_relay.db.EmailDatabaseRouter',
...
]
```
Expand Down
25 changes: 22 additions & 3 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,11 @@
PY_DEFAULT = PY38

DJ32 = "3.2"
DJ40 = "4.0"
DJ41 = "4.1"
DJ42 = "4.2"
DJMAIN = "main"
DJMAIN_MIN_PY = PY310
DJ_VERSIONS = [DJ32, DJ40, DJ41, DJ42, DJMAIN]
DJ_VERSIONS = [DJ32, DJ41, DJ42, DJMAIN]
DJ_DEFAULT = DJ32


Expand Down Expand Up @@ -50,4 +49,24 @@ def tests(session, django):
else:
session.install(f"django=={django}")

session.run("pytest", "-n", "auto", "--dist", "loadfile")
session.run("python", "-m", "pytest", "-n", "auto", "--dist", "loadfile")


@nox.session
def coverage(session):
session.install(".[dev]")
session.run("python", "-m", "pytest")
session.run("python", "-m", "coverage", "html", "--skip-covered", "--skip-empty")
session.run("python", "-m", "coverage", "report", "--fail-under=50")


@nox.session
def lint(session):
session.install(".[lint]")
session.run("pre-commit", "run", "--all-files")


@nox.session
def mypy(session):
session.install(".[dev]")
session.run("mypy")
34 changes: 17 additions & 17 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,20 +41,21 @@ requires-python = ">=3.8"

[project.optional-dependencies]
dev = [
"black",
"coverage[toml]",
"django-stubs",
"django-stubs-ext",
"hatch",
"model_bakery",
"mypy",
"nox",
"pytest",
"pytest-cov",
"pytest-django",
"pytest-randomly",
"pytest-reverse",
"pytest-xdist",
"ruff",
]
lint = ["pre-commit"]
psycopg = ["psycopg[binary]"]
psycopg2 = ["psycopg2-binary"]

Expand Down Expand Up @@ -112,21 +113,25 @@ version_pattern = "YYYY.INC1"

[tool.coverage.run]
omit = [
"src/email_relay/*/migrations/*",
"tests/*",
"manage.py",
"service.py"
"service.py",
"src/email_relay/migrations/*",
"tests/*",
]
source = ["src/email_relay"]
source = ["email_relay"]

[tool.coverage.paths]
source = ["src"]


[tool.django-stubs]
django_settings_module = "tests.settings"
strict_settings = false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've probably already explored this, but is line 128 above this the culprit causing mypy issues, since that tests.settings module doesn't exist anymore in this PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joshuadavidthomas django_settings_module was required. Some of the mypy* setting ordering was causing grief. I didn't figure out why, I just got it working and moved on.


[tool.mypy]
mypy_path = "src/"
namespace_packages = false
check_untyped_defs = true
files = [
"src.email_relay",
]
no_implicit_optional = true
plugins = [
"mypy_django_plugin.main",
Expand All @@ -137,24 +142,19 @@ warn_unused_ignores = true

[[tool.mypy.overrides]]
ignore_errors = true
module = [
"src.email_relay.*.migrations.*",
]

[[tool.mypy.overrides]]
ignore_missing_imports = true
module = []
module = "tests.*"

[tool.mypy_django_plugin]
ignore_missing_model_attributes = true

[tool.pytest.ini_options]
DJANGO_SETTINGS_MODULE = "tests.settings"
django_find_project = false
pythonpath = ". src"
addopts = "--create-db -n auto --dist loadfile"
addopts = "--create-db --cov=email_relay -n auto --dist loadfile"
norecursedirs = ".* bin build dist *.egg htmlcov logs node_modules templates venv"
python_files = "tests.py test_*.py *_tests.py"
testpaths = ["tests"]

[tool.ruff]
ignore = ["E501", "E741"] # temporary
Expand Down
Loading