Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

app-sf-46: Add email infrastructure #98

Merged
merged 77 commits into from
Dec 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
72640e9
chore: Add symplify/config-transformer
pan93412 Dec 3, 2024
7cc7453
refactor(config): Migrate routes.yaml to PHP
pan93412 Dec 3, 2024
967dd5c
refactor(config): Migrate services.yaml to PHP
pan93412 Dec 3, 2024
d0782cc
feat: Include var/cache in PHPStan and IDE configuration
pan93412 Dec 3, 2024
52a99af
refactor(config): Migrate sensiolabs_typescript.yaml to PHP
pan93412 Dec 3, 2024
654e079
refactor(config): Migrate translation.yaml to PHP
pan93412 Dec 3, 2024
5010b22
refactor(config): Migrate routing.yaml to PHP
pan93412 Dec 3, 2024
fedbc70
refactor(config): Remove lock.yaml
pan93412 Dec 3, 2024
503e7df
refactor(config): Migrate asset_mapper.yaml to PHP
pan93412 Dec 3, 2024
56d43b4
fix(phpstan): Scan dependency-injection explicitly
pan93412 Dec 3, 2024
8370d60
refactor(config): Migrate cache.yaml to PHP
pan93412 Dec 3, 2024
83a58c1
refactor(config): Migrate csrf.yaml to PHP
pan93412 Dec 3, 2024
29994e4
refactor(config): Migrate debug.yaml to PHP
pan93412 Dec 3, 2024
2cf33ba
refactor(config): Migrate doctrine_migrations.yaml to PHP
pan93412 Dec 3, 2024
cdd86fd
refactor(config): Migrate mailer.yaml to PHP
pan93412 Dec 3, 2024
6531102
refactor(config): Migrate routes to PHP
pan93412 Dec 3, 2024
3d9dca9
refactor(config): Migrate notifier.yaml to PHP
pan93412 Dec 3, 2024
a7bb8dc
refactor(config): Migrate twig_component.yaml to PHP
pan93412 Dec 3, 2024
0d6cc92
refactor(config): Migrate validator.yaml to PHP
pan93412 Dec 3, 2024
e18e121
refactor(config): Migrate web_profiler.yaml to PHP
pan93412 Dec 3, 2024
55651cb
refactor(phpstan): Add container configuration
pan93412 Dec 3, 2024
d2b3ff5
refactor: Simplify getParameter determination
pan93412 Dec 3, 2024
d854b45
refactor(config): Migrate framework.yaml to PHP
pan93412 Dec 3, 2024
96aec55
refactor(config): Migrate http_discovery.yaml to PHP
pan93412 Dec 3, 2024
6697a72
fix(frankenphp): Dependency installation
pan93412 Dec 3, 2024
0b6f5d3
refactor(config): Migrate twig.yaml to PHP
pan93412 Dec 3, 2024
8e720d4
chore: Upgrade dependencies
pan93412 Dec 3, 2024
047900a
refactor(config): Migrate meilisearch.yaml to PHP
pan93412 Dec 3, 2024
2dbe31e
refactor(config): Migrate doctrine.yaml to PHP
pan93412 Dec 3, 2024
4c853eb
fix(twig): Use underscore instead of dot for global variables
pan93412 Dec 3, 2024
48bae10
refactor(config): Migrate monolog.yaml to PHP
pan93412 Dec 3, 2024
e2cf70d
refactor(config): Migrate security.yaml to PHP
pan93412 Dec 3, 2024
9aa88c9
refactor(config): Migrate messenger.yaml to PHP
pan93412 Dec 3, 2024
82f76ce
feat: Email Event, admin and preview page
pan93412 Dec 2, 2024
9441b7f
feat: Email Delivery Event, admin and preview page
pan93412 Dec 4, 2024
b8765f5
feat(email): Add "test" kind
pan93412 Dec 4, 2024
d3fdc56
feat(email): EmailCreatedSubscriber
pan93412 Dec 5, 2024
6231bc6
test(email): EmailCreatedSubscriberTest
pan93412 Dec 5, 2024
32dbdca
chore: Upgrade dependencies
pan93412 Dec 8, 2024
bead57e
fix(command): Remove unneeded conditions
pan93412 Dec 8, 2024
8edd260
feat(email): Allow adding plain text content
pan93412 Dec 8, 2024
4259ba7
feat(email): Allow adding plain text content
pan93412 Dec 8, 2024
7e05ee8
fix(email): Set SEQUENCE explicitly
pan93412 Dec 8, 2024
7ba375c
refactor: Migrate from deprecated auto-mapping
pan93412 Dec 8, 2024
b21874d
feat(email): Hide empty rendered result
pan93412 Dec 8, 2024
dddc2aa
chore: Add Amazon SES mailer
pan93412 Dec 8, 2024
f42feed
fix(config): "adapter" does not support in PHP
pan93412 Dec 8, 2024
3f0d296
fix(docker): Remove debug-only config for production image
pan93412 Dec 8, 2024
957aede
feat(docker): Add app-sf-worker for asynchronous message handling
pan93412 Dec 8, 2024
acf13b9
docs: Add how to deploy worker on Zeabur
pan93412 Dec 8, 2024
ab4e8c6
docs: Add variable for Amazon SES configuration
pan93412 Dec 8, 2024
cf6d08b
refactor(docker): Move RUN_MIGRATIONS=false to worker.Dockerfile
pan93412 Dec 8, 2024
c70b5f9
ci(docker): Build worker image
pan93412 Dec 8, 2024
80f4870
ci(docker): Always build but push conditionally
pan93412 Dec 8, 2024
fcf6826
chore: Upgrade dependencies
pan93412 Dec 8, 2024
ca0097a
refactor: Consume all messages in a worker
pan93412 Dec 8, 2024
52da7b3
refactor(messenger): Make LINE Notify send asynchronously
pan93412 Dec 8, 2024
60e7667
refactor(docker): Don't make a worker run forever
pan93412 Dec 8, 2024
d9039c4
fix(migrations): Drop default value of the required text_content column
pan93412 Dec 8, 2024
5d45b29
refactor: Use Redis as the messenger
pan93412 Dec 8, 2024
60e8946
fix(docker): Unique messenger customer name
pan93412 Dec 8, 2024
e163898
chore: Exclude .pnpm-store from .gitignore
pan93412 Dec 8, 2024
e7711bd
test(email): Add tests for EmailKind
pan93412 Dec 9, 2024
8372b00
feat(email): EmailDto and its test
pan93412 Dec 9, 2024
1ead53a
feat(email): EmailService
pan93412 Dec 9, 2024
33a886c
feat(email): Add MJML template, previewers & iframe fix for preview
pan93412 Dec 9, 2024
dff6220
fix(overview): Handle high-level correctly
pan93412 Dec 9, 2024
7147682
chore: Upgrade dependencies
pan93412 Dec 9, 2024
f7fd8b6
chore(phpstan): Add doctrine object manager loader
pan93412 Dec 9, 2024
acbce54
refactor(repository): Remove unneeded assertions
pan93412 Dec 9, 2024
4cfa453
refactor(admin): Move email template preview to admin panel
pan93412 Dec 9, 2024
fb7d3b2
feat(command): Add last login and completed questions statistics
pan93412 Dec 9, 2024
906513d
fix(admin): Disable delete, edit, and new actions by default
pan93412 Dec 9, 2024
626f461
refactor: Remove unused dependencies
pan93412 Dec 9, 2024
824b6c9
refactor: Move dev-only configuration to "dev" folder
pan93412 Dec 9, 2024
727c2c5
docs: Update README with new variables
pan93412 Dec 9, 2024
63b026a
chore: Remove non-matched comment
pan93412 Dec 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ OPENAI_API_KEY=!ChangeMe!
###> symfony/messenger ###
# Choose one of the transports below
# MESSENGER_TRANSPORT_DSN=amqp://guest:guest@localhost:5672/%2f/messages
# MESSENGER_TRANSPORT_DSN=redis://localhost:6379/messages
MESSENGER_TRANSPORT_DSN=doctrine://default?auto_setup=0
MESSENGER_TRANSPORT_DSN=redis://redis.app-sf.orb.local:6379/messages
#MESSENGER_TRANSPORT_DSN=doctrine://default?auto_setup=0
###< symfony/messenger ###

# SQL Runner
Expand All @@ -62,3 +62,11 @@ SQLRUNNER_URL=http://sqlrunner.app-sf.orb.local:8080
###> symfony/line-notify-notifier ###
# LINE_NOTIFY_DSN=linenotify://TOKEN@default
###< symfony/line-notify-notifier ###

###> symfony/amazon-mailer ###
# MAILER_DSN=ses://ACCESS_KEY:SECRET_KEY@default?region=eu-west-1
# MAILER_DSN=ses+smtp://ACCESS_KEY:SECRET_KEY@default?region=eu-west-1
###< symfony/amazon-mailer ###

# Email
[email protected]
79 changes: 71 additions & 8 deletions .github/workflows/docker.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
name: Create and Publish app-sf image
name: Create and Publish app-sf and worker image

on:
push:
branches:
- master
- distro/*
- "*"
- "*/*"
tags:
- v*
workflow_dispatch:
Expand All @@ -17,12 +17,64 @@ permissions:

env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}

jobs:
docker-image:
docker-app-sf:
runs-on: ubuntu-latest

env:
# app-sf
IMAGE_NAME: ${{ github.repository }}

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Log in to the Container registry
uses: docker/login-action@master
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@master
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

- name: Build Docker image
uses: docker/build-push-action@master
with:
context: .
push: false
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

- name: Conditional push for main branch
if: startsWith(github.ref, 'refs/heads/master') || startsWith(github.ref, 'refs/heads/distro/') || startsWith(github.ref, 'refs/tags/v')
uses: docker/build-push-action@master
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

- name: Generate artifact attestation
if: startsWith(github.ref, 'refs/heads/master') || startsWith(github.ref, 'refs/heads/distro/') || startsWith(github.ref, 'refs/tags/v')
uses: actions/attest-build-provenance@main
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true

docker-app-sf-worker:
runs-on: ubuntu-latest

env:
# app-sf-worker
IMAGE_NAME: ${{ github.repository }}-worker

steps:
- name: Checkout repository
uses: actions/checkout@v4
Expand All @@ -40,18 +92,29 @@ jobs:
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

- name: Build and push Docker image
id: push
- name: Build Docker image
uses: docker/build-push-action@master
with:
file: worker.Dockerfile
context: .
push: false
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

- name: Conditional push for main branch
if: startsWith(github.ref, 'refs/heads/master') || startsWith(github.ref, 'refs/heads/distro/') || startsWith(github.ref, 'refs/tags/v')
uses: docker/build-push-action@master
with:
file: worker.Dockerfile
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

- name: Generate artifact attestation
if: startsWith(github.ref, 'refs/heads/master') || startsWith(github.ref, 'refs/heads/distro/') || startsWith(github.ref, 'refs/tags/v')
uses: actions/attest-build-provenance@main
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,5 @@ phpstan.neon
###< phpstan/phpstan ###

# node_modules
node_modules/
node_modules/
.pnpm-store/
6 changes: 6 additions & 0 deletions .idea/app-sf.iml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions .idea/php.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

25 changes: 21 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,11 @@ The Database Playground is a platform designed to enhance your SQL skills throug
1. Deploy Redis, PostgreSQL, Meilisearch, and Umami (for statistics) on Zeabur.
2. Deploy [SQL runner](https://github.com/database-playground/sqlrunner-v2) on Zeabur, and rename the service host to
`sqlrunner`.
3. Deploy the application in Git mode on Zeabur.
4. Add the following environment variables to the application:
3. Deploy the application in Git mode on Zeabur. You can use our prebuilt image at
the GitHub Registry.
4. Deploy the worker in Git mode on Zeabur. You can use our prebuilt image at
the GitHub Registry. Also, it is recommended to create more than 1 worker.
5. Add the following environment variables to the application:
```env
DATABASE_URL=postgresql://${POSTGRES_USERNAME}:${POSTGRES_PASSWORD}@postgresql.zeabur.internal:5432/${POSTGRES_DATABASE}?serverVersion=16&charset=utf8
REDIS_URI=${REDIS_CONNECTION_STRING}
Expand All @@ -56,14 +59,28 @@ The Database Playground is a platform designed to enhance your SQL skills throug
OPENAI_API_KEY=your-openai-api-key
LINE_NOTIFY_DSN=linenotify://line-notify-token@default
SQLRUNNER_URL=http://sqlrunner.zeabur.internal:8080
MAILER_DSN=ses://ACCESS_KEY:SECRET_KEY@default?region=eu-west-1
MESSENGER_TRANSPORT_DSN=${REDIS_URI}/messages
```
5. Bind your domain, and the application will be ready for use. The Meilisearch index will be automatically created on start up.
6. Add the following environment variables to the worker:
```env
DATABASE_URL=postgresql://${POSTGRES_USERNAME}:${POSTGRES_PASSWORD}@postgresql.zeabur.internal:5432/${POSTGRES_DATABASE}?serverVersion=16&charset=utf8
REDIS_URI=${REDIS_CONNECTION_STRING}
MEILISEARCH_URL=http://meilisearch.zeabur.internal:7700
MEILISEARCH_API_KEY=${MEILI_MASTER_KEY}
APP_SECRET=${PASSWORD}
LINE_NOTIFY_DSN=linenotify://line-notify-token@default
MAILER_DSN=ses://ACCESS_KEY:SECRET_KEY@default?region=eu-west-1
MESSENGER_TRANSPORT_DSN=${REDIS_URI}/messages
MESSENGER_CONSUMER_NAME=app-sf-worker-1 # Change the number for each worker
```
7. Bind your domain, and the application will be ready for use. The Meilisearch index will be automatically created on start up.

### Docker

We provide a Docker Compose configuration based on [Symfony Docker](https://github.com/dunglas/symfony-docker) for
deployment. The prebuilt image is available at
the [GitHub Registry](https://github.com/database-playground/app-sf/pkgs/container/app-sf).
the [GitHub Registry](https://github.com/orgs/database-playground/packages).

To deploy the application, you may need to update the secret or environment variables in the `compose.yaml` and
`compose.prod.yaml` files, and then run the following command:
Expand Down
7 changes: 7 additions & 0 deletions assets/styles/app.scss
Original file line number Diff line number Diff line change
Expand Up @@ -413,3 +413,10 @@ ul.credit {
}
}
}

.app-email-preview {
&__rendered__content__html {
height: 60vh;
width: 100%;
}
}
6 changes: 5 additions & 1 deletion compose.override.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,8 @@ services:
environment:
MP_SMTP_AUTH_ACCEPT_ANY: 1
MP_SMTP_AUTH_ALLOW_INSECURE: 1
###< symfony/mailer ###
###< symfony/mailer ###

worker:
env_file:
- .env.local
24 changes: 23 additions & 1 deletion compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ services:
sqlrunner:
image: ghcr.io/database-playground/sqlrunner-v2:main
php:
image: ${IMAGES_PREFIX:-}app-php
image: ${IMAGES_PREFIX:-}app-sf
restart: unless-stopped
environment:
SERVER_NAME: ${SERVER_NAME:-localhost}, php:80
Expand All @@ -55,6 +55,7 @@ services:
MEILISEARCH_URL: "http://meilisearch:7700"
MEILISEARCH_API_KEY: ${MEILI_MASTER_KEY:-!MasterChangeMe!}
SQLRUNNER_URL: "http://sqlrunner:8080"
MESSENGER_TRANSPORT_DSN: "${REDIS_URI}/messages"
volumes:
- caddy_data:/data
- caddy_config:/config
Expand All @@ -72,6 +73,27 @@ services:
- target: 443
published: ${HTTP3_PORT:-443}
protocol: udp
worker:
image: ${IMAGES_PREFIX:-}app-sf-worker
build:
dockerfile: worker.Dockerfile
context: .
environment:
DATABASE_URL: "postgresql://${POSTGRES_USER:-app}:${POSTGRES_PASSWORD:-!ChangeMe!}@database:5432/${POSTGRES_DB:-app}?serverVersion=${POSTGRES_VERSION:-16}&charset=${POSTGRES_CHARSET:-utf8}"
REDIS_URI: "redis://redis:6379"
MEILISEARCH_URL: "http://meilisearch:7700"
MEILISEARCH_API_KEY: ${MEILI_MASTER_KEY:-!MasterChangeMe!}
MESSENGER_TRANSPORT_DSN: "${REDIS_URI}/messages"
MESSENGER_CONSUMER_NAME: app-sf-worker-1
restart: unless-stopped
depends_on:
php:
condition: service_healthy
worker-2:
extends:
service: worker
environment:
MESSENGER_CONSUMER_NAME: app-sf-worker-2

volumes:
###> doctrine/doctrine-bundle ###
Expand Down
3 changes: 3 additions & 0 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,14 @@
"jblond/php-diff": "dev-master",
"league/commonmark": "dev-main",
"meilisearch/search-bundle": "dev-main",
"notfloran/mjml-bundle": "dev-main",
"nyholm/psr7": "dev-master",
"openai-php/client": "dev-main",
"oro/doctrine-extensions": "dev-master",
"phpdocumentor/reflection-docblock": "5.*",
"runtime/frankenphp-symfony": "dev-main",
"sensiolabs/typescript-bundle": "dev-main",
"symfony/amazon-mailer": "7.3.*",
"symfony/asset": "7.3.*",
"symfony/asset-mapper": "7.3.*",
"symfony/console": "7.3.*",
Expand All @@ -45,6 +47,7 @@
"symfony/notifier": "7.3.*",
"symfony/password-hasher": "7.3.*",
"symfony/process": "7.3.*",
"symfony/redis-messenger": "7.3.*",
"symfony/runtime": "7.3.*",
"symfony/security-bundle": "7.3.*",
"symfony/serializer": "7.3.*",
Expand Down
Loading
Loading