Command to send debug emails, and documentation of email development …

…workflow (#2838) * Fix missing async awaits in email pathways * sendDebugEmail command * Make nodemailer secure option configurable in env vars * Update email development docs * Add more graceful cancellation of sending --------- Co-authored-by: TylerHendrickson <[email protected]>
usdigitalresponse · Apr 2, 2024 · 2b0faf1 · 2b0faf1
1 parent 658294a
commit 2b0faf1
Show file tree

Hide file tree

Showing 13 changed files with 160 additions and 36 deletions.
diff --git a/docs/development.md b/docs/development.md
@@ -66,6 +66,10 @@ See [here](../docker/README.md) for more information about commands to use when
 
    **NOTE:** if you get `Error: Invalid login: 534-5.7.9 Application-specific password required.` then you'll need to set an App Password (<https://myaccount.google.com/apppasswords>) (See Step 4)
 
+## Email
+
+If you need to work on emails, see the [instructions for setting up sending of debug emails in dev](./setup-email.md).
+
 ## Linting
 
 ### VSCode

diff --git a/docs/img/setup-email-ethereal-inbox.png b/docs/img/setup-email-ethereal-inbox.png
diff --git a/docs/img/setup-email-ethereal-welcome.png b/docs/img/setup-email-ethereal-welcome.png
diff --git a/docs/img/setup-email-ethereal.png b/docs/img/setup-email-ethereal.png
diff --git a/docs/img/setup-email-smtp-configuration.png b/docs/img/setup-email-smtp-configuration.png
diff --git a/docs/setup-email.md b/docs/setup-email.md
@@ -0,0 +1,54 @@
+# Email Setup
+
+## Ethereal Mail Setup (sandboxed email environment)
+
+[Ethereal](https://ethereal.email/) is the recommended way to set up your development environment to work on emails. 
+
+Ethereal is a free "fake" SMTP sandbox service. You can instantly create a temporary email account that pretends it can send and receive emails. Emails received and "sent" by this temporary account can be viewed in the messages inbox, but the address will never actually send anything outbound. This makes it a much safer way to test email sending in development. 
+
+1. Go to [Ethereal](https://ethereal.email/) and click "Create Ethereal Account". 
+
+   ![Ethereal homepage](./img/setup-email-ethereal.png)
+
+2. Find the SMTP Configuration details for the new account on the created page. 
+
+   ![SMTP configuration details](./img/setup-email-smtp-configuration.png)
+
+3. Update your `NODEMAILER_*` environment variables in your `server/.env` file with the SMTP details. 
+   - `NODEMAILER_HOST=smtp.ethereal.email`
+   - `NODEMAILER_PORT=587`
+   - `NODEMAILER_SECURE=false` — Ethereal doesn't use a secure SMTP connection, so you'll turn secure off here
+   - `NODEMAILER_EMAIL={{ email }}` — set the new email address from Ethereal's SMTP details here
+   - `NODEMAILER_EMAIL_PW={{ password }}` — set the new password from Ethereal's SMTP details here
+4. Rebuild your app docker container so it picks up the new environment variables
+   - `docker compose down app`
+   - `docker compose up -d`
+5. Use the send debug email tool to send a demo of any email type for your Ethereal email to capture
+   - `docker compose exec app yarn workspace server run send-debug-email`
+6. Click over to the "Messages" tab on Ethereal to see your inbox and view the message
+
+   ![Ethereal inbox](./img/setup-email-ethereal-inbox.png)
+   ![Ehtereal message](./img/setup-email-ethereal-welcome.png)
+
+## Gmail Setup (DANGER — live email environment)
+
+> [!WARNING]
+> Gmail setup is generally not required for local development. Note that with this setup you will send real emails — please ensure you don't have real external email addresses in your database that you could accidentally mail. Please revert these environment variables to disable email sending anytime you're not actively intending to send real email.
+
+Users log into the app by means of a single-use link that is sent to their email. In order to set your app up to send this email, you'll need to setup an App Password in Gmail.
+
+Visit: <https://myaccount.google.com/apppasswords> and set up an "App Password" (see screenshot below). *Note: Select "Mac" even if you're not using a Mac.*
+
+In `packages/server/.env`, set `NODEMAILER_EMAIL` to your email/gmail and set your `NODEMAILER_EMAIL_PW` to the new generated PW.
+
+**Note:** Environment variable changes will require rebuilding your docker container to be picked up. 
+
+![Gmail App Password screen](./img/gmail-app-password.png)
+
+**NOTE:** In order to enable App Password MUST turn on 2FA for gmail.
+
+If running into `Error: Invalid login: 535-5.7.8 Username and Password not accepted.` then ["Allow Less Secure Apps"](https://myaccount.google.com/lesssecureapps) - [source](https://stackoverflow.com/a/59194512)
+
+**NOTE:** Much more reliable and preferable to go the App Password route vs Less Secure Apps.
+
+![Email Error](./img/error-gmail.png)
diff --git a/docs/setup-gmail.md b/docs/setup-gmail.md
diff --git a/docs/setup-mac.md b/docs/setup-mac.md
@@ -22,11 +22,6 @@ To use [NVM](https://nvm.sh/), follow the install directions at <https://github.
 If using [Nodenv](https://github.com/nodenv/nodenv) follow the instructions [here](https://github.com/nodenv/nodenv#installation) to install, then run run `nodenv install`.
 
 ***Make sure to use new terminals after completing install***
-
-### Gmail Setup
-
-See [./setup-gmail.md](./setup-gmail.md) for instructions on setting up Gmail to send emails.
-
 ## Installation
 
 1. Install dependencies

diff --git a/packages/server/.env.example b/packages/server/.env.example
@@ -30,6 +30,7 @@ [email protected]
 # Email Server:
 NODEMAILER_HOST=smtp.gmail.com
 NODEMAILER_PORT=465
+NODEMAILER_SECURE=true
 NODEMAILER_EMAIL=[email protected]
 NODEMAILER_EMAIL_PW=""
 

diff --git a/packages/server/package.json b/packages/server/package.json
@@ -27,6 +27,7 @@
     "migrate:up": "knex migrate:up",
     "parse-output-templates": "node src/scripts/parseOutputTemplates.mjs",
     "pre-commit": "yarn lint",
+    "send-debug-email": "node src/scripts/sendDebugEmail.js",
     "serve": "nodemon src",
     "start:dev": "nodemon src",
     "start": "node src",

diff --git a/packages/server/src/lib/email.js b/packages/server/src/lib/email.js
@@ -96,7 +96,7 @@ function addBaseBranding(emailHTML, brandDetails) {
     return brandedHTML;
 }
 
-function sendPassCode(email, passcode, httpOrigin, redirectTo) {
+async function sendPassCode(email, passcode, httpOrigin, redirectTo) {
     if (!httpOrigin) {
         throw new Error('must specify httpOrigin in sendPassCode');
     }
@@ -132,7 +132,7 @@ function sendPassCode(email, passcode, httpOrigin, redirectTo) {
         console.log(`${BLUE}${message}`);
         console.log(`${BLUE}${'-'.repeat(message.length)}`);
     }
-    return module.exports.deliverEmail({
+    await module.exports.deliverEmail({
         toAddress: email,
         emailHTML,
         emailPlain: `Your link to access USDR's Grants tool is ${href}. It expires in ${expiryMinutes} minutes`,
@@ -162,7 +162,7 @@ async function sendReportErrorEmail(user, reportType) {
         },
     );
 
-    return module.exports.deliverEmail({
+    await module.exports.deliverEmail({
         toAddress: user.email,
         ccAddress: HELPDESK_EMAIL,
         emailHTML,
@@ -295,7 +295,7 @@ async function sendGrantAssignedNotficationForAgency(assignee_agency, grantDetai
             subject: emailSubject,
         },
     ));
-    asyncBatch(inputs, module.exports.deliverEmail, 2);
+    await asyncBatch(inputs, module.exports.deliverEmail, 2);
 }
 
 async function sendGrantAssignedEmail({ grantId, agencyIds, userId }) {
@@ -311,7 +311,7 @@ async function sendGrantAssignedEmail({ grantId, agencyIds, userId }) {
         const agencies = await db.getAgenciesByIds(agencyIds);
         await asyncBatch(
             agencies,
-            (agency) => { module.exports.sendGrantAssignedNotficationForAgency(agency, grantDetail, userId); },
+            async (agency) => { await module.exports.sendGrantAssignedNotficationForAgency(agency, grantDetail, userId); },
             2,
         );
     } catch (err) {
@@ -386,7 +386,7 @@ async function sendGrantDigest({
             },
         ),
     );
-    asyncBatch(inputs, module.exports.deliverEmail, 2);
+    await asyncBatch(inputs, module.exports.deliverEmail, 2);
 }
 
 async function getAndSendGrantForSavedSearch({
@@ -478,7 +478,7 @@ async function sendAsyncReportEmail(recipient, signedUrl, reportType) {
         },
     );
 
-    return module.exports.deliverEmail({
+    await module.exports.deliverEmail({
         toAddress: recipient,
         emailHTML,
         emailPlain: `Your ${reportType} report is ready for download. Paste this link into your browser to download it: ${signedUrl} This link will remain active for 7 days.`,

diff --git a/packages/server/src/lib/email/email-nodemailer.js b/packages/server/src/lib/email/email-nodemailer.js
@@ -33,7 +33,7 @@ function createTransport() {
     return nodemailer.createTransport({
         host: process.env.NODEMAILER_HOST, // e.g. 'smtp.ethereal.email'
         port: process.env.NODEMAILER_PORT, // e.g. 465
-        secure: true, // true for 465, false for other ports
+        secure: process.env.NODEMAILER_SECURE !== 'false', // In dev, it can be useful to turn this off, e.g. to work with ethereal.email
         auth: {
             user: process.env.NODEMAILER_EMAIL,
             pass: process.env.NODEMAILER_EMAIL_PW,
@@ -58,7 +58,7 @@ async function sendEmail(message) {
     if (message.ccAddress) {
         params.cc = message.ccAddress;
     }
-    transport.sendMail(params);
+    await transport.sendMail(params);
 }
 
 module.exports = { sendEmail };

diff --git a/packages/server/src/scripts/sendDebugEmail.js b/packages/server/src/scripts/sendDebugEmail.js
@@ -0,0 +1,91 @@
+#!/usr/bin/env node
+const inquirer = require('inquirer');
+const knex = require('../db/connection');
+const { TABLES } = require('../db/constants');
+const db = require('../db');
+const email = require('../lib/email');
+const seedGrants = require('../../seeds/dev/ref/grants');
+const seedUsers = require('../../seeds/dev/ref/users');
+
+async function sendWelcome() {
+    await email.sendWelcomeEmail(
+        '[email protected]',
+        process.env.WEBSITE_DOMAIN,
+    );
+}
+
+async function sendPassCode() {
+    const loginEmail = '[email protected]';
+    const passcode = await db.createAccessToken(loginEmail);
+    await email.sendPassCode(
+        loginEmail,
+        passcode,
+        process.env.WEBSITE_DOMAIN,
+        null,
+    );
+}
+
+async function sendGrantDigest() {
+    const grantIds = seedGrants.grants.slice(0, 3).map((grant) => grant.grant_id);
+    const grants = await knex(TABLES.grants).whereIn('grant_id', grantIds);
+    await email.sendGrantDigest({
+        name: 'Test agency',
+        matchedGrants: grants,
+        matchedGrantsTotal: grantIds.length,
+        recipients: ['[email protected]'],
+        openDate: '2024-01-01',
+    });
+}
+
+async function sendGrantAssigned() {
+    // Use Dallas since there's only one user in the agency, so we should get only 1 email sent
+    const user = seedUsers.find((seedUser) => seedUser.email === '[email protected]');
+    await email.sendGrantAssignedEmail({
+        grantId: seedGrants.grants[0].grant_id,
+        agencyIds: [user.agency_id],
+        userId: user.id,
+    });
+}
+
+async function sendAsyncReport() {
+    await email.sendAsyncReportEmail(
+        '[email protected]',
+        `${process.env.API_DOMAIN}/api/audit_report/fake_key`,
+        'Audit',
+    );
+}
+
+async function sendReportError() {
+    const userId = seedUsers.find((seedUser) => seedUser.email === '[email protected]').id;
+    const user = await db.getUser(userId);
+    await email.sendReportErrorEmail(user, 'Audit');
+}
+
+const emailTypes = {
+    'new user welcome': sendWelcome,
+    'login passcode': sendPassCode,
+    'grant digest': sendGrantDigest,
+    'grant assigned': sendGrantAssigned,
+    'report generation completed': sendAsyncReport,
+    'report generation failed': sendReportError,
+};
+
+async function main() {
+    const answers = await inquirer.prompt([
+        { name: 'emailType', type: 'list', choices: Object.keys(emailTypes) },
+    ]);
+    try {
+        const sendEmail = emailTypes[answers.emailType];
+        await sendEmail();
+    } catch (e) {
+        console.error('Error sending email. Have you run the DB seed?', e);
+    }
+}
+
+if (require.main === module) {
+    process.on('SIGINT', () => {
+        console.log('Exiting.');
+        process.exit();
+    });
+    main().then(() => process.exit());
+}