From a97cd77713b81382488cf65b5f387d8a8b616480 Mon Sep 17 00:00:00 2001 From: "David A. Wheeler" Date: Thu, 7 Sep 2023 15:46:14 -0400 Subject: [PATCH] Fix email (#2054) * Modify email sending configuration We need to move off SendGrid. First step: modify how we send email so it's much easier to configure. Signed-off-by: David A. Wheeler * Fix documentation so it doesn't say we use SendGrid Signed-off-by: David A. Wheeler * Add link to Amazon SES docs Signed-off-by: David A. Wheeler * Send email using TLS directly (without STARTTLS) Switch to sending email using TLS directly, without using STARTTLS at all. STARTTLS lets us request an upgrade to TLS, and we *could* choose to not send an email if the upgrade fails. However, it's simpler to simply use TLS right from the beginning. Amazon supports this, and many others probably do as well. This is simpler, and thus less prone to failures if an "upgrade to TLS" fails. There's never an opportunity for any party to be "confused" about whether or not TLS is currently enabled. Of course, this *only* applies to the hop from our application to the MTA who will then forward the email on. MTAs communicate with each other in a variety of ways. But if we can securely get the email to an MTA we trust, the MTA can then forward the email using the best mechanism available to it. Signed-off-by: David A. Wheeler --------- Signed-off-by: David A. Wheeler --- README.md | 1 - config/environments/production.rb | 54 ++++++++++++++++++++++++++----- docs/implementation.md | 23 ++++++++++--- docs/security.md | 12 ++++--- 4 files changed, 73 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index 2cfd06963..bc2a83ad7 100644 --- a/README.md +++ b/README.md @@ -188,7 +188,6 @@ to the CII. The domain name is much shorter, too. We use the "www" subdomain because there are technical challenges using a top-level domain with our CDN; it's more efficient to use the subdomain. - ## License All material here is released under the [MIT license](./LICENSE). diff --git a/config/environments/production.rb b/config/environments/production.rb index fc36decde..5ffca4e3d 100644 --- a/config/environments/production.rb +++ b/config/environments/production.rb @@ -98,20 +98,58 @@ # https://github.com/ankane/secure_rails config.action_controller.asset_host = host - # Configure email server. + # Configure email server we use for sending email to others. This server will, + # by definition, be a Mail Transfer Agent (MTA). # For discussion about how to do this, see: + # https://guides.rubyonrails.org/action_mailer_basics.html # https://www.railstutorial.org/book/account_activation_password_reset - config.action_mailer.raise_delivery_errors = true + # We used to use SendGrid; see its documentation for more about that. + # For more about configuring Amazon SES SMTP interface, see: + # https://docs.aws.amazon.com/ses/latest/dg/send-email-smtp.html + # + # TODO: Eventually we should set raise_delivery_errors = true and handle exceptions, + # so that we can do retries, and put email sending in a separate thread. + # However, in the current code, if we raise delivery errors we cause paths to fail + # such as updating a project when the email system isn't working. We'd rather let the + # projects update, even when the email system isn't working properly. + config.action_mailer.raise_delivery_errors = false config.action_mailer.delivery_method = :smtp config.action_mailer.default_url_options = { host: host } ActionMailer::Base.smtp_settings = { - address: 'smtp.sendgrid.net', - port: '587', + # NOTE: 'address' is NOT the email address of the *sender* but the domain of + # the remote mail server (the MTA) that will be *receiving* the email and then send it on. + address: ENV.fetch('BADGEAPP_SEND_EMAIL_ADDRESS', 'smtp.sendgrid.net'), + # Avoid using port 25, use a TLS port instead. + port: ENV.fetch('BADGEAPP_SEND_EMAIL_PORT', '465'), + # "plain" authtication would send the password in the clear if sent on port 25. + # However, we're assuming that we're using TLS to send email to the MTA. authentication: :plain, - user_name: ENV.fetch('SENDGRID_USERNAME', nil), - password: ENV.fetch('SENDGRID_PASSWORD', nil), - domain: 'heroku.com', - enable_starttls_auto: true + # This is the authentication data we'll send to the MTA to be authorized to send email + user_name: ENV.fetch('BADGEAPP_SEND_EMAIL_USERNAME', nil), + password: ENV.fetch('BADGEAPP_SEND_EMAIL_PASSWORD', nil), + # This is used for the HELO announcement to the MTA: + domain: ENV.fetch('BADGEAPP_SEND_EMAIL_DOMAIN', 'heroku.com'), + # The following controls how we use TLS for hop-to-hop (point-to-point) encryption of + # our emails to the MTA. + # *Force* the direct use of TLS when sending email to the MTA. It's simple, which is an + # advantage for security. This typically uses port 465 (not 587). + # Amazon calls this "TLS Wrapper" in: + # https://docs.aws.amazon.com/ses/latest/dg/smtp-connect.html + tls: true + # + # This would use TLS opportunistically, via STARTTLS. + # However, it would send email and passwords in the clear + # if it can't. It's enough to counter most passive threats, but an active threat agent + # could cause TLS to fail & force us to send things in the clear. So we won't use it. + # This would typically use port 587. + # enable_starttls_auto: true + # + # This would use STARTTLS but force the use of TLS. This works, but there are concerns + # that starting the protocol *not* in TLS mode and *then* switching to TLS creates a + # window of opportunity for problems as well as unnecessary complexity. + # So while this is a reasonable alternative, for now we won't use it. + # This would typically use port 587. + # enable_starttls: true } # Enable locale fallbacks for I18n (makes lookups for any locale fall back to diff --git a/docs/implementation.md b/docs/implementation.md index baf793cc6..5623b5afa 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -88,10 +88,10 @@ The application is configured by various environment variables: "production" or "fake_production". Plausible values are "debug", "info", and "warn". Default is "info". See: -* EMAIL_ENCRYPTION_KEY: Key to decrypt email addresses. +* EMAIL_ENCRYPTION_KEY: Key to decrypt the stored user email addresses. Hexadecimal string, must be 64 hex digits (==32 bytes==256 bits). Used by aes-256-gcm (256-bit AES in GCM mode). -* EMAIL_BLIND_INDEX_KEY: Key for blind index created for email +* EMAIL_BLIND_INDEX_KEY: Key for blind index created for user email addresses (used by PBKDF2-HMAC-SHA256). Must be 64 hex digits (==32 bytes==256 bits). * BADGEAPP_DENY_LOGIN: If a non-blank value is set ("true" is recommended), then no on can log in, no one can create a new account (sign up), @@ -155,6 +155,20 @@ The application is configured by various environment variables: * `BADGE_CACHE_STALE_AGE` : Time (in seconds) badges are served by the CDN if it can't get a new value from us. Default 8640000 (100 days), is forced to be at least 2x`BADGE_CACHE_MAX_AGE` +* `BADGEAPP_SEND_EMAIL_*`: Various environment variables that configure how the + BadgeApp sends email to an email server (a kind of Mail Transfer Agent (MTA)) + that will then be sent on to others. See `config/environments/production.rb`. +* `BADGEAPP_SEND_EMAIL_ADDRESS`: The domain of the MTA to send email to, e.g., + `smtp.sendgrid.net`. Note that this is NOT the email address of the + *sender* but the domain of the remote mail server (the MTA) that will be + *receiving* the email and then send it on. +* `BADGEAPP_SEND_EMAIL_PORT`: Port of the MTA. Use one that forces TLS or + or modify the config file. +* `BADGEAPP_SEND_EMAIL_USERNAME`: Username for logging into the MTA +* `BADGEAPP_SEND_EMAIL_PASSWORD`: Password for logging into the MTA +* `BADGEAPP_SEND_EMAIL_DOMAIN`: Domain to report to the MTA (for HELO) +* `SENDGRID_USERNAME` and `SENDGRID_PASSWORD`: These were once used for sending email, + but these are *not* used any more. You can make cryptographically random values (such as keys) using "rails secret". E.g., to create 64 random hexadecimal digits, use: @@ -1112,14 +1126,15 @@ See the Action mailer basics guide at and Hartl's Rails tutorial, e.g.: -To install sendgrid on Heroku to make this work, use: +We used to use SendGrid. You *can* just use SendGrid directly. +You can also install sendgrid on Heroku to provide some extra functions doing this: ~~~~sh heroku addons:create sendgrid:starter ~~~~ If you plan to handle a lot of queries, you probably want to use a CDN. -It's currently set up for Fastly. +It's currently set up to use Fastly. ## Badge SVG diff --git a/docs/security.md b/docs/security.md index e1c4ffb3e..3bc6abb4e 100644 --- a/docs/security.md +++ b/docs/security.md @@ -394,7 +394,8 @@ learning about our users' activities (and thus maintaining user privacy): (e.g., to activate a local account), but those links go directly back to our site for that given purpose, and do not reveal information to anyone else. - We use SendGrid to send email, but we have specifically configured the + You need to configure the MTA used to send email before you can use it. + We used to use SendGrid to send email; we have specifically configured the [SendGrid X-SMTPAPI header to disable all of its trackers we know of](https://sendgrid.com/docs/ui/account-and-settings/tracking/), which are clicktrack, ganalytics, subscriptiontrack, and opentrack. For example, we have never used ganalytics, but by expressly disabling it, @@ -1956,17 +1957,20 @@ as of 2015-12-14: but the data is sufficiently low value, and there aren't good alternatives for low value data like this. This isn't as bad as it might appear, because we prefer encrypted - channels for transmitting all emails. Our application attempts to send + channels for transmitting all emails. + Historically our application attempts to send messages to its MTA using TLS (using `enable_starttls_auto: true`), - and that MTA (SendGrid) then attempts to transfer the email the rest + and that MTA then attempts to transfer the email the rest of the way using TLS if the recipient's email system supports it (see ). This is good protection against passive attacks, and is relatively decent protection against active attacks if the user chooses an email system that supports TLS (an active attacker has to get between the email MTAs, which is often not easy). + More recently we're using `enable_starttls: true` which *forces* email to be + encrypted point-to-point. If users don't like that, they can log in via GitHub and use GitHub's - forgotten password system. + system for dealing with forgotten passwords. The file `config/initializers/filter_parameter_logging.rb` intentionally filters passwords so that they are not included in the log. We require that local user passwords have a minimum length