Skip to content

Emails

Jump to bottom
Hanyuan Li edited this page Aug 12, 2022 · 3 revisions

In this project, we're going to be writing automated emails fairly frequently - for example, when a user requests to have their username or password changed. You should be using mocking when testing email sending/receiving, since actually sending/receiving emails takes a lot of time, and requires wrangling with 3rd-party apps that could go down in an instant.

Mocking

Mocking is the practice of substituting a system for a simplified version of that system that we've made. For example, the most common thing to mock is a database - calling a database when testing can be costly, so it makes sense to simulate some database behaviour in a custom class, and use that in our testing instead of an actual PostgresQL instance.

In test/mock/mock_mail.py, I've set up a fake mail server that stores all emails being sent into an array that can be accessed via mailbox.get_message(). Below is the code:

import email
import flask_mail

class MockMail:
    def __init__(self):
        self.ascii_attachments = False
        self.messages = []

    def init_app(self, app):
        app.extensions = getattr(app, 'extensions', {})
        app.extensions['mail'] = self

    def send(self, message: flask_mail.Message):
        self.messages.append(message.as_bytes())

    def get_message(self, n):
        return email.message_from_bytes(self.messages[n])

mailbox = MockMail()

There's a couple of functions in there to ensure that our mock system plays nicely with the rest of Flask, but otherwise it's very straightforward - when we send an email using regular Flask, it gets stored in self.messages as raw bytes, and when we receive a message it gets formatted using the email library.

Sending emails

You can use the functionality provided by flask-mail to send emails. Sending emails works the same regardless of whether or not you're using a real mail server, MockMail or Mailtrap - if any issues arise, ping me (@hanyuone) on Discord.

All you have to do is construct a Message according to flask-mail's specs for messages, and import the mail variable from common.plugins. An example to send an email:

from flask_mail import Message
from common.plugins import mail

def register():
    # OTHER CODE HERE #

    # Send it over to email
    message = Message(
        "Account registered for Week in Wonderland",
        sender="[email protected]",
        recipients=[json["email"]]
    )
    
    message.body = f"Your code is: {code}"

    mail.send(message)

To write a message in HTML (which is what we'll be doing), you will need to use message.html instead of message.body.

Receiving emails

Receiving emails is extremely easy, all we have to do is from test.mock.mock_mail import mailbox, and access individual emails through mailbox.get_message(). Use .get_message(-1) to get the most recent email, since all we're doing under the hood in that function is accessing an array via its index.

You will need to use the mocker fixture from pytest_mock as well in all tests where you need to fake sending/receiving emails - this can be done by adding an extra argument besides client in your test function. Furthermore, you will need to patch any endpoints that send emails (or anywhere that imports from common.plugins import mail) with our mailbox.

A commentated example of this is shown below, in a unit test that tests whether an email is received properly or not when registering:

def test_register_success(client, mocker):
    # In `routes/auth.py`, we import `from common.plugins import mail`. Python essentially "copy-pastes"
    # that `mail` object into `routes.auth`. The way `mocker` works is that we need to mock every single
    # place that the object we're mocking is *used*, not where it's *defined* - even though we've already
    # mocked `common.plugins.mail` in other places, that's not enough. This is why we need this line.
    mocker.patch("routes.auth.mail", mailbox)

    clear_all()

    # Now, we can use `mailbox` completely normally, and requests via `client.post` should
    # alter `mailbox` instead of actually sending emails!
    before = len(mailbox.messages)

    # Register normally
    response = client.post("/auth/register", json={
        "email": "[email protected]",
        "username": "asdf",
        "password": "foobar123"
    })

    assert response.status_code == 200

    # Check that an email was in fact sent
    after = len(mailbox.messages)

    assert after == before + 1

    # Verify recipient
    parsed_email = mailbox.get_message(-1)

    assert parsed_email["To"] == "[email protected]"

Mailtrap

Mailtrap should be used as a LAST RESORT, ideally only for testing styling. This guide is kept here for posterity, but please refer to the guide on mocking above for functionality testing.

Mailtrap is a service that allows us to simulate the sending and receiving of emails without actually doing so. This is very useful, since it allows us to make changes to our automated emails (add more info, change styling) without these emails being released to the general public. Furthermore, because Google disabled less secure apps, we can no longer use regular Gmail accounts to send/receive emails programmatically, and thus we have to rely on a third-party platform.

I have created a Mailtrap account under [email protected], and you can log in to the account to see if your tests work properly. Be sure to empty the inbox frequently, as there is a limit of 50 emails in the free plan.

Receiving emails in tests

When testing out our backend, we will need to see if emails are actually being sent, and if the content inside those emails are what we expect. Mailtrap uses POP3, so to read emails, we will have to use poplib (which is a pre-installed package in Python).

def test_success():
    # Check that we get an email sent
    mailbox = poplib.POP3("pop3.mailtrap.io", 1100)
    mailbox.user(os.environ["MAILTRAP_USERNAME"])
    mailbox.pass_(os.environ["MAILTRAP_PASSWORD"])

    (before, _) = mailbox.stat()

    # OTHER CODE HERE #

    # Verify recipient
    raw_email = b"\n".join(mailbox.retr(1)[1])
    parsed_email = email.message_from_bytes(raw_email)

    assert parsed_email["To"] == "[email protected]"

Some important functions in both poplib and email (another builtin Python library):

  • POP3.user and .pass_ are used to "login" to the virtual mailbox itself, and provide credentials so that the other functions can be called.
  • .stat returns two values - the number of emails and the number of bytes in all emails. Usually, only the first return value is needed.
  • .retr takes in one argument, which is the index email we want to see more details of (1-indexed, so the first email would be index 1). .retr returns three values, but the second one (index 1) contains the raw bytes of the email itself, separated by line.

A raw email looks something like this:

Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: Account registered for Week in Wonderland
From: [email protected]
To: [email protected]
Date: Sat, 11 Jun 2022 12:56:09 +0000
Message-ID: <165495216700.88.6066714241386512699@b981ac2a75b6>

Your code is: 219381

If we want to access a specific field (say, the recipient of the email as seen in the To: field), we want to be able to parse this email into some data structure that's more intuitive in Python. Luckily, the email module does just that with email.message_from_bytes (bytes is needed because mailbox.retr()[1] is an array of bytes, not an actual string). We can then access the recipient by simply doing ["To"].

Clone this wiki locally