Set up
Install the proxymail-qa package and put an API key in PROXYMAIL_QA_API_KEY, in your shell, a .env file kept out of Git, or your CI’s secrets. The Free plan is enough.
npm install --save-dev proxymail-qa
The test
Import test and expect from proxymail-qa/playwright: every test gets a qa fixture, a client that also names the test in each call for the report described below.
import { test, expect } from 'proxymail-qa/playwright';
test('sign-up sends a working code', async ({ page, qa }) => {
const { address, password } = await qa.createAddress({ persona: true, password: true });
const submitted = new Date();
await page.goto('/signup');
await page.getByLabel('Email').fill(address);
await page.getByLabel('Password').fill(password);
await page.getByRole('button', { name: 'Sign up' }).click();
const { value } = await qa.waitForCode(address, { since: submitted });
await page.getByLabel('Code').fill(value);
await expect(page).toHaveURL(/welcome/);
await qa.deleteAddress(address);
});
For a link instead of a code:
const { value: link } = await qa.waitForLink(address, { type: 'verify', since: submitted });
await page.goto(link);
type is verify, magic_signin, invite, reset or any. Match the labels and routes to your app rather than copying these.
Check the email, not just the code
A passing sign-up can still hide a broken email. These catch the usual ones:
// Exactly one verification email, not two.
await new Promise((resolve) => setTimeout(resolve, 10_000)); // time for a duplicate
expect((await qa.countEmails(address, { since: submitted, type: 'link' })).count).toBe(1);
// Its link goes to staging, not localhost or production, and it passes SPF, DKIM and DMARC.
const result = await qa.checkEmail(address, { since: submitted, type: 'link', expectHost: 'staging.example.com' });
expect(result.failed).toEqual([]);
checkEmail also flags emails that don’t say when the code expires, that may not fit a phone, and, with expectText and expectLanguage, missing phrases or the wrong language. It returns verdicts, never the email.
Run it in CI, with a report on the pull request
Add the GitHub Action before your tests and the reporter to your Playwright config. The report on the pull request then lists which emails arrived and how fast, which test each came from, and the tests that failed with the step that failed.
- uses: aaquib90/proxymail-qa-action@v1
with:
api-key: ${{ secrets.PROXYMAIL_QA_API_KEY }}
- run: npx playwright test
env:
PROXYMAIL_QA_API_KEY: ${{ secrets.PROXYMAIL_QA_API_KEY }}
export default defineConfig({
reporter: [['list'], ['proxymail-qa/playwright/reporter']],
});
Each report is compared with the last runs of the same name, so it also says when emails got slower, a test now gets two emails, or a test fails now and then.
More in the Playwright docs, Node.js client and Running in CI.