Running in CI

The same tests that pass on your machine run in CI with one change: the API key comes from the CI system’s secrets. This page covers that, how to use addresses so parallel runs don’t trip over each other, a report of each run’s emails on the pull request, and how to give cloud coding agents access.

The API key as a secret

Create a key in the app under Project settings → API keys (see API keys) and store it as a secret named PROXYMAIL_QA_API_KEY. In GitHub Actions:

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 22
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test
        env:
          PROXYMAIL_QA_API_KEY: ${{ secrets.PROXYMAIL_QA_API_KEY }}

Other CI systems work the same way: a masked CI/CD variable in GitLab, a context or project variable in CircleCI, an environment variable in Jenkins credentials. Keep the key out of the repository and out of logs.

A key works within one project, so give CI its own project (or at least its own key). Its addresses then stay apart from the ones you use by hand, and revoking the CI key doesn’t affect anyone.

One address per run, with plus-tags

Free plans make up to 50 new addresses a day, which a busy pipeline can use up. You don’t need a new address for every test: mail to name+anything@ arrives at name@, and waiting on the tagged address finds only the mail sent to that tag.

So create one address for the run, and give each test its own tag:

// Once per run (Playwright's globalSetup, say): POST /v1/addresses
const base = 'k7f2x9ab12cd@mailclub.dev';

// In each test: sign up with a tagged version of it…
const address = base.replace('@', `+${process.env.GITHUB_RUN_ID}-${testInfo.workerIndex}-${Date.now()}@`);
// …then wait on that same tagged address.
// GET /v1/addresses/{address}/code only finds mail sent to that tag.

With the Node.js client, that’s qa.createAddress() once and qa.waitForCode(tagged, { since }) per test.

Some apps refuse a + in sign-up forms; for those, create an address per test instead. Unusual email addresses lists which characters a tag can hold.

Pass since with the time the test submitted the form, so a wait never picks up an email from an earlier step or run:

const submitted = new Date();
await page.getByRole('button', { name: 'Sign up' }).click();
// GET /v1/addresses/{address}/code?since=<submitted, ISO 8601>

Parallel workers

Each key can make 60 requests a minute on Free and 600 on Team (limits). A wait is one request however long it holds (up to 30 seconds), so a handful of parallel workers on Free is fine. For large suites, use the Team plan or one key per pipeline.

Clean up

Delete addresses at the end of the run with DELETE /v1/addresses/{address}, for example in Playwright’s globalTeardown. Anything you leave is deleted when it expires. Deleting a base address also removes the mail sent to its tags.

A report on the pull request

The Proxy Mail QA action groups a job’s email tests into a run and posts what it saw on the pull request: which emails arrived and how long after the test submitted, the codes, where each link goes, the email checks you ran, and any email that never came. The same report goes in the job summary, and everyone in the project can open the run in the app.

jobs:
  e2e:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v7
      - uses: aaquib90/proxymail-qa-action@v1
        with:
          api-key: ${{ secrets.PROXYMAIL_QA_API_KEY }}
      - run: npm ci
      - run: npx playwright test
        env:
          PROXYMAIL_QA_API_KEY: ${{ secrets.PROXYMAIL_QA_API_KEY }}

The action sets PROXYMAIL_QA_RUN for the steps after it. The Node.js client (0.5.0 or later), the Python helper and the Detox and Appium helper read it, so the calls your tests already make add to the run. The other mobile helpers take it the way they take the key: an environment variable for XCUITest, an instrumentation argument for Espresso, a --dart-define for Flutter, and -e for Maestro. From other tools, send the id in a header: Proxy-Mail-QA-Run: $PROXYMAIL_QA_RUN.

After the job, whether it passed or not, the action finishes the run and posts the report, updating its comment on each push. A run fails when an email the tests waited for never came, one came when none should have, or an email check failed. To fail the job then too, even when the tests passed, set fail-job: true. Runs with the same name share a comment, so give matrix jobs their own: name: e2e (${{ matrix.browser }}).

The report also compares the run with the last 10 finished runs of the same name, and lists what changed: emails that took much longer than usual to arrive, a test that now gets two emails instead of one, tests that fail for the first time or now and then, email checks that newly fail, and a new sender or link site. Keeping the name the same from run to run is what makes that work.

Pull requests from forks don’t get secrets, so there the action warns and starts no run.

A run keeps the addresses, the codes, timings, check verdicts and each link’s site and path, with anything in the path that looks like a token hidden. It never keeps an email’s text or subject, or a link’s query, where sign-in tokens usually are. Runs are kept for 90 days.

Elsewhere (another CI system, or an agent’s session), start a run with POST /v1/runs, send its id in the Proxy-Mail-QA-Run header, finish it with POST /v1/runs/{id}/finish, and get the report with GET /v1/runs/{id}?format=markdown. With the Node.js client, that’s qa.startRun(), qa.finishRun() and qa.runReport(). See the REST API.

Which test each email came from

The report can say which test each email belongs to, and list the tests that failed with what their emails did.

Playwright: import test from proxymail-qa/playwright and use its qa fixture, a client that names its test in every call, and add the reporter, which records failed and flaky tests and the totals. A failed test’s line names the step or browser action that failed (such as At “Enter the code › locator.fill(…)”):

export default defineConfig({
  reporter: [['list'], ['proxymail-qa/playwright/reporter']],
});

import { test, expect } from 'proxymail-qa/playwright';

test('sign-up sends one code', async ({ page, qa }) => {
  const { address } = await qa.createAddress();
  // …sign up, wait for the code, check the email…
});

pytest: import the Python helper’s hooks into conftest.py:

from proxymail_qa import pytest_runtest_setup, pytest_runtest_teardown, pytest_runtest_logreport, pytest_sessionfinish

Mobile tests: each mobile helper takes the test’s name: ProxyMailQA(test:) in Swift, Kotlin and Dart, TEST in a Maestro script’s env, and the current Jest test (or setTest) for Detox and Appium.

Other tools send the test’s name, percent-encoded, in a Proxy-Mail-QA-Test header, and results to POST /v1/runs/{id}/tests (REST API). Both do nothing outside a run, and never fail the tests themselves.

When nothing arrives in CI

When a wait comes back empty, the response says why as far as Proxy Mail QA can tell (hint, and refused for mail we turned away). The usual cause in CI is the staging environment’s email provider: in sandbox or test mode it only sends to verified addresses, so the email never leaves. GET /v1/addresses/{address}/delivery checks an address without waiting. More in Test email not arriving?

If every run fails at once, or the API answers with 5xx errors, check our status page before changing your tests.

Coding agents in the cloud

Signing in to the MCP server through the browser needs a person, so agents that run in the cloud use an API key instead, sent as a bearer token. Give them a key for a project made for the purpose.

Claude Code GitHub Action

Write the MCP configuration in a step before the action, with the key from a secret, and allow the tools the agent needs:

      - name: Proxy Mail QA for Claude
        run: |
          cat > /tmp/mcp.json << 'EOF'
          {
            "mcpServers": {
              "proxymail-qa": {
                "type": "http",
                "url": "https://api.proxymail.qa/mcp",
                "headers": { "Authorization": "Bearer ${{ secrets.PROXYMAIL_QA_API_KEY }}" }
              }
            }
          }
          EOF
      - uses: anthropics/claude-code-action@v1
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          claude_args: |
            --mcp-config /tmp/mcp.json
            --allowedTools mcp__proxymail-qa__get_project,mcp__proxymail-qa__create_address,mcp__proxymail-qa__wait_for_code,mcp__proxymail-qa__wait_for_link,mcp__proxymail-qa__check_delivery,mcp__proxymail-qa__delete_address

GitHub Copilot coding agent

In the repository’s Settings → Copilot → MCP servers, add:

{
  "mcpServers": {
    "proxymail-qa": {
      "type": "http",
      "url": "https://api.proxymail.qa/mcp",
      "headers": { "Authorization": "Bearer $COPILOT_MCP_PROXYMAIL_QA_API_KEY" },
      "tools": ["get_project", "create_address", "wait_for_code", "wait_for_link", "check_delivery", "delete_address"]
    }
  }
}

Then add the key as an Agents secret named COPILOT_MCP_PROXYMAIL_QA_API_KEY in the repository’s copilot environment: Copilot only passes secrets whose names start with COPILOT_MCP_ to MCP servers. Copilot’s coding agent can’t sign in through the browser, which is why it needs the key.

Codex

For the Codex CLI in a CI job (codex exec), add the server to ~/.codex/config.toml and set PROXYMAIL_QA_API_KEY from the CI secret:

[mcp_servers.proxymail-qa]
url = "https://api.proxymail.qa/mcp"
bearer_token_env_var = "PROXYMAIL_QA_API_KEY"

Codex’s documentation doesn’t say whether its cloud tasks can reach remote MCP servers. If yours can’t, ask Codex to call the REST API instead, with the key from the environment’s settings.