--- title: "Running in CI | Proxy Mail QA" description: "Run sign-up email tests in CI: the API key as a secret, one address per run, parallel workers, a report on each pull request, and coding agents in the cloud." url: https://proxymail.qa/docs/ci --- # Running in CI The same tests that pass on your machine run in CI with one change: the API key comes from the CI system’s secrets. This page covers that, how to use addresses so parallel runs don’t trip over each other, a report of each run’s emails on the pull request, and how to give cloud coding agents access. ## The API key as a secret Create a key in the app under **Project settings → API keys** (see [API keys](https://proxymail.qa/docs/api-keys)) and store it as a secret named `PROXYMAIL_QA_API_KEY`. In GitHub Actions: .github/workflows/e2e.yml ```yaml jobs: e2e: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 - uses: actions/setup-node@v7 with: node-version: 22 - run: npm ci - run: npx playwright install --with-deps - run: npx playwright test env: PROXYMAIL_QA_API_KEY: ${{ secrets.PROXYMAIL_QA_API_KEY }} ``` Other CI systems work the same way: a masked CI/CD variable in GitLab, a context or project variable in CircleCI, an environment variable in Jenkins credentials. Keep the key out of the repository and out of logs. A key works within one project, so give CI its own project (or at least its own key). Its addresses then stay apart from the ones you use by hand, and revoking the CI key doesn’t affect anyone. ## One address per run, with plus-tags Free plans make up to 50 new addresses a day, which a busy pipeline can use up. You don’t need a new address for every test: mail to `name+anything@` arrives at `name@`, and waiting on the tagged address finds only the mail sent to that tag. So create one address for the run, and give each test its own tag: One address, a tag per test ```ts // Once per run (Playwright's globalSetup, say): POST /v1/addresses const base = 'k7f2x9ab12cd@mailclub.dev'; // In each test: sign up with a tagged version of it… const address = base.replace('@', `+${process.env.GITHUB_RUN_ID}-${testInfo.workerIndex}-${Date.now()}@`); // …then wait on that same tagged address. // GET /v1/addresses/{address}/code only finds mail sent to that tag. ``` With the [Node.js client](https://proxymail.qa/docs/node), that’s `qa.createAddress()` once and `qa.waitForCode(tagged, { since })` per test. Some apps refuse a `+` in sign-up forms; for those, create an address per test instead. [Unusual email addresses](https://proxymail.qa/docs/unusual-addresses) lists which characters a tag can hold. Pass `since` with the time the test submitted the form, so a wait never picks up an email from an earlier step or run: ```ts const submitted = new Date(); await page.getByRole('button', { name: 'Sign up' }).click(); // GET /v1/addresses/{address}/code?since= ``` ## Parallel workers Each key can make 60 requests a minute on Free and 600 on Team ([limits](https://proxymail.qa/docs/limits)). A wait is one request however long it holds (up to 30 seconds), so a handful of parallel workers on Free is fine. For large suites, use the Team plan or one key per pipeline. ## Clean up Delete addresses at the end of the run with `DELETE /v1/addresses/{address}`, for example in Playwright’s `globalTeardown`. Anything you leave is deleted when it expires. Deleting a base address also removes the mail sent to its tags. ## A report on the pull request The Proxy Mail QA action groups a job’s email tests into a **run** and posts what it saw on the pull request: which emails arrived and how long after the test submitted, the codes, where each link goes, the [email checks](https://proxymail.qa/docs/rest-api#checking-an-emails-sending-and-build) you ran, and any email that never came. The same report goes in the job summary, and everyone in the project can open the run in the app. .github/workflows/e2e.yml ```yaml jobs: e2e: runs-on: ubuntu-latest permissions: contents: read pull-requests: write steps: - uses: actions/checkout@v7 - uses: aaquib90/proxymail-qa-action@v1 with: api-key: ${{ secrets.PROXYMAIL_QA_API_KEY }} - run: npm ci - run: npx playwright test env: PROXYMAIL_QA_API_KEY: ${{ secrets.PROXYMAIL_QA_API_KEY }} ``` The action sets `PROXYMAIL_QA_RUN` for the steps after it. The [Node.js client](https://proxymail.qa/docs/node) (0.5.0 or later), the [Python helper](https://proxymail.qa/docs/python) and the [Detox and Appium helper](https://proxymail.qa/docs/detox-appium) read it, so the calls your tests already make add to the run. The other [mobile helpers](https://proxymail.qa/docs/mobile) take it the way they take the key: an environment variable for XCUITest, an instrumentation argument for Espresso, a `--dart-define` for Flutter, and `-e` for Maestro. From other tools, send the id in a header: `Proxy-Mail-QA-Run: $PROXYMAIL_QA_RUN`. After the job, whether it passed or not, the action finishes the run and posts the report, updating its comment on each push. A run fails when an email the tests waited for never came, one came when none should have, or an email check failed. To fail the job then too, even when the tests passed, set `fail-job: true`. Runs with the same name share a comment, so give matrix jobs their own: `name: e2e (${{ matrix.browser }})`. The report also compares the run with the last 10 finished runs of the same name, and lists what changed: emails that took much longer than usual to arrive, a test that now gets two emails instead of one, tests that fail for the first time or now and then, email checks that newly fail, and a new sender or link site. Keeping the name the same from run to run is what makes that work. Pull requests from forks don’t get secrets, so there the action warns and starts no run. A run keeps the addresses, the codes, timings, check verdicts and each link’s site and path, with anything in the path that looks like a token hidden. It never keeps an email’s text or subject, or a link’s query, where sign-in tokens usually are. Runs are kept for 90 days. Elsewhere (another CI system, or an agent’s session), start a run with `POST /v1/runs`, send its id in the `Proxy-Mail-QA-Run` header, finish it with `POST /v1/runs/{id}/finish`, and get the report with `GET /v1/runs/{id}?format=markdown`. With the Node.js client, that’s `qa.startRun()`, `qa.finishRun()` and `qa.runReport()`. See the [REST API](https://proxymail.qa/docs/rest-api). ### Which test each email came from The report can say which test each email belongs to, and list the tests that failed with what their emails did. **Playwright:** import `test` from `proxymail-qa/playwright` and use its `qa` fixture, a client that names its test in every call, and add the reporter, which records failed and flaky tests and the totals. A failed test’s line names the step or browser action that failed (such as `At “Enter the code › locator.fill(…)”`): playwright.config.ts ```ts export default defineConfig({ reporter: [['list'], ['proxymail-qa/playwright/reporter']], }); ``` signup.spec.ts ```ts import { test, expect } from 'proxymail-qa/playwright'; test('sign-up sends one code', async ({ page, qa }) => { const { address } = await qa.createAddress(); // …sign up, wait for the code, check the email… }); ``` **pytest:** import the [Python helper’s](https://proxymail.qa/docs/python#which-test-each-email-came-from) hooks into `conftest.py`: conftest.py ```python from proxymail_qa import pytest_runtest_setup, pytest_runtest_teardown, pytest_runtest_logreport, pytest_sessionfinish ``` **Mobile tests:** each [mobile helper](https://proxymail.qa/docs/mobile) takes the test’s name: `ProxyMailQA(test:)` in Swift, Kotlin and Dart, `TEST` in a Maestro script’s `env`, and the current Jest test (or `setTest`) for Detox and Appium. Other tools send the test’s name, percent-encoded, in a `Proxy-Mail-QA-Test` header, and results to `POST /v1/runs/{id}/tests` ([REST API](https://proxymail.qa/docs/rest-api#runs-a-report-for-the-pull-request)). Both do nothing outside a run, and never fail the tests themselves. ## When nothing arrives in CI When a wait comes back empty, the response says why as far as Proxy Mail QA can tell (`hint`, and `refused` for mail we turned away). The usual cause in CI is the staging environment’s email provider: in sandbox or test mode it only sends to verified addresses, so the email never leaves. `GET /v1/addresses/{address}/delivery` checks an address without waiting. More in [Test email not arriving?](https://proxymail.qa/troubleshooting) If every run fails at once, or the API answers with 5xx errors, check [our status page](https://proxymailqa.statuspage.io) before changing your tests. ## Coding agents in the cloud Signing in to the MCP server through the browser needs a person, so agents that run in the cloud use an API key instead, sent as a bearer token. Give them a key for a project made for the purpose. ### Claude Code GitHub Action Write the MCP configuration in a step before the action, with the key from a secret, and allow the tools the agent needs: .github/workflows/claude.yml ```yaml - name: Proxy Mail QA for Claude run: | cat > /tmp/mcp.json << 'EOF' { "mcpServers": { "proxymail-qa": { "type": "http", "url": "https://api.proxymail.qa/mcp", "headers": { "Authorization": "Bearer ${{ secrets.PROXYMAIL_QA_API_KEY }}" } } } } EOF - uses: anthropics/claude-code-action@v1 with: anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} claude_args: | --mcp-config /tmp/mcp.json --allowedTools mcp__proxymail-qa__get_project,mcp__proxymail-qa__create_address,mcp__proxymail-qa__wait_for_code,mcp__proxymail-qa__wait_for_link,mcp__proxymail-qa__check_delivery,mcp__proxymail-qa__delete_address ``` ### GitHub Copilot coding agent In the repository’s **Settings → Copilot → MCP servers**, add: ```json { "mcpServers": { "proxymail-qa": { "type": "http", "url": "https://api.proxymail.qa/mcp", "headers": { "Authorization": "Bearer $COPILOT_MCP_PROXYMAIL_QA_API_KEY" }, "tools": ["get_project", "create_address", "wait_for_code", "wait_for_link", "check_delivery", "delete_address"] } } } ``` Then add the key as an Agents secret named `COPILOT_MCP_PROXYMAIL_QA_API_KEY` in the repository’s `copilot` environment: Copilot only passes secrets whose names start with `COPILOT_MCP_` to MCP servers. Copilot’s coding agent can’t sign in through the browser, which is why it needs the key. ### Codex For the Codex CLI in a CI job (`codex exec`), add the server to `~/.codex/config.toml` and set `PROXYMAIL_QA_API_KEY` from the CI secret: \~/.codex/config.toml ```toml [mcp_servers.proxymail-qa] url = "https://api.proxymail.qa/mcp" bearer_token_env_var = "PROXYMAIL_QA_API_KEY" ``` Codex’s documentation doesn’t say whether its cloud tasks can reach remote MCP servers. If yours can’t, ask Codex to call the [REST API](https://proxymail.qa/docs/rest-api) instead, with the key from the environment’s settings. [Previous Python](https://proxymail.qa/docs/python.html) [Next Overview](https://proxymail.qa/docs/mobile.html)