End-to-end testing of signup and password reset emails in CI
Give each CI run its own real email address, assert that signup and reset emails arrive, extract the OTP or link with a transformer, and clean up after.
The most relied-on flow nobody actually tests
Every product has signup, password reset, or magic link emails, and almost every test suite fakes them. The mailer is mocked, the test asserts that send was called with roughly the right arguments, and the suite goes green. Nothing in that proves a user can sign up.
The failures that reach production live below the mock: a template variable renamed on one side only, a confirmation URL built against the wrong host, a provider API key that expired in staging, a DKIM record lost in a DNS migration. All of them sail past a mocked mailer.
The fix is to test the real thing end to end: trigger the flow, receive the actual email at a real address, pull out the OTP or the link, and use it. With the right plumbing this is about forty lines of test code.
Give every test run its own real address
Shared inboxes are where email E2E tests go to flake. A single mail account polled over IMAP mixes messages from parallel runs, rate-limits you, and one day demands a verification code in the middle of your pipeline.
The pattern that works is a unique real address per run. With addresses of the form anything@yournick.hidemy.world, the address ci-run-83@yournick.hidemy.world is simply there to receive mail, no inbox to create. Embed the CI run id so parallel runs cannot collide and any email can be traced back to the run that caused it.
Per-flow granularity works the same way: ci-run-83-reset@ and ci-run-83-signup@ keep assertions unambiguous. If you want to keep things tidy rather than letting test aliases accumulate, the REST API can provision and remove aliases programmatically.
Assert delivery first, content second
The first assertion is blunt: an email arrived at the address within a deadline. This alone catches expired provider credentials, stuck worker queues, and misrouted environments, the failures that otherwise surface as a support ticket.
Then assert content. The subject matches what the template should produce. The OTP has the right shape. The confirmation URL points at the environment under test, not at production, a bug that is embarrassing exactly once. Finally, use the artifact: submit the OTP or fetch the URL and assert the account state changed.
Authentication results deserve a look too. Each received email's payload includes SPF, DKIM, and DMARC outcomes, so CI can fail the build the day your staging sender stops passing DKIM. That is a DNS problem caught before customers meet it as spam-foldering.
Assert the envelope while you are at it. From and Reply-To are template configuration too, and a reset email that suddenly arrives from noreply at the wrong subdomain erodes trust and sometimes breaks replies. One equality assertion each, and they never flake.
Extract the OTP or URL with a transformer, not test regex
The tempting move is regex inside the test: fetch the body, scan for six digits, hope. It works until a footer gains a six digit street number, and it spreads template knowledge across every test file that touches email.
Extraction belongs in the pipeline. A transformer attached to the address pulls the OTP with an anchored regex, or the confirmation link with a CSS selector such as a[href*='/confirm/'], and delivers clean JSON with named fields. Tests read email.extracted.otp and stay one line long.
When the template changes, you fix one transformer instead of twelve tests. You can also verify the transformer itself by uploading the current template as a .eml file and checking the extracted JSON, before any CI run depends on it.
A sketch in JavaScript
Here is the whole loop with Node and any test runner. The API base is api.hidemy.world/v1 and auth is a Bearer key read from the environment.
The polling helper is deliberately dumb: ask for the latest email to the address every two seconds until the deadline passes.
An OTP flow has the same shape: read email.extracted.otp, POST it to the verification endpoint, and assert the session exists afterward. The helper below is the only infrastructure either variant needs.
const API = "https://api.hidemy.world/v1";
const KEY = process.env.HIDEMY_API_KEY; // hm_live_...
async function waitForEmail(to, timeoutMs = 90000) {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
const res = await fetch(`${API}/emails?to=${encodeURIComponent(to)}&limit=1`, {
headers: { Authorization: `Bearer ${KEY}` },
});
const { data } = await res.json();
if (data && data.length > 0) return data[0];
await new Promise((r) => setTimeout(r, 2000));
}
throw new Error(`no email for ${to} within ${timeoutMs}ms`);
}
test("signup sends a working confirmation link", async () => {
const to = `ci-run-${process.env.CI_RUN_ID}@yournick.hidemy.world`;
await signupViaApi({ email: to, password: "long-test-passphrase-9" });
const email = await waitForEmail(to);
expect(email.subject).toMatch(/confirm your account/i);
const url = email.extracted.confirm_url; // set by the transformer
expect(url.startsWith("https://staging.example.app/confirm/")).toBe(true);
const visit = await fetch(url, { redirect: "manual" });
expect([200, 302]).toContain(visit.status);
});Clean up and keep runs independent
Cleanup is mostly a non-problem if you lean on retention: stored emails age out on their own after 7 to 365 days depending on plan, so CI debris does not pile up forever. The discipline that matters is never reusing an address across runs, which keeps every query unambiguous.
If you prefer active cleanup, the same REST API that reads emails can remove aliases and pipelines, so a nightly job can delete anything matching ci-run- older than a week. Count your volume honestly too: every test signup is one received email against your monthly quota, so a suite running on every push may justify a bigger plan than you expected.
Set the delivery deadline with slack for provider queues (60 to 90 seconds is realistic) and poll gently. A test that fails after 90 seconds is telling you something true about your signup flow.
What this catches
In practice this one test class catches template regressions, broken URL builders, environment variables missing from the mail worker, provider outages, and DNS drift that silently kills DKIM. Each of those is invisible to a mocked mailer and obvious to a user.
It also upgrades conversations. Signup emails work stops being a belief backed by a mock and becomes a green check backed by a real message that arrived, parsed, and clicked through. For forty lines of test code, that is a good trade.
Scope it sensibly: these tests belong in the pipeline that deploys staging, not in the unit test loop developers run every minute. A handful of flows, each exercised once per deploy, gives you the signal without making anyone wait on email delivery to merge a refactor.