Email webhook deduplication: accept once, recover safely

Which identifier should your consumer deduplicate on?

A repeated notification should find the same durable record, even when another request is processing it. That is the job of your webhook deduplication key. It is separate from removing duplicate addresses or duplicate messages in a mailbox.

Mailera names its event header X-MailHub-Event and its delivery header X-MailHub-Delivery. Choose identifiers by their documented meaning and scope, not by those names alone. Use the Mailera API documentation when implementing the source-specific adapter.

Scope the logical event key to its source namespace. Include subscription identity when the source contract requires it, and decide separately whether overlapping subscriptions should trigger the same business action.

Mailera supports both message.received and message.junk as distinct event types. As a general consumer rule, do not collapse distinct event types just because they could concern the same message. If a source permits separate occurrences of the same event type for one message, your key must distinguish those occurrences while still recognizing retries of each one.

Avoid timestamps, subject/sender combinations, and raw payload hashes as substitutes for a logical event ID. They encode delivery details or content similarity rather than the event identity your application needs.

Keep key resolution in an application adapter: given an authenticated notification and source configuration, return a namespace and stable event key. This consumer fits into the broader reliable email-agent workflow; it does not replace action-level recovery.

How do you accept an event without a race condition?

Use a durable inbox: a database table that is both the acceptance ledger and the worker’s job source. Do not record an event in one system and then separately enqueue it in another without a transactional handoff.

Mailera lists X-MailHub-Signature and X-Mailera-Signature as signature header names, and names its timestamp header X-MailHub-Timestamp. Keep signature verification and timestamp-based replay checks in the authenticity component, separate from event deduplication.

Follow this receiver sequence:

1. Verify authenticity and apply replay checks before accepting the notification. Follow the signing contract for signature and timestamp handling; see Mailera webhook authenticity verification.
2. Validate the supported event shape and source scope.
3. Resolve the logical event key using the source contract.
4. Attempt an atomic insert into the durable inbox.
5. Commit the transaction. On a conflict, establish that the existing record is durably accepted.
6. Acknowledge acceptance without waiting for slow business work.

Here is an application-owned PostgreSQL schema. Every table and column below belongs to your consumer, not the Mailera API. The Python examples use psycopg; install it with pip install "psycopg[binary]" and set DATABASE_URL to your application database connection string.

import os
import psycopg

with psycopg.connect(os.environ["DATABASE_URL"]) as db:
    db.execute("""
        CREATE TABLE webhook_inbox (
            namespace text NOT NULL, event_key text NOT NULL,
            payload jsonb NOT NULL,
            state text NOT NULL DEFAULT 'pending'
                CHECK (state IN ('pending', 'processing', 'completed')),
            accepted_at timestamptz NOT NULL DEFAULT now(),
            lease_until timestamptz, claim_token text,
            PRIMARY KEY (namespace, event_key)
        )
    """)

The receiver calls this helper after verification, validation, and key resolution. Pass only the processing data your worker needs in payload.

from psycopg.types.json import Jsonb

def accept_event(namespace, event_key, payload):
    with psycopg.connect(os.environ["DATABASE_URL"]) as db:
        inserted = db.execute("""
            INSERT INTO webhook_inbox (namespace, event_key, payload)
            VALUES (%s, %s, %s)
            ON CONFLICT (namespace, event_key) DO NOTHING
            RETURNING event_key
        """, (namespace, event_key, Jsonb(payload))).fetchone()
    return inserted is not None

The connection context commits before the function returns. True means a new row was accepted; False means the uniqueness constraint found an existing one. Neither means the business action is complete. Do not create another job on the conflict path, and do not acknowledge success if the transaction raises an error.

If a duplicate carries conflicting processing data under the same key, flag it for investigation rather than overwriting the accepted job.

When should the endpoint return success?

Return the sender’s accepted response only after durable acceptance. If persistence fails before commit, return a failure response according to that sender’s contract.

For a concrete provider example, Microsoft Graph’s webhook delivery documentation recommends validating and persisting a notification in a queue, then returning 202 Accepted within three seconds when processing cannot finish in that window. It specifies a 5xx response when the notification is neither processed nor queued. Those settings belong to Graph; configure your Mailera-facing endpoint from its own contract.

An HTTP response is not a database transaction. RFC 9110’s HTTP semantics distinguish the observable interface from implementation behavior behind it. Your consumer must connect acknowledgment to persistence explicitly.

What happens if a worker crashes after acceptance?

Use application-owned pending, processing, and completed states. A processing record also needs a recoverable lease, not a permanent lock in application memory.

Claim work inside a short database transaction. Select an eligible pending row, or an expired processing row, with FOR UPDATE SKIP LOCKED. While holding its row lock, set processing, assign a new claim token and lease deadline, then commit before doing slow work.

This is different from checking whether an event exists and inserting afterward. The uniqueness constraint controls acceptance; the atomic claim controls which worker owns a processing attempt.

Choose the lease duration from your workload. Renew it for long-running work, with updates conditional on the current claim token. A worker that loses ownership must not mark the record completed.

For a business change in the same database, lock the inbox row and validate the current claim in the same transaction that applies the change and marks completion. A rollback then leaves neither change committed. If another worker has reclaimed the row, the old claim must be rejected.

External actions have another failure boundary

A remote operation can succeed just before the worker crashes. The local row then still says processing, and recovery cannot infer whether repeating the call is safe.

Use a durable business-action key with downstream idempotency where available. Otherwise, reconcile remote state or route an uncertain outcome for review before repeating the action. A lease alone cannot stop a stalled worker’s remote request from completing after another worker takes ownership.

Mailera’s POST /mailboxes/{id}/send accepts Idempotency-Key, with a 24-hour API idempotency window. Persist the intended send’s key and reuse it for retries within the applicable window; do not generate a new key for each worker attempt. See prevent duplicate sends with Idempotency-Key.

For uncertain sends, GET /sends/{id} exposes queued, sending, sent, failed, or unknown. Keep the send-attempt reference and follow send-status recovery before retrying. An unknown outcome needs reconciliation, not an assumption of failure.

This design is not a universal exactly-once guarantee. Event acceptance and external side effects have different recovery requirements.

How long should you retain deduplication records?

Set retention from the source’s retry horizon, your manual replay policy, and your operational recovery window. The record must survive for as long as you intend to recognize a replay as already accepted.

Expiry changes behavior: once the key is deleted, replaying that event can create a new job. Document that boundary and test it. Do not purge pending or unresolved processing records just because their acceptance timestamp is old.

Keep compact dedupe metadata separately from sensitive content retention. You may need the namespace, event key, completion state, and action references longer than an email body. Design payload cleanup so recovery still has the data it needs.

Make deliberate reprocessing an explicit, audited operation. Decide whether it should recalculate local results or also repeat external actions. Deleting a ledger row is not a sufficient reprocessing policy.

Which mistakes cause duplicate work or lost events?

Run these failure tests against your consumer using synthetic notifications and your own connected test mailbox:

Assert business effects as well as row counts. One inbox row does not prove that only one remote call happened.

Log dedupe decisions, claim transitions, and action references. Exclude keys, signing secrets, and complete message bodies. Alert on stuck work and unresolved external outcomes rather than treating every duplicate as an error.

What changes when an AI agent consumes the event?

Incoming email is untrusted data, not instructions for the worker or agent. Deduplication does not authorize an action or defend against prompt injection. Keep email prompt-injection defenses separate from event processing, and require confirmation before sending.

Separate webhook administration from message reading. Mailera provides webhooks and read scopes for those needs; do not grant send to a read-only consumer. Use least-privilege mailbox-agent architecture when separating provisioning credentials from runtime credentials, alongside Mailera’s security information.

Mailera connects existing mailboxes through its developer interfaces; it does not create new mailboxes or email addresses. Use an existing connected test mailbox. Keep this consumer’s PostgreSQL ledger, worker leases, and business-action policy in your application.