How to Monitor Email Deliveries

Overview

A production email-to-webhook pipeline requires monitoring. Emails that fail to deliver can represent lost leads, missed orders, or unprocessed support requests. JsonHook maintains a detailed delivery log for every email received and every webhook delivery attempted, queryable via API.

Each delivery log entry includes:

• Delivery ID and timestamp
• Source address and target webhook URL
• HTTP response status returned by your endpoint
• Response time in milliseconds
• Retry count and next retry time (for failed deliveries)
• Whether the delivery was replayed manually

By querying this API regularly (or setting up automated alerts based on failure rates), you maintain full visibility into your pipeline's health without instrumenting each individual webhook handler.

Prerequisites

Monitoring requirements:

• A JsonHook API key for querying the delivery log API
• A monitoring system or alerting tool (Datadog, Grafana, PagerDuty, a simple cron job, etc.)
• Defined SLAs for your email pipeline (acceptable failure rate, maximum acceptable latency)

Step-by-Step Instructions

Set up delivery monitoring for your JsonHook pipeline:

1. Query the delivery log API:

   GET https://api.jsonhook.com/v1/deliveries?status=failed&limit=100
   Authorization: Bearer YOUR_API_KEY

   Returns the most recent 100 failed deliveries with status, response code, and retry information.
2. Filter by address:

   GET /v1/deliveries?addressId=addr_xxx&status=failed&since=2025-01-01T00:00:00Z

3. Build a monitoring script that runs every 5 minutes, counts failures, and alerts if the failure rate exceeds a threshold.
4. Set up a dashboard showing delivery volume, success rate, and P95 response time over time.
5. Alert on specific conditions: more than 5 consecutive failures for any address, response times above 5 seconds, or any delivery entering the permanent failure state (all retries exhausted).
6. Replay failed deliveries after fixing issues:

   POST /v1/deliveries/{deliveryId}/replay
   Authorization: Bearer YOUR_API_KEY

Code Example

A monitoring script that checks for failures and sends a Slack alert:

import fetch from "node-fetch";

const API_KEY = process.env.JSONHOOK_API_KEY!;
const SLACK_URL = process.env.SLACK_ALERT_WEBHOOK!;
const FAILURE_THRESHOLD = 5;

async function checkDeliveryHealth() {
  const since = new Date(Date.now() - 15 * 60 * 1000).toISOString();
  const url =
    `https://api.jsonhook.com/v1/deliveries?status=failed&since=${since}&limit=100`;

  const res = await fetch(url, {
    headers: { Authorization: `Bearer ${API_KEY}` },
  });
  const { deliveries } = (await res.json()) as any;

  if (deliveries.length >= FAILURE_THRESHOLD) {
    const addresses = [...new Set(deliveries.map((d: any) => d.addressId))];
    await fetch(SLACK_URL, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        text: `:warning: *JsonHook delivery failures*: ${deliveries.length} failures in the last 15 minutes on addresses: ${addresses.join(", ")}`,
      }),
    });
    console.log(`Alerted: ${deliveries.length} failures`);
  } else {
    console.log(`Health OK: ${deliveries.length} failures in last 15 min`);
  }
}

checkDeliveryHealth();
// Run this script via cron: */5 * * * * node monitor.js

Common Pitfalls

Monitoring pitfalls:

• Only monitoring for failures, not latency. A handler that always returns 200 but takes 8 seconds is problematic — it is near the timeout limit. Monitor P95 response time, not just success rate.
• Not accounting for retries in failure counts. Each retry creates a separate delivery log entry. Counting total failed entries inflates the failure count. Query for unique delivery IDs with failed status instead.
• Not monitoring the monitoring script. If your monitoring cron job fails silently, you lose visibility. Use a dead man's switch (e.g., Cronitor, Dead Man's Snitch) to ensure monitoring runs on schedule.
• Relying only on JsonHook's delivery log. JsonHook logs whether your endpoint returned 200, but does not know if your application successfully processed the email after returning 200. Add application-level metrics (e.g., emails processed per minute) for full end-to-end visibility.
• Not setting up replay automation. After fixing a bug in your handler, manually replaying hundreds of failed deliveries is tedious. Write a script that queries all failures in a time window and replays them in batches.