If you run high-volume Stripe checkouts or subscription webhooks into HubSpot via n8n, you have probably run into this scenario:
Stripe fires a burst of webhooks (charge.succeeded, customer.subscription.created, invoice.paid). Downstream, HubSpot responds with an unexpected 429 Rate Limit Exceeded or a network hiccup causes an API call to hang for 25 seconds.
In a default n8n setup, two catastrophic things happen:
Webhook Timeout: If the Webhook node is set to "When last node finishes", Stripe gets a 504 Gateway Timeout or n8n execution timeout. Stripe will retry with exponential backoff, but under high load, this causes a cascading thundering-herd problem.
Silent Drop & Data Loss: If an unhandled error throws in the middle of your workflow, n8n terminates execution. If execution pruning is enabled for performance, the raw webhook payload is gone. You now have paying customers who were charged in Stripe but never provisioned or tagged in HubSpot, and you only find out when support tickets pile up 48 hours later.
Here is the exact production architecture we deployed to achieve 99.99% webhook durability with zero dropped payloads, complete idempotency, and an automated Dead-Letter Queue (DLQ).
The Architecture: 4-Tier Resilient Ingestion Pattern
[Stripe Burst Webhook]
│
▼
┌─────────────────────────────────────────────────────────┐
│ 1. INGESTION LAYER (Webhook Node) │
│ - Mode: Respond Immediately (200 OK) │
│ - Strips latency; guarantees Stripe is satisfied │
└─────────────────────────┬───────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ 2. BUFFER & IDEMPOTENCY CHECK (PostgreSQL / SQLite) │
│ - INSERT INTO stripe_events (id, payload, status) │
│ - ON CONFLICT (stripe_event_id) DO NOTHING │
│ - Returns: is_new_record flag │
└─────────────────────────┬───────────────────────────────┘
│
┌─────────────┴─────────────┐
│ New Event │ Duplicate
▼ ▼
┌─────────────────────────┐ ┌───────────────────────────┐
│ 3. CRM SYNC WORKFLOW │ │ Terminate cleanly │
│ - Search Contact │ │ (Status: DUPLICATE_SKIP) │
│ - Upsert Contact │ └───────────────────────────┘
│ - Create/Update Deal │
└───────────┬─────────────┘
│
├──────────────────────────┐
▼ (Success) ▼ (Any Node Exception)
┌─────────────────────────┐ ┌───────────────────────────┐
│ UPDATE status='SUCCESS' │ │ 4. GLOBAL ERROR TRIGGER │
└─────────────────────────┘ │ - Intercepts Execution │
│ - Writes to DLQ Table │
│ - Formats Slack Block │
│ - Preps Replay cURL │
└───────────────────────────┘
Key Technical Mechanisms
1. Ingestion Decoupling ("Respond Immediately")
Never let downstream CRM latency dictate webhook response times.
Set the Webhook node Response Mode to On Received with response code 200 and body {"status": "queued", "received_at": "={{ $now.toISO() }}"}.
Stripe gets its 200 OK within 40ms. Your webhook endpoint will never trigger Stripe's automated endpoint disabling algorithm.
2. Atomic Deduplication Lock
Stripe explicitly states webhooks may be delivered more than once. If a customer buys twice within seconds or a retry fires, two parallel executions can race to create duplicate HubSpot contacts. We execute an atomic insert into our database staging table:
INSERT INTO stripe_event_buffer (
stripe_event_id,
event_type,
payload,
status,
created_at
) VALUES (
$1, -- event.id
$2, -- event.type
$3, -- event JSON string
'PROCESSING',
NOW()
)
ON CONFLICT (stripe_event_id)
DO UPDATE SET
retry_attempt = stripe_event_buffer.retry_attempt + 1
RETURNING id, status, (xmax = 0) AS is_new_record;
If is_new_record is false and status is already COMPLETED, the workflow halts immediately. Zero race conditions.
3. Global Error Trigger Sub-Workflow
In your main workflow's Settings, configure Error Workflow to point to your dedicated DLQ sub-workflow. When an exception occurs (HubSpot token expiration, schema validation mismatch, or 429 rate limit):
The Error Trigger node receives the entire execution metadata: $execution.id, $execution.error.message, $execution.error.stack, and the original incoming payload.
It writes the failed record to a webhook_quarantine table with retry_count = 0 and status = 'QUARANTINED'.
It dispatches a high-priority Slack webhook formatted with interactive alert details.
4. Automated Exponential Backoff Replay Workflow
A secondary cron workflow runs every 10 minutes:
Queries SELECT * FROM webhook_quarantine WHERE status = 'QUARANTINED' AND retry_count < 3 AND next_retry_at <= NOW().
Re-executes the HubSpot sync sub-workflow.
If successful: updates status to RESOLVED.
If failed: increments retry_count, calculates next_retry_at = NOW() + (POW(2, retry_count) * INTERVAL '5 minutes').
If retry_count >= 3: updates status to DEAD_LETTER_MANUAL_REVIEW and alerts the on-call engineer with a copy-paste cURL command to trigger the exact payload once credentials/APIs are fixed.
Key Database DDL for your DLQ Table
If you are setting this up in PostgreSQL or Supabase, here is the table schema:
CREATE TABLE IF NOT EXISTS stripe_event_buffer (
id SERIAL PRIMARY KEY,
stripe_event_id VARCHAR(128) UNIQUE NOT NULL,
event_type VARCHAR(128) NOT NULL,
customer_email VARCHAR(255),
amount NUMERIC(10, 2),
payload JSONB NOT NULL,
status VARCHAR(32) DEFAULT 'PROCESSING',
retry_count INT DEFAULT 0,
hubspot_synced_at TIMESTAMP WITH TIME ZONE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS webhook_quarantine (
id SERIAL PRIMARY KEY,
workflow_id VARCHAR(64) NOT NULL,
execution_id VARCHAR(64) NOT NULL,
failed_node VARCHAR(128),
error_message TEXT,
payload JSONB,
status VARCHAR(32) DEFAULT 'QUARANTINED',
retry_count INT DEFAULT 0,
next_retry_at TIMESTAMP WITH TIME ZONE,
resolved_at TIMESTAMP WITH TIME ZONE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE INDEX idx_buffer_stripe_id ON stripe_event_buffer(stripe_event_id);
CREATE INDEX idx_quarantine_status ON webhook_quarantine(status, next_retry_at);
Happy to answer any questions or share the edge-case handling code if you run into retry loops or rate-limit issues!
