> ## Documentation Index
> Fetch the complete documentation index at: https://developers.bloobank.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Retry & delivery

> Delivery guarantees, retry schedule, dead-letter handling, and replay tools.

<Note>
  **Proposed defaults.** The retry schedule below follows industry conventions (Stripe's 3-day, exponential-backoff retry; GitHub's similar policy). Concrete intervals will be finalized with the BlooBank platform team before this scheme exits "Proposed" status.
</Note>

## Delivery guarantees

| Property               | Guarantee                                                                                                              |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| **Delivery semantics** | **At-least-once.** A single state transition may arrive multiple times. Your consumer must deduplicate by `messageId`. |
| **Ordering**           | **None.** Two events on the same resource may arrive out of order. Use the `data` payload as the source of truth.      |
| **Latency**            | Typically ≤ 1 second under normal conditions; can be minutes during platform load or after recovery from an outage.    |
| **Retention**          | Failed deliveries remain replayable from the Dashboard for **30 days**.                                                |

## What counts as a successful delivery

| Condition                                      | Outcome                                                                    |
| ---------------------------------------------- | -------------------------------------------------------------------------- |
| Your endpoint responds `2xx` within 10 seconds | ✅ Success. No retry.                                                       |
| Your endpoint responds `4xx`                   | ❌ **Permanent failure.** No retry. Visible in Dashboard for manual replay. |
| Your endpoint responds `5xx`                   | 🔄 Transient failure. Retried per schedule.                                |
| Connection refused / TLS error / DNS error     | 🔄 Transient failure. Retried per schedule.                                |
| No response within 10 s                        | 🔄 Transient failure (treated as timeout). Retried per schedule.           |

<Warning>
  **`4xx` is permanent.** If your endpoint is temporarily broken and you accidentally return `400` or `404`, the event is **not** retried. Either fail-safe to `5xx` on any error you cannot classify, or fix the bug and trigger a manual replay from the Dashboard.
</Warning>

## Retry schedule

When a delivery fails transiently, BlooBank retries on an exponential schedule with jitter. The schedule below is the proposed default:

|     Attempt | Delay since previous | Cumulative time   |
| ----------: | -------------------- | ----------------- |
| 1 (initial) | —                    | immediate         |
|           2 | \~10 seconds         | \~10 s            |
|           3 | \~1 minute           | \~1 min           |
|           4 | \~10 minutes         | \~11 min          |
|           5 | \~1 hour             | \~1 h             |
|           6 | \~6 hours            | \~7 h             |
|           7 | \~12 hours           | \~19 h            |
|           8 | \~24 hours           | \~43 h            |
|           9 | \~24 hours           | \~67 h (≈ 3 days) |

After attempt 9, the delivery is marked **permanently failed** and surfaced in the Dashboard. You can trigger a manual replay any time within the 30-day retention window.

Actual delays apply random jitter in the range `[delay × 0.5, delay × 1.5]` to avoid thundering-herd recoveries.

`deliveryAttempt` in the event body and in the `X-Bloobank-Delivery-Attempt` header carries the 1-based attempt count — use it to recognize that you are receiving a redelivery (not necessarily a new event).

## What your endpoint must guarantee

To play well with at-least-once delivery, your endpoint must be:

| Property                             | Why                                                                                                                   |
| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------- |
| **Idempotent**                       | Receiving the same `messageId` twice must produce the same outcome as receiving it once.                              |
| **Fast (≤ 10 s)**                    | Real work happens on a background queue. Acknowledge first.                                                           |
| **Resilient to out-of-order events** | Always trust the `data` payload's current state; never rely on the order of receipt.                                  |
| **Verifying signatures**             | An unverified webhook is an attack surface. See [Verifying signatures](/get-started/webhooks/signature-verification). |

See [Best practices](/get-started/webhooks/best-practices) for the implementation pattern.

## Manual replay

Any delivery (whether successful or failed) can be **replayed** from the Dashboard:

1. Find the delivery by `messageId` or by the affected resource id.
2. Click **Replay**. BlooBank re-sends the original payload to your configured endpoint.
3. The replay has the **same** `messageId` and `event` as the original — your idempotent consumer treats it as a duplicate of the original event.

Use this when:

* Your endpoint was broken and you fixed it — replay missed events.
* A bug in your consumer skipped processing — replay to reprocess.
* You are testing changes to your endpoint — replay known-good events from history.

Replay does **not** generate a new `messageId`. Replays are visually distinguished in the Dashboard log but otherwise indistinguishable from a normal retry.

## Permanent failure → reconcile

If a delivery fails permanently and you do not catch it within 30 days, the event is **gone from BlooBank's delivery queue**. But: the underlying state is still in the resource. You can always:

```http theme={"dark"}
GET /wallets/{wallet}/paymentOrders/{paymentOrder} HTTP/1.1
```

The order's current `status` is authoritative — webhooks are a convenience layer for push notifications, not the only source of truth. A daily reconciliation pass that compares your local ledger against `GET /paymentOrders?filter=...&order_by=createdAt%20desc` will catch any missed events.

## Operational best practices

### Monitor your endpoint

| Metric                          | Why it matters                                                                  |
| ------------------------------- | ------------------------------------------------------------------------------- |
| Webhook ack latency (p50, p99)  | Spikes correlate with downstream issues; tail latency near 10 s risks timeouts. |
| Webhook ack error rate          | Should be near zero; spikes trigger retries.                                    |
| Background job lag              | A 200-ms ack does not mean the work is done; track end-to-end.                  |
| Signature verification failures | Should be zero. Non-zero means key drift or attempted abuse.                    |

### Use a queue between ack and work

```mermaid theme={"dark"}
flowchart LR
    A[Webhook arrives] --> B[Verify signature]
    B --> C[Enqueue messageId + payload]
    C --> D[Respond 200 OK]
    C --> E[Background worker]
    E --> F[Dedupe by messageId<br/>then process]
```

The 200 OK leaves your endpoint before the real work starts. If the work fails, you have already acknowledged — your retry logic is **internal**, not via BlooBank redelivery.

This pattern keeps BlooBank's retry contract clean and decouples your reliability from theirs.

### Test with the Dashboard

The Dashboard allows you to send **test events** to your endpoint without a real transaction occurring. Use these to validate signature verification, JSON parsing, and downstream handling in non-production environments.

## Tools

| Action                        | Where                                               |
| ----------------------------- | --------------------------------------------------- |
| Configure endpoint URLs       | Dashboard → Settings → Webhooks                     |
| View delivery history         | Dashboard → Webhooks → Deliveries                   |
| Manually replay a delivery    | Click any delivery, then **Replay**                 |
| Send a test event             | Dashboard → Webhooks → **Send test event**          |
| Rotate webhook signing secret | Dashboard → Settings → Webhooks → **Rotate secret** |

## Next

<CardGroup cols={2}>
  <Card title="Best practices" icon="check-double" href="/get-started/webhooks/best-practices">
    The idempotent consumer pattern.
  </Card>

  <Card title="Verifying signatures" icon="signature" href="/get-started/webhooks/signature-verification">
    Verify every delivery first.
  </Card>
</CardGroup>