Webhooks

Instead of polling GET /me/scrapes/{id} until status flips to completed, you can subscribe a URL and we'll POST to it the moment that happens. One event exists today: scrape_session.completed.

The payload is a pointer, not the data

{
  "event": "scrape_session.completed",
  "user_id": "3e2f9a10-df34-4b8a-9c31-8e1d2f6a7b90",
  "scrape_session_id": "b1f8b6b0-2f7d-4a1a-9a3c-2f6d8e9a1234",
  "status": "completed",
  "stats": { "total": 40, "pending": 0, "processing": 0, "merged": 38, "rejected": 1, "flagged": 1 },
  "url": "https://data.marketninja.ru/v1/me/scrapes/b1f8b6b0-2f7d-4a1a-9a3c-2f6d8e9a1234/products"
}

We deliberately don't embed the scraped data in the event itself — GET the url field (same Bearer auth as everything else on this site, and it's billed against your normal daily quota, the same as calling that endpoint directly) to actually retrieve it. Two reasons: your daily data quota is only ever enforced on that one endpoint, and a thin payload means one webhook works the same whether the session had 1 item or 10,000.

user_id identifies which of your Market Ninja accounts the event belongs to — useful if you ever point more than one account's webhooks at the same URL (a shared automation endpoint, for instance) and need to tell them apart.

Verifying a delivery

Every request carries two headers:

• X-Marketninja-Event: scrape_session.completed
• X-Marketninja-Signature: t=1735689600,v1=5257a869e7bcd1d2...

The signature is HMAC-SHA256 over ${timestamp}.${rawBody}, using the signing secret shown once when you created the subscription. Verify it against the raw request body string exactly as received — never against a re-serialized JSON.stringify(JSON.parse(body)), since that isn't guaranteed to produce identical bytes (key ordering, whitespace) and will silently break the comparison.

import crypto from "node:crypto";

function isValidSignature(rawBody, signatureHeader, secret) {
  const parts = Object.fromEntries(
    signatureHeader.split(",").map((p) => p.split("="))
  );
  const expected = crypto
    .createHmac("sha256", secret)
    .update(`${parts.t}.${rawBody}`)
    .digest("hex");
  return crypto.timingSafeEqual(
    Buffer.from(parts.v1),
    Buffer.from(expected)
  );
}

Delivery, retries, and when we give up

• Any 2xx response is treated as delivered; the body isn't read.
• We don't follow redirects — a 3xx is treated the same as a hard failure.
• Each attempt times out after 5 seconds.
• A failed attempt (network error, timeout, 429, or 5xx) is retried on a backoff schedule: roughly 1 min, 5 min, 30 min, 2h, 6h, then 24h later — up to 7 attempts total, spanning about a day and a half — before that specific delivery is marked permanently failed. A 429 with a Retry-After header is honored instead of the fixed schedule.
• Any other non-2xx (e.g. 400, 404) is treated as permanent immediately — we assume retrying won't change your endpoint's mind.
• If a subscription accumulates enough consecutive failed attempts, it's automatically disabled — there's no way to re-enable a disabled subscription, only create a new one from the extension's options page.

Example: trigger an LLM pricing summary in n8n

A common shape: get notified, pull the data, hand it to an LLM, post the result somewhere. Two nodes do the Market-Ninja-specific part of this; everything after is entirely up to you.

1. A Webhook node as the trigger — HTTP Method: POST, any path you choose. This is the URL you register as the subscription's target.

2. An HTTP Request node right after it, configured to GET the incoming event's url field with Bearer authentication:

   • URL: {{ $json.body.url }}
   • Authentication: Generic Credential Type → Bearer Auth, with a stored credential holding one of your mn_live_... API keys.

   Its output is { data: [...], meta: {...} } — the same shape GET /me/scrapes/{id}/products always returns, one entry per submission with product (null for anything not yet merged).

3. From there, do whatever you actually want with $json.data — an LLM node summarizing pricing trends, a Slack/Telegram notification, a database write, or all three chained together. None of that is Market-Ninja-specific; wire it up like any other n8n workflow.