Syncing a catalogue

How to keep a store in step with a feed that changes without telling you.

Paging

Search is cursor-paged. Pass the nextCursor you were given as cursor on the next call, and stop when it comes back null.

let cursor = undefined;
const products = [];

do {
  const res = await fetch(
    `https://api.dyj.one/v1/market/feeds/${FEED_ID}/items/search`,
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.DYJ_SECRET_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ cursor, limit: 200 }),
    },
  );
  if (!res.ok) throw new Error(`Feed sync failed: ${res.status}`);

  const page = await res.json();
  products.push(...page.items);
  cursor = page.nextCursor ?? undefined;
} while (cursor);

There is no offset parameter, on purpose. A feed is a live query, and rows shift under a paging client — an offset would silently skip products as the marketplace changes mid-sync. The cursor cannot.

Cadence

Hourly is plenty for most catalogues; daily is fine for a slow-moving one. Responses carry Cache-Control: private, max-age=60, which is the floor worth respecting — polling faster than once a minute returns the same answer and spends your rate limit.

Products that disappear

A piece leaves your feed when the designer delists it, when their plan lapses, when an admin suspends it, when they block your business, or when a re-price moves it out of your cost band. You are not asked to approve any of that, and none of it is an error.

Handle it by treating each full sync as authoritative: anything you hold that is not in the results is no longer orderable. Deactivate rather than delete — products come back, and keeping your own id stable means a returning piece keeps its reviews, its URL and its SEO history.

You will be told, too

When a product leaves a feed you carry, DYJ raises a notification in your dashboard — one per product, and one per business rather than one per feed if you run several. New products arriving are deliberately silent: that is the ordinary operation of an automatic feed, and announcing each one would bury the notices that matter.

Prices

retailMinor is recomputed on every read from the designer’s current cost and your current margin schedule. Store it with your product, refresh it on each sync, and do not hard-code it into anything long-lived. If a price moves more than you expect, check matchedRuleId — a cost change can move a piece into a different rule or a different tier of the same schedule.