The ERP never pushes

STP pulls supplier prices with a cursor, commits them before announcing them, and makes every consumer safe to run twice on the same message, so no failure leaves a price it can't account for.

Project
STP: Vehicle Cost Platform, part 6 of 10
Published
October 5, 2026
Reading time
3 min read

Integrating with a system you don't control means planning for it to be down, slow, reset or answering twice. The ERP is the system of record for supplier prices, and STP doesn't control it, so the integration was designed around one constraint: the ERP never calls STP. It never pushes prices and it never provides BOMs. The controller, one of STP's services, goes and gets what changed.

Pull, with a cursor

Every 5 seconds, the controller asks the ERP for the offers that changed since its cursor:

GET /components/updates?afterCursor=812&limit=100
  • Each page is one transaction. Its offers are written to PostgreSQL together with the new cursor, so a restart resumes exactly where the last committed page ended.
  • An unreachable ERP changes nothing. Nothing is written, the cursor doesn't move, and the next cycle tries again.
  • A cursor ahead of the ERP is an incident. If the ERP says the cursor is ahead of it (its catalog was reset), the cursor goes back to zero and an alert is logged, because silently rereading everything would hide it.

When everything works, the trip is short. Measured on the stack, pull to PostgreSQL to Kafka takes 16 to 25 ms, and Kafka to the backend under 10 ms. The delay a user notices is the pull interval itself, which is why it went from 30 seconds to 5.

Database first, then Kafka

Only after the transaction commits does the controller publish one component-metrics-updated event per changed offer to Kafka. PostgreSQL is the source of truth, and an event never announces something the database doesn't have.

ERPcontrollerPostgreSQLKafkabackendupdates after cursor 812?100 offers, next cursor 912offers + cursor, one COMMIT✗publish event e1kept in memory, retried after 5 s, 30 s, 2 min, 5 min, 10 minsame e1, same payload ✓e1: offer at revision 7every minute: reconcile revisions
The database commits before anything is announced. A failed publish is retried with the same event id, and reconciliation catches anything lost for good.

Publishing can still fail after the commit. The controller then keeps the event in memory and sends it again, with the same event_id and the same payload, after 5 s, 30 s, 2 min, 5 min and 10 min. That queue lives in memory, so a controller that crashes in between loses it.

The usual way to close that gap is a transactional outbox, where the event is written in the same transaction as the data and published from that table. STP closes it from the other side: every minute, each backend reconciles its in-memory prices and rates against the revisions in PostgreSQL. That catches a lost event whatever lost it, a controller crash or Kafka being down for longer than the retries, at the cost of rereading offer revisions every minute whether anything changed or not. In the worst case, a price reaches the backend a minute late, never wrong.

The retries live in the controller's memory, so a restart loses them; the backend's reconciliation every minute catches the price anyway, from PostgreSQL. Timings illustrative, on the real schedule.

Consumers that can see a message twice

Kafka delivers at least once, so every consumer has to be an idempotent consumer. Events are keyed by component id, so updates to one part stay in order, and each one carries a full snapshot of the offer at a given revision, not a patch, so applying it twice gives the same state. A consumer then does four things, in this order:

  1. It validates the message.
  2. It checks a processed_events table keyed by (event_id, consumer), and skips anything already seen.
  3. It applies the update only if the revision is strictly higher than the one in memory.
  4. It records the event and commits the Kafka offset.

A crash between two of those steps leads to a redelivery, and the redelivery has no effect. A malformed message is logged and its offset committed, so there's no dead-letter queue, but no infinite redelivery either.

From Kafkae7, rev 5e7, rev 5e9, rev 4garbled1. valid?parse the message2. new event?processed_events3. newer revision?than the one in memory4. applyrecord event, commitoffer in memory:rev 5e7, rev 5applied: the offer is now at rev 5· offset committede7, rev 5redelivered: e7 already processed, skipped· offset committede9, rev 4late: rev 4 isn't newer than rev 5, skipped· offset committedgarbledmalformed: logged, skipped· offset committed
A redelivered event stops at the second check, a late one at the third. Whatever happens, running the consumer twice on a message changes nothing.

Exporting without doubles

In the other direction, a published BOM is sent to the ERP as OData v4 JSON.

  • The payload is built once, when the export is triggered, and stored with the job, so every retry sends exactly the same bytes.
  • Every attempt carries an Idempotency-Key equal to the job id. The ERP deduplicates on it, so a response lost on the way back can't create the BOM twice, and a timeout can safely count as a failure and be retried.
  • Failed exports are listed in the admin view and can be replayed by hand.