Triple-buffering client state without ever blocking a reader

A risk check needs a client's confirmed balance on every single order, and it runs on the order path, which means it can't block, not even on a lock held for a handful of instructions. The problem is that the same balance is also being updated all the time, by another thread. The gateway has one states writer, which applies fills from the multicast stream of trade reports and snapshots coming back from the account service, both handed to it through queues. So one thread writes and another reads, the same data, constantly, and the reader can never be the one that waits.

Why not a mutex, or one shared struct

A mutex solves the correctness problem and creates the exact problem risk checks can't have: every read now has to acquire a lock that some writer might be holding, and however short that hold is, it's non-zero, unpredictable, and sitting directly on the hot path.

Dropping the lock and using one unguarded ClientState struct instead just trades a wait for a different failure: a reader can land in the middle of a write and get a torn read, an old Confirmed value next to a new Attempt value, both fields nonsense together.

An index and three buffers

The actual structure is three full copies of a client's state, and one atomic index saying which of the three is the one readers should trust right now:

alignas(64) std::vector<ClientState> States1;
alignas(64) std::vector<ClientState> States2;
alignas(64) std::vector<ClientState> States3;
alignas(64) std::vector<std::atomic<uint8_t>> Complete;

Complete[clientFd] holds 0, 1, or 2, and a reader's whole job is to load it once and pick a buffer:

[[nodiscard]] ClientState
GetClientState(const uint32_t clientFd) const noexcept
{
    uint8_t complete =
        Complete[clientFd].load(std::memory_order_acquire);
    return complete == 0 ? States1[clientFd]
        : complete == 1  ? States2[clientFd]
                         : States3[clientFd];
}

That returns a copy, not a reference, which matters: once a reader has it, nothing happening in another thread can change the numbers it's holding. And the buffer it copied from is one no writer is ever allowed to touch.

The writer never touches what readers see

The writer never aims at States[complete]. It aims one buffer ahead, and even then it doesn't write the new value directly, it accumulates a delta:

void SetClientAssets(const uint32_t clientId, const int64_t confirmed,
                     const int64_t attempt, uint16_t assetId) noexcept
{
    uint8_t complete =
        Complete[clientId].load(std::memory_order_relaxed);

    ClientState &toChange = complete == 0
        ? States2[clientId]
        : (complete == 1 ? States3[clientId] : States1[clientId]);

    for (size_t i = 0; i < MaxPositions; ++i)
    {
        if (toChange.AssetId[i] == assetId)
        {
            Deltas[clientId][complete].Confirmed[i] += confirmed;
            Deltas[clientId][complete].Attempt[i] += attempt;
        }
    }
}

toChange is only read here, to find which asset slot matches. What actually changes is Deltas[clientId][complete]. Every generation, meaning the stretch between two flushes, gets its own delta slot, and an update only ever adds to the current generation's slot. It never touches States1, States2 or States3, so a reader can't catch one of them mid-write.

Flushing: fold, publish, prepare

FlushTripleBuffer is what turns accumulated deltas into an actual published state, and it's worth walking in three pieces. First, it folds this generation's deltas into the not-yet-published buffer:

ClientState *curState = complete == 0
    ? &States2[clientFd]
    : (complete == 1 ? &States3[clientFd] : &States1[clientFd]);

for (size_t i = 0; i < MaxPositions; ++i)
{
    curState->Confirmed[i] += curDelta[complete].Confirmed[i];
    curState->Attempt[i] += curDelta[complete].Attempt[i];
}

Then it publishes, with a single atomic store:

complete = complete == 2 ? 0 : complete + 1;
Complete[clientFd].store(complete, std::memory_order_release);

One instruction, and every reader that loads Complete after it sees the fully-updated buffer. Never a partial one, because the store only happens once curState is completely folded.

The last piece is the one that isn't obvious. The buffer the writer stages into next isn't fresh: it's the one that was published two flushes ago, and it has missed every update since, which is two whole generations of them. Catching it up means adding both of those generations' deltas. So the flush first clears the slot the new generation is about to use, then folds all three slots into that buffer:

for (size_t i = 0; i < MaxPositions; ++i)
{
    curDelta[complete].Confirmed[i] = 0;
    curDelta[complete].Attempt[i] = 0;
    newState->Confirmed[i] += curDelta[0].Confirmed[i];
    newState->Confirmed[i] += curDelta[1].Confirmed[i];
    newState->Confirmed[i] += curDelta[2].Confirmed[i];
    newState->Attempt[i] += curDelta[0].Attempt[i];
    newState->Attempt[i] += curDelta[1].Attempt[i];
    newState->Attempt[i] += curDelta[2].Attempt[i];
}

With the new slot zeroed, "all three" is exactly the two generations that buffer missed. Each generation's deltas live for two flushes, until all three buffers have absorbed them, and get cleared when their slot comes around again. Following it once with the buffers starting at the same value V and one update per generation: after the first flush the next staging buffer gets d1, after the second it gets d1 + d2, and at the third, the first slot has just been cleared, so the buffer that was sitting at V + d1 gets d2 + d3. Every buffer lands on the same total, one flush behind the other.

Why three, not two

With two buffers, the only place left to stage into after a flush is the buffer that was published a moment ago, and a reader might still be in the middle of copying it. Writing into it then would hand that reader half old and half new values, the torn read from earlier, just moved somewhere else. A third buffer gives a full generation of slack: one is published, one may still be being copied, and the writer only ever touches the third. That holds as long as a reader finishes its copy before the writer has flushed that client twice, which a pinned risk thread copying a small struct always does in practice.

None of this is free: every client's state exists three times over, and every flush is real write work, not a pointer swap. But the alternative was a lock on the order path.