Get the SDK

Guides/Handle failover

Guide

Handle failover

Detect that the current channel is no longer usable and move traffic onto another driver or another route — without the application rebuilding anything, and without silently dropping the DDOs already in flight.

A link can drop in the middle of an exchange—a cable pulled, a radio out of range, a peer that stops answering. Handling failover means detecting that the current channel is no longer usable and moving traffic onto another one—another driver to the same peer, or another route through a different node—without the application rebuilding anything and without silently dropping the DDOs already in flight.

All of it rests on one property: delivery is not tied to the driver.

Delivery is not tied to the driver

The application addresses a destination DIN and produces a DDO. The driver used for the next transfer is chosen by the core, not the application. So when the channel changes, the message does not:

the same DDOorigin · destination · typeset · timestamp · payload
was carried by
driver Adropped
now carried by
driver B
The channel changes; the message does not.

Origin and timestamp belong to the core, so retries and re-routing never alter the message's identity; the typeset and payload are the application's and are not touched. Only the channel through which the DME travels changes. That is why failover requires neither rebuilding the data nor a separate procedure per transport.

Two responses to a broken path

A broken path has two fixes, and they are different decisions:

A
B
A
B
A
C
B
Another channel to the same peer, or another sequence of nodes.

Link failover picks another channel to the same peer. A node can keep several transports active at once—Ethernet, Wi-Fi, BLE—ordered by the quality the deployment needs. That order is a policy, not a fixed property of the protocols: a battery-powered device may prefer a low-power link, while a gateway prefers capacity and latency.

Re-routing picks a different sequence of intermediate nodes when the peer is not directly reachable or when another path offers better conditions.

The two cooperate rather than compete: a route selects the next hop, and failover selects which driver to use to reach it.

Detecting a dropped link

The driver is the first to notice. A timeout, a disconnection, or a send error marks the current path unusable, and ACK metrics feed the running view of reachability and channel quality. A saturated queue is reported distinctly—ERROR_TX_QUEUE_FULL separates a full queue from a channel error or an unknown destination, so backpressure is not mistaken for a dropped link.

On that signal, the core evaluates the available channels using driver state, peer reachability, and observed quality, then selects the next option:

outgoing DDO
Ethernetusable?
send on Ethernet
Wi-Fiusable?
send on Wi-Fi
BLEusable?
send on BLE
queue · alternate route · or error
Each channel is tested in policy order; falling through the last one is not silent.

The switch can use a second direct channel to the same peer, or a route through a node that owns the required driver—which is where failover hands off to re-routing.

Re-routing in flight

Route recovery happens inside the core. The search can be explicit, before sending, or reactive: the core tries the known path first and starts routing only when direct delivery fails. When a next hop fails, the route is invalidated before its natural expiry, and if an alternative exists the core searches for it—bounded by a hop limit and a timeout—and re-delegates the DME to a new intermediate node.

A message being recovered moves through explicit states, which is exactly what keeps it in play instead of being dropped on first failure:

queued
delegatedto another node
retry waitbackoff
sent
error
Recovery states keep a message in play; error is the last resort, not the first.

Each intermediate node resolves only its own next hop using its local view, so the detour can cross a different technology at every step while origin and destination stay fixed in the DME.

Not losing messages: what is preserved, what is retried

"Without losing messages" is a matter of what the core does on failure, and it is precise:

The honest boundary is that this is at-least-once, not exactly-once. Retries and alternate paths can duplicate a DDO or change the order of arrival, and a saturated queue or exhausted retries can still produce an error. What failover guarantees is that a single dropped link does not drop the message: the core keeps it moving through retry, queueing, and re-routing. Making that safe is the application's job—deduplication and ordering live in the payload, defined by the typeset:

The core will not deduplicate or reorder for you; it preserves identity so that you can.

Avoiding oscillation: failover vs failback

An intermittent link must not cause the node to flip channels continuously. Hysteresis separates two events:

preferredchannel
fallbackchannel
Leaving is immediate; coming back is earned.

Failover is fast: the moment the active channel is unusable, traffic moves. Failback is deliberate: the node returns to the preferred channel only after a window of stability or when its quality beats the current path by a sufficient margin. Routing applies the same restraint—it does not change the next hop for minor, temporary variations, only when the advantage of the new route is stable enough.

Failover respects policy and time

Re-routing never bypasses trust boundaries. A path is considered only if every hop is both reachable and authorized—constraints come before optimization—and each node applies its own acceptance policy before forwarding, so an intermediary's refusal invalidates that path. Failover changes the channel, not the permissions.

Time survives the detour, too. Intermediate hops do not rewrite the origin timestamp, so the destination can still measure end-to-end delay after a re-route. If the change of topology also changes the reachable time reference, dATS resynchronizes and marks the temporal guarantee as suspended until it reconverges.

Configure it

Failover needs something to fail over to, and a policy for how hard to try:

C++ · libDaas
node.enableDriver(_LINK_INET4, "192.168.1.2:4000");   // preferred channel
node.enableDriver(_LINK_BLE,   "...");                // fallback channel

node.setOptions(/* queues, max retries, auto-forwarding on failed send */);
node.setDDOPolicy(/* retry with backoff */);

node.push(remote_din, ddo);   // on a dropped link, the core re-tries / re-routes
node.doPerform(PERFORM_CORE_THREAD);

Method names and signatures are illustrative; each SDK exposes the same behavior—driver ordering, retry policy, auto-routing—through its own idiomatic surface, and its reference documentation gives the exact types.

Invariants

Drop a link and watch the mesh re-route.

Open the Lab Simulator