Orchestration track
Route every task.
Never run out of fuel.
Task queue, pellets in order
chat · code · embed · summarize
Large pellet = power task, routed first to the node with the most free memory.
Node lanes, memory fit
Node A
24 GB unified
No model resident. All four candidates flagged too big for full-speed residency.
Fallback
Node B
illustrative
One model resident, 62% of memory budget used.
Primary
Remote
API endpoint
Used only when no local node passes the fit check.
Last resort
01
Fit before pull
Check memory budget and context length before any download.
02
Local first
Route to the node that keeps the model resident at full speed.
03
Escape to remote
Fall back to a remote endpoint only when no lane can take the task.
Node A and Node B values are illustrative, not measured.
PACCAR · Orchestration concept