Customize Routing Strategies in OmniRoute

Built-in Routing Strategies¶

OmniRoute’s combo engine ships 19 built-in routing strategies (see 🎯 Combos — The Flagship) . OmniRoute also provides zero-config auto routing combos . More docs on routing strategies can be found at OmniRoute Docs - Routing .

Define Customized Routing Strategies¶

In OmniRoute, customized routing strategies are configured through Combos (chains of upstream providers/models with specific rules) or by registering a custom JavaScript/TypeScript strategy implementation. We focus on defining customized routing strategies through Combos in this article.

The Combo schema¶

Every combo — whatever built-in strategy it uses — shares one unified top-level schema:

{ name: string; strategy: RoutingStrategy; models: ComboStep[]; config?: {...} }

models[] is an array of ComboSteps (src/lib/combos/steps.ts) — the same union is used by every strategy, and a plain "provider/model" string is also accepted as shorthand (it auto-normalizes to the kind: "model" shape):

type ComboStep =
  | { kind: "model"; model; providerId?; connectionId?; allowedConnectionIds?; weight; label?; tags?;
      prompt?; fallbackOnlyOnQuotaExhaustion? }
  | { kind: "combo-ref"; comboName; weight; label?; fallbackOnlyOnQuotaExhaustion? }
  | { kind: "provider-wildcard"; providerId; modelPattern; connectionId?; allowedConnectionIds?; weight; label? }

What differs between strategies is only which of these fields they actually read, and which extra strategy-specific keys live under config:

(reset-window, headroom, cache-optimized, fusion, pipeline are the remaining strategies from the full list in the previous section and aren’t detailed here.)

Nesting: a models[] entry can itself be another Combo¶

A models[] entry isn’t limited to a single provider/model — it can also be {"kind": "combo-ref", "comboName": "..."}, referencing another, already-created combo by name. This is the entire nesting mechanism: there’s no inline/anonymous {strategy, models} sub-object, and no nodes/targets key — comboName is always a lookup against existing combos, so the inner combo has to be created first, then referenced.

config.nestedComboMode (default "flatten") controls what happens to that reference:

• "flatten" — the referenced combo’s models[] is expanded inline into the parent’s flat list, in the child’s declared order. The child’s own strategy is not honored — it behaves like a priority list once flattened.

• "execute" — the combo-ref becomes a black-box unit: the parent strategy picks it as one opaque target, and if selected, the child combo actually runs its own strategy, retries, and sticky state (open-sse/services/combo/runtimeUnits.ts::executeRuntimeUnitCombo). This is what genuinely enables hierarchical/nested strategies. Only honored when the parent’s strategy is one of: priority, round-robin, random, strict-random, weighted, fill-first — scoring strategies (cost-optimized, context-optimized, reset-aware, auto) and legacy ones (lkgp, p2c, least-used) still flatten.

Safety rails: maxComboDepth (default 3, hard cap 10), cycle detection (validateComboDAG, visitedComboNames; diamond graphs allowed, back-references rejected), and a shared attemptBudget across the whole tree so nested retries can’t blow up request cost.

For example, a combo that performs weighted load balancing between two primary providers, and if both fail, falls back via priority to a cheap local model — first create the inner combo, then reference it from the outer one:

// 1. create the inner combo first
{
  "name": "weighted-primary-pool",
  "strategy": "weighted",
  "models": [
    { "kind": "model", "model": "anthropic/claude-3-5-sonnet", "weight": 70 },
    { "kind": "model", "model": "openai/gpt-4o", "weight": 30 }
  ]
}

// 2. reference it from the outer combo, with execute mode so it runs its own strategy
{
  "name": "my-resilient-route",
  "strategy": "priority",
  "config": { "nestedComboMode": "execute" },
  "models": [
    { "kind": "combo-ref", "comboName": "weighted-primary-pool" },
    "groq/llama-3.3-70b"
  ]
}

Add/Update Customized Combo Route Via the REST API¶

References¶

• OmniRoute @ GitHub