Customize Routing Strategies in OmniRoute

Built-in Routing Strategies¶

OmniRoute’s combo engine ships 19 built-in routing strategies (see 🎯 Combos — The Flagship) . OmniRoute also provides zero-config auto routing combos . More docs on routing strategies can be found at OmniRoute Docs - Routing .

Define Customized Routing Strategies¶

In OmniRoute, customized routing strategies are configured through Combos (chains of upstream providers/models with specific rules) or by registering a custom JavaScript/TypeScript strategy implementation. We focus on defining customized routing strategies through Combos in this article.

The Combo schema¶

Every combo — whatever built-in strategy it uses — shares one unified top-level schema:

{ name: string; strategy: RoutingStrategy; models: ComboStep[]; config?: {...} }

models[] is an array of ComboSteps (src/lib/combos/steps.ts) — the same union is used by every strategy, and a plain "provider/model" string is also accepted as shorthand (it auto-normalizes to the kind: "model" shape):

type ComboStep =
  | { kind: "model"; model; providerId?; connectionId?; allowedConnectionIds?; weight; label?; tags?;
      prompt?; fallbackOnlyOnQuotaExhaustion? }
  | { kind: "combo-ref"; comboName; weight; label?; fallbackOnlyOnQuotaExhaustion? }
  | { kind: "provider-wildcard"; providerId; modelPattern; connectionId?; allowedConnectionIds?; weight; label? }

What differs between strategies is only which of these fields they actually read, and which extra strategy-specific keys live under config:

(reset-window, headroom, cache-optimized, fusion, pipeline are the remaining strategies from the full list in the previous section and aren’t detailed here.)

Nesting: a models[] entry can itself be another Combo¶

A models[] entry isn’t limited to a single provider/model — it can also be {"kind": "combo-ref", "comboName": "..."}, referencing another, already-created combo by name. This is the entire nesting mechanism: there’s no inline/anonymous {strategy, models} sub-object, and no nodes/targets key — comboName is always a lookup against existing combos, so the inner combo has to be created first, then referenced.

config.nestedComboMode (default "flatten") controls what happens to that reference:

• "flatten" — the referenced combo’s models[] is expanded inline into the parent’s flat list, in the child’s declared order. The child’s own strategy is not honored — it behaves like a priority list once flattened.

• "execute" — the combo-ref becomes a black-box unit: the parent strategy picks it as one opaque target, and if selected, the child combo actually runs its own strategy, retries, and sticky state (open-sse/services/combo/runtimeUnits.ts::executeRuntimeUnitCombo). This is what genuinely enables hierarchical/nested strategies. Only honored when the parent’s strategy is one of: priority, round-robin, random, strict-random, weighted, fill-first — scoring strategies (cost-optimized, context-optimized, reset-aware, auto) and legacy ones (lkgp, p2c, least-used) still flatten.

Safety rails: maxComboDepth (default 3, hard cap 10), cycle detection (validateComboDAG, visitedComboNames; diamond graphs allowed, back-references rejected), and a shared attemptBudget across the whole tree so nested retries can’t blow up request cost.

For example, a combo that performs weighted load balancing between two primary providers, and if both fail, falls back via priority to a cheap local model — first create the inner combo, then reference it from the outer one:

// 1. create the inner combo first
{
  "name": "weighted-primary-pool",
  "strategy": "weighted",
  "models": [
    { "kind": "model", "model": "anthropic/claude-3-5-sonnet", "weight": 70 },
    { "kind": "model", "model": "openai/gpt-4o", "weight": 30 }
  ]
}

// 2. reference it from the outer combo, with execute mode so it runs its own strategy
{
  "name": "my-resilient-route",
  "strategy": "priority",
  "config": { "nestedComboMode": "execute" },
  "models": [
    { "kind": "combo-ref", "comboName": "weighted-primary-pool" },
    "groq/llama-3.3-70b"
  ]
}

Add/Update Customized Combo Route Via the REST API¶

Trust the Notebook¶

You have to trust the notebook so that the widget shows up.

1. Open the Command Palette by pressing Ctrl + Shift + C (or Cmd + Shift + C on macOS), or by navigating to View > Command Palette.

2. Type Trust Notebook into the search bar.

3. Click or press Enter on the Trust Notebook command.

References¶

• OmniRoute @ GitHub