Model Routing Kit

Intelligently route prompts to the best model based on complexity, cost, latency, and quality targets.

Routing Strategy

80
2000ms

Available Models

GPT-4o
Q:95$0.0051400ms
GPT-4o-mini
Q:82$0.00015380ms
Claude 3.5 Sonnet
Q:93$0.0031200ms
Claude 3 Haiku
Q:78$0.00025250ms
Llama 3 70B
Q:85$0.0007600ms

Integration Code

import { createRouter } from 'agent-tools-kit/model-abstraction'

const router = createRouter({
  models: ['gpt-4o', 'gpt-4o-mini', 'claude-3.5-sonnet', 'claude-3-haiku', 'llama-3-70b'],
  strategy: 'cost-optimized',
  constraints: {
    minQuality: 80,
    maxLatency: 2000,
    maxCostPerRequest: 0.05,
  },
  classifier: 'auto',  // auto-classify prompt complexity
})

const response = await router.complete({
  messages: [{ role: 'user', content: prompt }],
})
// response.model: 'claude-3-haiku'
// response.routingReason: 'cost-optimized: simple prompt'