Model Fallback Kit

Automatic failover across model providers with prioritized fallback chains.

Fallback Chain

1GPT-4oprimary

2Claude 3.5 Sonnet

3GPT-4o-mini

4Llama 3 70B

Fail primary model

Fail secondary model

Click "Simulate Request" to see the fallback chain in action

Integration Code

import { createFallbackChain } from 'agent-tools-kit/model-abstraction'

const chain = createFallbackChain({
  models: ['gpt-4o', 'claude-3.5-sonnet', 'gpt-4o-mini', 'llama-3-70b'],
  timeout: 30000,
  retryOn: [429, 500, 502, 503, 'ETIMEDOUT'],
  onFallback: (from, to, error) => {
    logger.warn(`Falling back from ${from} to ${to}: ${error}`)
  },
})

const response = await chain.complete({
  messages: [{ role: 'user', content: '...' }],
})
// Automatically tries next model on failure