Synthetic Data Kit

Generate realistic test datasets for agent evaluation, including conversations, tool calls, and edge cases.

Generator Settings

Template

Row count20

Add noise/variations

Fields

iduser_messageexpected_responsecategorydifficulty

id	user_message	expected_response	category	difficulty
Configure settings and click "Generate Data"

Integration Code

import { createDataGenerator } from 'agent-tools-kit/evaluation'

const generator = createDataGenerator({
  template: 'conversations',
  count: 20,
  noise: false,
  seed: 42,  // reproducible generation
  model: 'gpt-4o',  // for LLM-augmented generation
})

const dataset = await generator.generate()

// Export in multiple formats
await dataset.toCSV('test-data.csv')
await dataset.toJSON('test-data.json')
await dataset.toHuggingFace('my-org/agent-eval-data')

// Use directly in evaluations
const eval = createEvalSuite({ testData: dataset })