> Discover all available pages from the documentation index: https://mastra.zisheng.pro/zh-TW/llms.txt

# 在 CI 中執行評分器

在 CI 管線中執行評分器，可提供量化指標來衡量 Agent 品質隨時間的變化。`runEvals` 函式會透過 Agent 或 Workflow 處理多個測試案例，並傳回彙總分數。

## 基本設定

你可以使用任何支援 ESM 模組的測試框架，例如 [Vitest](https://vitest.dev/)、[Jest](https://jestjs.io/) 或 [Mocha](https://mochajs.org/)。

## 建立測試案例

使用 `runEvals` 針對多個測試案例評估 Agent。此函式接受資料項目陣列，每個項目都包含 `input`，並可選擇提供 `groundTruth` 供評分器驗證。

```typescript
import { describe, it, expect } from 'vitest'
import { createScorer, runEvals } from '@mastra/core/evals'
import { weatherAgent } from './weather-agent'
import { locationScorer } from '../scorers/location-scorer'

describe('Weather Agent Tests', () => {
  it('should correctly extract locations from queries', async () => {
    const result = await runEvals({
      data: [
        {
          input: 'weather in Berlin',
          groundTruth: { expectedLocation: 'Berlin', expectedCountry: 'DE' },
        },
        {
          input: 'weather in Berlin, Maryland',
          groundTruth: { expectedLocation: 'Berlin', expectedCountry: 'US' },
        },
        {
          input: 'weather in Berlin, Russia',
          groundTruth: { expectedLocation: 'Berlin', expectedCountry: 'RU' },
        },
      ],
      target: weatherAgent,
      scorers: [locationScorer],
    })

    // Assert aggregate score meets threshold
    expect(result.scores['location-accuracy']).toBe(1)
    expect(result.summary.totalItems).toBe(3)
  })
})
```

## 了解結果

`runEvals` 函式會傳回包含以下內容的物件：

- `scores`：每個評分器在所有測試案例中的平均分數
- `summary.totalItems`：處理的測試案例總數

```typescript
{
  scores: {
    'location-accuracy': 1.0,  // Average score across all items
    'another-scorer': 0.85
  },
  summary: {
    totalItems: 3
  }
}
```

## 多種測試情境

為不同的評估情境建立個別測試案例：

```typescript
describe('Weather Agent Tests', () => {
  const locationScorer = createScorer({/* ... */})

  it('should handle location disambiguation', async () => {
    const result = await runEvals({
      data: [
        {
          input: 'weather in Berlin',
          groundTruth: {/* ... */},
        },
        {
          input: 'weather in Berlin, Maryland',
          groundTruth: {/* ... */},
        },
      ],
      target: weatherAgent,
      scorers: [locationScorer],
    })

    expect(result.scores['location-accuracy']).toBe(1)
  })

  it('should handle typos and misspellings', async () => {
    const result = await runEvals({
      data: [
        {
          input: 'weather in Berln',
          groundTruth: { expectedLocation: 'Berlin', expectedCountry: 'DE' },
        },
        {
          input: 'weather in Parris',
          groundTruth: { expectedLocation: 'Paris', expectedCountry: 'FR' },
        },
      ],
      target: weatherAgent,
      scorers: [locationScorer],
    })

    expect(result.scores['location-accuracy']).toBe(1)
  })
})
```

## 後續步驟

- 了解如何[建立自訂評分器](https://mastra.zisheng.pro/zh-TW/docs/evals/custom-scorers)
- 探索[內建評分器](https://mastra.zisheng.pro/zh-TW/docs/evals/built-in-scorers)
- 對[啟用記憶體的 Agent](https://mastra.zisheng.pro/zh-TW/docs/evals/evals-with-memory) 執行評分器
- 閱讀 [runEvals API 參考](https://mastra.zisheng.pro/zh-TW/reference/evals/run-evals)