> Discover all available pages from the documentation index: https://mastra.zisheng.pro/llms.txt

# 结合 Memory 使用 Evals

使用 `thread` 作用域 Memory（包括 Observational Memory）的 Agent 在运行时需要 thread ID。当 Evals 调用 Agent 时未提供该 ID，你会看到：

```text
ObservationalMemory (scope: 'thread') requires a threadId, but none was found in RequestContext or MessageList.
```

本页介绍针对启用了 Memory 的 Agent 运行 Mastra Evals 的三种可用模式、每种路径支持的功能，以及如何选择。三种方式的完整可运行复现均位于 [`examples/evals-with-memory`](https://github.com/mastra-ai/mastra/tree/main/examples/evals-with-memory)。

## 如何选择使用方式

| 目标                           | 方式                                                                            |
| ---------------------------- | ----------------------------------------------------------------------------- |
| 所有项目共享一次对话                   | [使用 `runEvals` 和全局 `targetOptions.memory`](#shared-thread-with-runevals)      |
| 每个项目使用独立 thread，专注于 CI 循环    | [对每个项目运行 `runEvals`](#per-item-threads-with-runevals)                         |
| 由已存储 `Dataset` 驱动的逐项目 thread | [使用内联任务的 `dataset.startExperiment`](#dataset-experiments-with-an-inline-task) |

预先填充 `RequestContext` 并设置 `MastraMemory` **并不是**将 Memory 传入 Agent 的受支持方式。解析 thread 时会读取 `args.memory.thread`；而 `RequestContext.MastraMemory` 是在 Agent 已经解析完 thread 后，由 `prepare-memory-step` 填充的。

## 使用 `runEvals` 共享 thread

`runEvals` 接受 `targetOptions`，并将其转发给 `agent.generate()`。传入 `memory: { thread, resource }` 会让所有数据项目针对同一个 thread 运行，适用于测试多轮对话中的召回能力。

```typescript
import { runEvals } from '@mastra/core/evals'
import { supportAgent } from './support-agent'
import { recallScorer } from '../scorers/recall-scorer'

const memory = await supportAgent.getMemory()
await memory!.createThread({ threadId: 'eval-thread', resourceId: 'ci-user' })

const result = await runEvals({
  target: supportAgent,
  scorers: [recallScorer],
  targetOptions: {
    memory: { thread: 'eval-thread', resource: 'ci-user' },
  },
  data: [
    { input: 'My order number is 12345' },
    { input: 'What is my order number?', groundTruth: '12345' },
  ],
})
```

`targetOptions` 对每次调用而言是**全局的**。目前 `RunEvalsDataItem` 不支持逐项目覆盖。

## 使用 `runEvals` 为每个项目创建 thread

当每个数据项目都需要自己的 thread 时（常见 CI 形式），请针对每个项目调用一次 `runEvals`，使用唯一的 `targetOptions.memory`，并自行聚合分数。

```typescript
import { randomUUID } from 'node:crypto'
import { runEvals } from '@mastra/core/evals'
import { supportAgent } from './support-agent'
import { recallScorer } from '../scorers/recall-scorer'

const memory = await supportAgent.getMemory()
const resourceId = 'ci-user'

const items = [
  { input: 'Cats are mammals', groundTruth: 'mammals' },
  { input: 'Dogs are mammals too', groundTruth: 'mammals' },
]

// `runEvals` returns `{ scores: Record<string, number>; summary: { totalItems } }`.
const scores: number[] = []
for (const item of items) {
  const threadId = `eval-${randomUUID()}`
  await memory!.createThread({ threadId, resourceId, title: item.input })

  const result = await runEvals({
    target: supportAgent,
    scorers: [recallScorer],
    targetOptions: { memory: { thread: threadId, resource: resourceId } },
    data: [item],
  })

  scores.push(result.scores[recallScorer.id])
}

const average = scores.reduce((a, b) => a + b, 0) / scores.length
```

> **备注:** 请在运行 Evals 前创建 thread。`thread` 作用域中的 Observational Memory 会从必须已存在的记录中读取数据。

## 使用内联任务运行 Dataset 实验

`dataset.startExperiment({ target: agent })` **不会**将 `memory` 选项转发给 Agent，只会转发 `requestContext`。要针对启用了 Memory 的 Agent 运行已存储的 Dataset，请使用内联 `task` 函数，并将 `{ threadId, resourceId }` 存入每个项目的 `metadata`。Scorer Pipeline 仍会照常运行。

```typescript
import { randomUUID } from 'node:crypto'
import { mastra } from '../index'
import { supportAgent } from '../agents/support-agent'
import { recallScorer } from '../scorers/recall-scorer'

const memory = await supportAgent.getMemory()
const resourceId = 'ci-user'

const items = [
  { input: 'Cats are mammals', groundTruth: 'mammals', thread: `ds-${randomUUID()}` },
  { input: 'Dogs are mammals too', groundTruth: 'mammals', thread: `ds-${randomUUID()}` },
]

for (const it of items) {
  await memory!.createThread({ threadId: it.thread, resourceId, title: it.input })
}

const dataset = await mastra.datasets.create({
  name: 'support-recall',
  description: 'Per-item memory via inline task + item metadata',
})

await dataset.addItems({
  items: items.map(it => ({
    input: it.input,
    groundTruth: it.groundTruth,
    metadata: { threadId: it.thread, resourceId },
  })),
})

const summary = await dataset.startExperiment({
  scorers: [recallScorer],
  task: async ({ input, metadata }) => {
    const { threadId, resourceId: rid } = (metadata ?? {}) as {
      threadId: string
      resourceId: string
    }
    const result = await supportAgent.generate(input as string, {
      memory: { thread: threadId, resource: rid },
    })
    return result.text
  },
})
```

内联 `task` 会接收项目的 `metadata`，因此每一行都能驱动自己的 thread，而不必更改 Agent 或任何 Scorer。完整配置请访问 [runEvals Reference](https://mastra.zisheng.pro/reference/evals/run-evals) 和 [Dataset Reference](https://mastra.zisheng.pro/reference/datasets/dataset)。

## 相关内容

- [在 CI 中运行 Scorer](https://mastra.zisheng.pro/docs/evals/running-in-ci)
- [运行实验](https://mastra.zisheng.pro/docs/datasets/running-experiments)
- [Observational Memory](https://mastra.zisheng.pro/docs/memory/observational-memory)
- [runEvals API Reference](https://mastra.zisheng.pro/reference/evals/run-evals)