Testing

Scenario tests are colocated with routes as run.test.ts files. Each scenario declares an input state and the expected output, and dawn test runs them against the real route runtime.

A minimal test

A scenario file is a default-exported array of plain scenario records — there is no describe() or test() wrapper, and the route is inferred from the file's directory:

export default [
  {
    name: "greets a tenant",
    input: { tenant: "acme" },
    expect: {
      status: "passed",
      output: { tenant: "acme", greeting: "Hello, acme!" },
    },
  },
]

Run all scenarios:

dawn test

Dawn discovers every run.test.ts under src/app/, invokes the route for each scenario, and evaluates the declarative expectation.

Scenario shape

Each entry in the default-exported array is { name, input, expect, run?, assert? }:

• name — human-readable scenario name.
• input — the JSON state passed to the route.
• expect — declarative expectation (see below).
• run? — optional run config: { url?: string } to target a live dev server.
• assert? — optional (result) => void callback for custom assertions, evaluated alongside expect.

The expect object accepts:

• status (required) — "passed" | "failed".
• output? — deep-equal match against the route's returned state.
• meta? — match against { mode, routeId, routePath, executionSource }.
• error? — for status: "failed", match { kind, message: string | { includes: string } }.

For programmatic assertions in assert(result), import the helpers from @dawn-ai/sdk/testing:

import { expectError, expectMeta, expectOutput } from "@dawn-ai/sdk/testing"
 
export default [
  {
    name: "custom assert",
    input: { tenant: "acme" },
    expect: { status: "passed" },
    assert: (result) => {
      expectOutput(result, { greeting: "Hello, acme!" })
      expectMeta(result, { mode: "agent", routeId: "/support/[tenant]" })
    },
  },
]

Against a live dev server

To exercise protocol parity against a running dawn dev, set run.url per-scenario. There is no command-level --url flag on dawn test.

export default [
  {
    name: "greets a tenant via dev server",
    input: { tenant: "acme" },
    run: { url: "http://127.0.0.1:3001" },
    expect: {
      status: "passed",
      output: { tenant: "acme", greeting: "Hello, acme!" },
    },
  },
]

Start the dev server first, then run dawn test:

dawn dev --port 3001 &
dawn test

Agents, retries, and middleware

Two cross-cutting features shape what scenarios assert:

• Agent retries — agent({ retry: { maxAttempts, baseDelay } }) retries on transient errors. To assert the exhausted-retry path, set expect.status: "failed" and use expect.error (or an assert callback with expectError). See Retry.
• Middleware — src/middleware.ts runs before every /threads/:id/runs/wait and /threads/:id/runs/stream request. Live-server scenarios (run: { url }) exercise middleware; in-process scenarios bypass it. A scenario whose middleware calls reject(...) should set expect.status: "failed" with a matching expect.error. See Middleware.

Mocking tools

Rules

1. 1

   File location

   run.test.ts must live in the route's directory, not a sibling or nested folder. Dawn matches tests to routes by directory.

2. 2

   Default-exported array

   The file's default export is the array of scenario records. There is no describe() or test() wrapper.

3. 3

   Use the helpers from @dawn-ai/sdk/testing for custom assertions

   expectOutput, expectMeta, and expectError cover deep-equal match (with helpful diffs). For partial or fuzzy matching today, use the declarative expect shape (output/meta/error) or write a custom predicate in assert(result).

4. 4

   Keep scenarios focused

   One scenario = one claim about the route's behavior. If you're adding branches, add more scenarios — don't inflate a single one.

CI

Use dawn verify as the integrity gate (it covers app, routes, typegen, and deps in one call), then run dawn test:

- run: pnpm exec dawn verify
- run: pnpm exec dawn test

dawn verify runs typegen and check internally, plus the deps check (missing packages, missing env vars) that bare dawn check && dawn typegen does not. dawn test exits non-zero on any failure and outputs a diff per mismatched scenario.

Unit-testing tools and middleware

The scenario harness drives a whole route through the runtime. Sometimes you want to test one unit in isolation — a single route tool, a FilesystemMiddleware, or ctx.fs-using code — without standing up an agent. @dawn-ai/testing ships three harnesses for this. They run against the real WorkspaceFs and filesystem backend over a temp directory, so real permission gating, realpath resolution, and parent-directory creation all apply.

All three are async create*Harness factories that return a handle with .close() and [Symbol.asyncDispose] — the same convention as createAgentHarness. Across @dawn-ai/testing, every harness/handle is created with a create* factory and torn down with close() (or await using).

A tool that uses ctx.fs

createToolHarness(tool) builds the DawnToolContext and gives you a reusable invoke(). Assert both the return value and what landed on disk via h.workspace.read(...):

import { afterEach } from "vitest"
import { createToolHarness } from "@dawn-ai/testing"
import type { DawnToolContext } from "@dawn-ai/sdk"
 
const saveNote = async (input: { name: string; body: string }, ctx: DawnToolContext) => {
  const { bytesWritten } = await ctx.fs.writeFile(`notes/${input.name}.md`, input.body)
  return { bytesWritten }
}
 
let h: Awaited<ReturnType<typeof createToolHarness>>
afterEach(() => h.close())
 
test("saveNote writes into the workspace", async () => {
  h = await createToolHarness(saveNote)
 
  const result = await h.invoke({ name: "todo", body: "ship it" })
 
  expect(result.bytesWritten).toBe(7)
  expect(await h.workspace.read("notes/todo.md")).toBe("ship it")
})

invoke() is reusable and shares one workspace across calls, so you can assert cumulative state across several invocations. Pass { workspace } to share a fixture you already own (the harness won't close it), or { permissions } to exercise allow/deny gating instead of the permissive default.

Testing ctx.fs code directly

createWorkspaceHarness() is the shared fixture the tool harness builds on. Use it directly when the code under test takes a WorkspaceFs. Seed the workspace with write, run your code against h.fs, and assert with read:

import { createWorkspaceHarness } from "@dawn-ai/testing"
import type { WorkspaceFs } from "@dawn-ai/sdk"
 
const appendLine = async (fs: WorkspaceFs, path: string, line: string) => {
  const prev = await fs.readFile(path)
  await fs.writeFile(path, `${prev}\n${line}`)
}
 
test("appendLine round-trips through the real WorkspaceFs", async () => {
  await using h = await createWorkspaceHarness()
  await h.write("log.txt", "first")
 
  await appendLine(h.fs, "log.txt", "second")
 
  expect(await h.read("log.txt")).toBe("first\nsecond")
})

The await using form auto-disposes the harness (cleaning up the temp dir) at the end of the block — the modern alternative to afterEach(() => h.close()). It works on all three harnesses.

A filesystem middleware

createMiddlewareHarness(mw) composes a FilesystemMiddleware over a temp localFilesystem and exposes the wrapped backend plus a ctx to call it with. assertForwardsAll() catches the most common middleware bug — silently dropping a backend method (required or optional) that the middleware doesn't intercept:

import { createMiddlewareHarness } from "@dawn-ai/testing"
import type { FilesystemMiddleware } from "@dawn-ai/workspace"
 
const uppercaseReads: FilesystemMiddleware = (next) => ({
  ...next,
  readFile: async (path, ctx) => (await next.readFile(path, ctx)).toUpperCase(),
})
 
test("uppercaseReads forwards every other backend method", async () => {
  await using h = await createMiddlewareHarness(uppercaseReads)
 
  await h.backend.writeFile("a.txt", "hello", h.ctx)
  expect(await h.backend.readFile("a.txt", h.ctx)).toBe("HELLO")
 
  // Fails loudly if the middleware forgot to spread a method like realPath.
  h.assertForwardsAll()
})

Related