testing-core-processors

Testing Core Error Processors

How to write integration tests for error processors in packages/core/src/processors/.

Prerequisites

• Build core before running tests: pnpm build:core (from repo root)
• If focused vitest runs fail to resolve @internal/test-utils/setup, the build step was skipped

Mock Model Pattern

Use MockLanguageModelV2 from @internal/ai-sdk-v5/test to simulate API errors and verify retry behavior.

import { APICallError } from '@internal/ai-sdk-v5';
import { convertArrayToReadableStream, MockLanguageModelV2 } from '@internal/ai-sdk-v5/test';

// Track calls and captured prompts
let callCount = 0;
const receivedPrompts: any[] = [];

const model = new MockLanguageModelV2({
  doGenerate: async ({ prompt }) => {
    callCount++;
    receivedPrompts.push(JSON.parse(JSON.stringify(prompt)));
    if (callCount === 1) {
      throw new APICallError({
        message: '...',
        url: '...',
        requestBodyValues: {},
        statusCode: 400,
        responseBody: '...',
        isRetryable: false,
      });
    }
    return {
      rawCall: { rawPrompt: null, rawSettings: {} },
      finishReason: 'stop',
      usage: { inputTokens: 10, outputTokens: 20, totalTokens: 30 },
      content: [{ type: 'text', text: 'response' }],
      warnings: [],
    };
  },
  doStream: async ({ prompt }) => {
    // Same error logic as doGenerate
    // IMPORTANT: Stream response must include all event types:
    return {
      rawCall: { rawPrompt: null, rawSettings: {} },
      warnings: [],
      stream: convertArrayToReadableStream([
        { type: 'stream-start', warnings: [] },
        { type: 'response-metadata', id: 'id-0', modelId: 'mock-model', timestamp: new Date(0) },
        { type: 'text-start', id: 'text-1' },
        { type: 'text-delta', id: 'text-1', delta: 'response text' },
        { type: 'text-end', id: 'text-1' },
        { type: 'finish', finishReason: 'stop', usage: { inputTokens: 10, outputTokens: 20, totalTokens: 30 } },
      ]),
    };
  },
});

Stream Mock Gotcha

The stream mock format requires stream-start, response-metadata, text-start, text-delta, text-end, and finish events. Using only text-delta + finish (the minimal format) will result in empty text output because the AI SDK expects the full event sequence. See prefill-error-recovery.test.ts for the reference pattern.

Test Structure

For each error processor, write at minimum:

1. Happy path: Processor catches the target error, modifies messages, retries successfully
   • Assert: agent.generate() succeeds, mock called 2x, retry prompt has expected modifications
2. Control test: Same scenario without the processor — error propagates
   • Assert: agent.generate() throws the expected error
3. Selectivity test: Processor ignores unrelated errors (e.g. rate limit 429)
   • Assert: Error propagates, mock called only 1x

Passing Messages with Tool Calls

When seeding conversation history for tool-related tests, pass messages as the second argument to agent.generate() or agent.stream() using the AI SDK message format:

const messages = [
  { role: 'user', content: 'Do something' },
  { role: 'assistant', content: [{ type: 'tool-call', toolCallId: 'some-id', toolName: 'myTool', args: {} }] },
  { role: 'tool', content: [{ type: 'tool-result', toolCallId: 'some-id', toolName: 'myTool', result: 'done' }] },
];
await agent.generate(messages);

Running Tests

# Run focused processor tests
npx vitest run packages/core/src/processors/my-processor.test.ts

# Run all processor tests
npx vitest run packages/core/src/processors/

# Full core test suite (slower)
pnpm test:core

Devin Secrets Needed

None for mock-based integration tests. For live API tests:

• ANTHROPIC_API_KEY — for testing against real Anthropic API
• OPENROUTER_API_KEY — for testing provider switching scenarios