Skip to main content

API Reference

The reference implementation, written in TypeScript, provides utilities for working with CTRF reports and is maintained alongside the specification.

It serves as the canonical guide for implementing CTRF in any language.

Installation​

npm install ctrf@0.3.0

Quick Start​

import {
  validate,
  parse,
  ReportBuilder,
  TestBuilder,
  filterTests,
  merge,
} from 'ctrf'

// Validate a report
const result = validate(report)
if (result.valid) {
  console.log('Report is valid!')
}

// Parse JSON
const parsedReport = parse(jsonString, { validate: true })

// Build reports programmatically
const report = new ReportBuilder()
  .runId('run-2026-08-15')
  .tool({ name: 'jest', version: '29.0.0' })
  .addTest(
    new TestBuilder()
      .testId('math/adds-numbers')
      .executionId('execution-123')
      .name('should add numbers')
      .status('passed')
      .duration(150)
      .build()
  )
  .build()

Core Functions​

validate​

Validate a CTRF report against the JSON schema.

function validate(report: unknown, options?: ValidateOptions): ValidationResult

Parameters:

  • report - The object to validate
  • options.specVersion - Specific spec version to validate against

Returns: ValidationResult with valid boolean and errors array

Example:

import { validate } from 'ctrf'

const result = validate(report)
if (!result.valid) {
  console.log('Validation errors:', result.errors)
}

// Validate against specific version
const result = validate(report, { specVersion: '0.0.0' })

isValid​

Type guard to check if a report is valid.

function isValid(report: unknown): report is CTRFReport

Example:

import { isValid } from 'ctrf'

if (isValid(report)) {
  // TypeScript knows report is CTRFReport
  console.log(report.results.summary.passed)
}

validateStrict​

Validate and throw if invalid.

function validateStrict(report: unknown): asserts report is CTRFReport

Throws: ValidationError if the report is invalid

Example:

import { validateStrict, ValidationError } from 'ctrf'

try {
  validateStrict(report)
  // TypeScript knows report is CTRFReport
} catch (e) {
  if (e instanceof ValidationError) {
    console.log(e.errors)
  }
}

parse​

Parse a JSON string into a CTRFReport.

function parse(json: string, options?: ParseOptions): CTRFReport

Parameters:

  • json - JSON string to parse
  • options.validate - Enable schema validation (default: false)

Throws: ParseError if JSON is invalid, ValidationError if validation fails

Example:

import { parse } from 'ctrf'

const report = parse(jsonString)

// With validation
const report = parse(jsonString, { validate: true })

stringify​

Serialize a CTRFReport to JSON.

function stringify(report: CTRFReport, options?: StringifyOptions): string

Parameters:

  • report - The CTRF report to serialize
  • options.pretty - Enable pretty printing (default: false)
  • options.indent - Indentation size (default: 2)

Example:

import { stringify } from 'ctrf'

const json = stringify(report)

// Pretty print
const json = stringify(report, { pretty: true })

// Custom indent
const json = stringify(report, { pretty: true, indent: 4 })

calculateSummary​

Calculate summary statistics from an array of tests.

function calculateSummary(tests: Test[], options?: SummaryOptions): Summary

Parameters:

  • tests - Array of test results
  • options.start - Override start timestamp
  • options.stop - Override stop timestamp

Example:

import { calculateSummary } from 'ctrf'

const summary = calculateSummary(tests)

// With timing overrides
const summary = calculateSummary(tests, {
  start: 1704067200000,
  stop: 1704067260000,
})

Builders​

ReportBuilder​

Fluent builder for constructing CTRF reports.

class ReportBuilder {
  constructor(options?: ReportBuilderOptions)

  specVersion(version: string): this
  reportId(uuid?: string): this
  runId(id: string): this
  timestamp(date?: Date | string): this
  generatedBy(name: string): this
  tool(tool: Tool): this
  environment(env: Environment): this
  addTest(test: Test): this
  addTests(tests: Test[]): this
  insights(insights: Insights): this
  baseline(baseline: Baseline): this
  extra(data: Record<string, unknown>): this
  summaryOverrides(overrides: Partial<Summary>): this
  build(): CTRFReport
}

Options:

  • autoGenerateId - Auto-generate report UUID
  • autoTimestamp - Auto-set current timestamp

Example:

import { ReportBuilder, TestBuilder } from 'ctrf'

const report = new ReportBuilder({ autoGenerateId: true, autoTimestamp: true })
  .specVersion('0.0.0')
  .runId('run-2026-08-15')
  .tool({ name: 'jest', version: '29.0.0' })
  .environment({ branchName: 'main', commit: 'abc123', shardId: 'shard-1-of-4' })
  .addTest(
    new TestBuilder()
      .name('should add numbers')
      .status('passed')
      .duration(150)
      .build()
  )
  .addTest(
    new TestBuilder()
      .name('should handle errors')
      .status('failed')
      .duration(200)
      .message('Expected 5 but got 4')
      .build()
  )
  .build()

TestBuilder​

Fluent builder for constructing Test objects.

class TestBuilder {
  constructor(options?: TestBuilderOptions)

  id(uuid?: string): this
  testId(id: string): this
  executionId(id: string): this
  name(name: string): this
  status(status: TestStatus): this
  duration(ms: number): this
  start(timestamp: number): this
  stop(timestamp: number): this
  suite(suite: string[]): this
  message(message: string): this
  trace(trace: string): this
  snippet(snippet: string): this
  ai(ai: string): this
  line(line: number): this
  rawStatus(rawStatus: string): this
  tags(tags: string[]): this
  labels(labels: Record<string, LabelValue>): this
  type(type: string): this
  filePath(filePath: string): this
  retries(retries: number): this
  addRetryAttempt(attempt: RetryAttempt): this
  flaky(flaky: boolean): this
  stdout(stdout: string[]): this
  stderr(stderr: string[]): this
  threadId(threadId: string): this
  browser(browser: string): this
  device(device: string): this
  screenshot(screenshot: string): this
  parameters(params: Record<string, unknown>): this
  addStep(step: Step): this
  addAttachment(attachment: Attachment): this
  extra(data: Record<string, unknown>): this
  build(): Test
}

Options:

  • autoGenerateId - Auto-generate test UUID based on name/suite/filePath

Example:

import { TestBuilder } from 'ctrf'

const test = new TestBuilder({ autoGenerateId: true })
  .testId('authentication/validates-user-input')
  .executionId('execution-123')
  .name('should validate user input')
  .suite(['Authentication', 'Login'])
  .status('passed')
  .duration(245)
  .filePath('tests/auth/login.test.ts')
  .tags(['smoke', 'auth'])
  .labels({ priority: 'high', owners: ['qa', 'platform'] })
  .type('integration')
  .build()

Query & Filter​

filterTests​

Filter tests in a report by criteria.

function filterTests(report: CTRFReport, criteria: FilterCriteria): Test[]

FilterCriteria:

  • id - Legacy test UUID
  • testId - Stable logical test case ID
  • executionId - Specific test execution ID
  • name - Exact test name
  • status - Single status or array of statuses
  • tags - Tags to match (test must have all)
  • suite - Suite name to match
  • flaky - Filter by flaky status
  • browser - Browser name to match
  • device - Device name to match

Example:

import { filterTests } from 'ctrf'

// Filter by status
const failed = filterTests(report, { status: 'failed' })

// Multiple statuses
const notPassed = filterTests(report, {
  status: ['failed', 'skipped'],
})

// Multiple criteria
const filtered = filterTests(report, {
  status: 'failed',
  tags: ['smoke'],
  flaky: true,
})

findTest​

Find a single test in a report.

function findTest(
  report: CTRFReport,
  criteria: FilterCriteria
): Test | undefined

Example:

import { findTest } from 'ctrf'

// Find by stable logical test ID
const logicalTest = findTest(report, { testId: 'authentication/login' })

// Find one particular execution
const execution = findTest(report, { executionId: 'execution-123' })

// Find by name
const namedTest = findTest(report, { name: 'should login successfully' })

// Find by criteria
const failedFlakyTest = findTest(report, { status: 'failed', flaky: true })

Merge​

merge​

Merge multiple CTRF reports into a single report.

function merge(reports: CTRFReport[], options?: MergeOptions): CTRFReport

MergeOptions:

  • deduplicateTests - Remove duplicate tests by executionId, then testId, then legacy id (default: false)
  • mergeSummary - Recalculate summary (default: true)
  • preserveEnvironment - How to handle environments: 'first', 'last', 'merge' (default: 'merge')

Example:

import { merge } from 'ctrf'

// Basic merge
const merged = merge([report1, report2, report3])

// With deduplication
const merged = merge(reports, {
  deduplicateTests: true,
})

// Keep first environment only
const merged = merge(reports, {
  preserveEnvironment: 'first',
})

The merged report receives a new reportId. A shared runId is preserved only when every source report has the same value. The inputs remain unchanged: CTRF treats emitted reports as immutable artifacts. See Report Immutability for the document lifecycle rules.

Insights​

addInsights​

Add historical insights to a report by analyzing previous reports.

function addInsights(
  report: CTRFReport,
  historicalReports?: CTRFReport[],
  options?: InsightsOptions
): CTRFReport

InsightsOptions:

  • baseline - Baseline report for comparison

Example:

import { addInsights } from 'ctrf'

// Add insights from historical data
const reportWithInsights = addInsights(currentReport, previousReports)

// With baseline comparison
const reportWithInsights = addInsights(currentReport, previousReports, {
  baseline: baselineReport,
})

isTestFlaky​

Determine if a test is flaky based on CTRF specification.

function isTestFlaky(test: Test): boolean

A test is flaky if:

  • flaky field is explicitly true, OR
  • Test has retries > 0 AND final status is 'passed'

Example:

import { isTestFlaky } from 'ctrf'

if (isTestFlaky(test)) {
  console.log('Test is flaky:', test.name)
}

ID Generation​

generateTestId​

Generate a deterministic UUID v5 for a test based on its properties.

function generateTestId(properties: {
  name: string
  suite?: string[]
  filePath?: string
}): string

The same inputs always produce the same UUID, enabling cross-run analysis.

Example:

import { generateTestId } from 'ctrf'

const id = generateTestId({
  name: 'should add numbers',
  suite: ['math', 'addition'],
  filePath: 'tests/math.test.ts',
})
// Always returns the same UUID for these inputs

generateReportId​

Generate a random UUID v4 for report identification.

function generateReportId(): string

Example:

import { generateReportId } from 'ctrf'

const reportId = generateReportId()
// => 'f47ac10b-58cc-4372-a567-0e02b2c3d479'

Schema & Versioning​

getSchema​

Get the JSON Schema for a specific CTRF spec version.

function getSchema(version: string): object

Throws: SchemaVersionError if version is not supported

Example:

import { getSchema } from 'ctrf'

const schema = getSchema('0.0.0')

getCurrentSpecVersion​

Get the current spec version.

function getCurrentSpecVersion(): string

getSupportedSpecVersions​

Get all supported spec versions.

function getSupportedSpecVersions(): readonly string[]

schema​

The current version CTRF JSON Schema object.

import { schema } from 'ctrf'
console.log(schema.$schema)

Type Guards​

Runtime type checking functions.

isCTRFReport​

Quick check if an object has CTRF report structure.

function isCTRFReport(report: unknown): report is { reportFormat: 'CTRF' }

Example:

import { isCTRFReport } from 'ctrf'

if (isCTRFReport(data)) {
  // data.reportFormat is 'CTRF'
}

isTest​

Type guard for Test objects.

function isTest(test: unknown): test is Test

isTestStatus​

Type guard for valid test statuses.

function isTestStatus(status: unknown): status is TestStatus

isRetryAttempt​

Type guard for RetryAttempt objects.

function isRetryAttempt(attempt: unknown): attempt is RetryAttempt

hasInsights​

Check if a report has insights data.

function hasInsights(report: CTRFReport): boolean

Constants​

import {
  REPORT_FORMAT, // 'CTRF'
  CURRENT_SPEC_VERSION, // '0.0.0'
  TEST_STATUSES, // ['passed', 'failed', 'skipped', 'pending', 'other']
  SUPPORTED_SPEC_VERSIONS, // ['0.0.0']
  CTRF_NAMESPACE, // UUID namespace for deterministic IDs
} from 'ctrf'

Error Classes​

All errors extend CTRFError.

CTRFError​

Base error class for all CTRF errors.

class CTRFError extends Error {
  name: 'CTRFError'
}

ValidationError​

Thrown when schema validation fails.

class ValidationError extends CTRFError {
  errors: ValidationErrorDetail[]
}

interface ValidationErrorDetail {
  message: string
  path: string
  keyword?: string
}

ParseError​

Thrown when JSON parsing fails.

class ParseError extends CTRFError {
  cause?: Error
}

SchemaVersionError​

Thrown when an unsupported spec version is encountered.

class SchemaVersionError extends CTRFError {
  version: string
  supportedVersions: string[]
}

FileError​

Thrown when file operations fail.

class FileError extends CTRFError {
  filePath: string
  cause?: Error
}

BuilderError​

Thrown when builder validation fails.

class BuilderError extends CTRFError {}

CTRF Report Types​

These are the core schema types that define the CTRF structure. Import these to type your own reports.

CTRFReport​

The root report object.

interface CTRFReport {
  reportFormat: 'CTRF'
  specVersion: string
  reportId?: string
  runId?: string
  timestamp?: string
  generatedBy?: string
  results: Results
  insights?: Insights
  baseline?: Baseline
  extra?: Record<string, unknown>
}

Test​

Individual test result.

interface Test {
  id?: string
  testId?: string
  executionId?: string
  name: string
  status: TestStatus
  duration: number
  start?: number
  stop?: number
  suite?: string[]
  message?: string
  trace?: string
  snippet?: string
  ai?: string
  line?: number
  rawStatus?: string
  tags?: string[]
  labels?: Record<string, LabelValue>
  type?: string
  filePath?: string
  retries?: number
  retryAttempts?: RetryAttempt[]
  flaky?: boolean
  stdout?: string[]
  stderr?: string[]
  threadId?: string
  browser?: string
  device?: string
  screenshot?: string
  parameters?: Record<string, unknown>
  steps?: Step[]
  attachments?: Attachment[]
  insights?: TestInsights
  extra?: Record<string, unknown>
}
type LabelPrimitive = string | number | boolean
type LabelValue = LabelPrimitive | [LabelPrimitive, ...LabelPrimitive[]]

Results​

Container for test results.

interface Results {
  tool: Tool
  summary: Summary
  tests: Test[]
  environment?: Environment
  extra?: Record<string, unknown>
}

Summary​

Aggregated test statistics.

interface Summary {
  tests: number
  passed: number
  failed: number
  skipped: number
  pending: number
  other: number
  flaky?: number
  suites?: number
  start: number
  stop: number
  duration?: number
  extra?: Record<string, unknown>
}

Tool​

Test tool information.

interface Tool {
  name: string
  version?: string
  extra?: Record<string, unknown>
}

Environment​

Environment metadata.

interface Environment {
  reportName?: string
  appName?: string
  appVersion?: string
  buildId?: string
  buildName?: string
  buildNumber?: number
  buildUrl?: string
  repositoryName?: string
  repositoryUrl?: string
  commit?: string
  branchName?: string
  osPlatform?: string
  osRelease?: string
  osVersion?: string
  testEnvironment?: string
  shardId?: string
  healthy?: boolean
  extra?: Record<string, unknown>
}

TestStatus​

Valid test status values.

type TestStatus = 'passed' | 'failed' | 'skipped' | 'pending' | 'other'

Supporting Types​

interface Attachment {
  attachmentId?: string
  name: string
  contentType: string
  path: string
  extra?: Record<string, unknown>
}

interface Step {
  name: string
  status: TestStatus
  extra?: Record<string, unknown>
}

interface RetryAttempt {
  attempt: number
  attemptId?: string
  status: TestStatus
  duration?: number
  message?: string
  trace?: string
  line?: number
  snippet?: string
  stdout?: string[]
  stderr?: string[]
  start?: number
  stop?: number
  attachments?: Attachment[]
  extra?: Record<string, unknown>
}

interface Insights {
  passRate?: MetricDelta
  failRate?: MetricDelta
  flakyRate?: MetricDelta
  averageRunDuration?: MetricDelta
  p95RunDuration?: MetricDelta
  averageTestDuration?: MetricDelta
  runsAnalyzed?: number
  extra?: Record<string, unknown>
}

interface TestInsights {
  passRate?: MetricDelta
  failRate?: MetricDelta
  flakyRate?: MetricDelta
  averageTestDuration?: MetricDelta
  p95TestDuration?: MetricDelta
  executedInRuns?: number
  extra?: Record<string, unknown>
}

interface MetricDelta {
  current?: number
  baseline?: number
  change?: number
}

interface Baseline {
  reportId: string
  timestamp?: string
  source?: string
  buildNumber?: number
  buildName?: string
  buildUrl?: string
  commit?: string
  extra?: Record<string, unknown>
}

Complete Example​

import {
  ReportBuilder,
  TestBuilder,
  validate,
  stringify,
  filterTests,
  addInsights,
  merge,
  type CTRFReport,
  type Test,
} from 'ctrf'

// Build a report
const report = new ReportBuilder({ autoGenerateId: true, autoTimestamp: true })
  .runId('run-2026-08-15-api-tests')
  .tool({ name: 'vitest', version: '1.0.0' })
  .environment({
    branchName: 'main',
    commit: 'abc123',
    buildNumber: 42,
    shardId: 'shard-1-of-3',
  })
  .addTest(
    new TestBuilder({ autoGenerateId: true })
      .testId('api/users/creates-user')
      .executionId('execution-create-user-42')
      .name('should create user')
      .suite(['API', 'Users'])
      .status('passed')
      .duration(120)
      .filePath('tests/api/users.test.ts')
      .tags(['api', 'smoke'])
      .build()
  )
  .addTest(
    new TestBuilder({ autoGenerateId: true })
      .testId('api/users/deletes-user')
      .executionId('execution-delete-user-42')
      .name('should delete user')
      .suite(['API', 'Users'])
      .status('failed')
      .duration(85)
      .message('User not found')
      .trace('Error: User not found\n    at deleteUser (users.ts:42)')
      .filePath('tests/api/users.test.ts')
      .build()
  )
  .build()

// Validate
const result = validate(report)
console.log('Valid:', result.valid)

// Filter failed tests
const failed = filterTests(report, { status: 'failed' })
console.log('Failed tests:', failed.length)

// Serialize
const json = stringify(report, { pretty: true })

// Add insights from history
const reportWithInsights = addInsights(report, previousReports)

// Merge parallel runs
const merged = merge([shard1, shard2, shard3], {
  deduplicateTests: true,
})