Quick Start

Get Cerberus running in your CI pipeline in under a minute.

1. Add the GitHub Action

.github/workflows/ci.yml
yaml
- uses: EvertonSt/cerberus-ci-action@v1
  with:
    ai-provider: claude
    ai-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
    github-token: ${{ secrets.GITHUB_TOKEN }}
    test-results-path: ./test-results/results.json

2. Run your tests with JSON output

Run your tests
bash
"comment"># Playwright
"function">npx playwright test --"keyword">reporter=json > test-results.json

"comment"># Jest
"function">npx jest --json --outputFile=test-results.json

"comment"># Vitest
"function">npx vitest "keyword">run --"keyword">reporter=json --outputFile=test-results.json

3. Cerberus runs automatically

On every PR, Cerberus will ingest your test results, classify failures, check for performance regressions, and post a quality report as a PR comment.

GitHub Action

Install Cerberus as a GitHub Action. Works with any test framework.

workflow.yml
yaml
- uses: EvertonSt/cerberus-ci-action@v1
  with:
    # Required
    test-results-path: ./test-results/results.json

    # AI Provider (default: claude)
    ai-provider: claude
    ai-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
    ai-base-url: ""  # only for openai-compatible

    # GitHub
    github-token: ${{ secrets.GITHUB_TOKEN }}

    # Optional
    config-path: cerberus.config.yml
    format: playwright-json  # or junit
    skip-report: "false"
    annotations: "true"  # inline PR feedback

Outputs

gate-result

pass or fail

flaky-count

Number of flaky tests

regression-count

Number of regressions

CLI Reference

npm install -g cerberus-ci — all 11 commands documented below.

cerberus init

Generate cerberus.config.yml with sensible defaults

Usage
cerberus init [--force]
Options
--force: Overwrite existing config file

cerberus ingest

Ingest test results from JUnit XML or Playwright JSON

Usage
cerberus ingest -i <paths> -f <format> --run-id <id> --commit <sha> --branch <branch> [--pr <number>]
Options
-i, --input <paths>: Path(s) to test result file(s), comma-separated
-f, --format <format>: junit | playwright-json
--run-id <id>: CI run identifier
--commit <sha>: Git commit SHA
--branch <name>: Git branch name
--pr <number>: Pull request number

cerberus classify

Classify failed tests as flaky or regression

Usage
cerberus classify --run-id <id>
Options
--run-id <id>: CI run identifier

cerberus gate

Determine if the CI gate passes or fails

Usage
cerberus gate --run-id <id> [--annotations]
Options
--run-id <id>: CI run identifier
--annotations: Emit GitHub Actions annotations

cerberus report

Generate quality report and post as PR comment

Usage
cerberus report --run-id <id> --pr <number> --repo <owner/repo>
Options
--run-id <id>: CI run identifier
--pr <number>: Pull request number
--repo <owner/repo>: GitHub repository

cerberus run

Run the full pipeline: ingest → classify → gate → report

Usage
cerberus run -i <path> -f <format> --run-id <id> --commit <sha> --branch <branch>
Options
-i, --input <path>: Path to test result file
-f, --format <format>: junit | playwright-json
--pr <number>: Pull request number
--repo <owner/repo>: GitHub repository
--no-report: Skip report generation
--annotations: Emit GitHub Actions annotations
--json: Output results as JSON

cerberus compare

Compare two CI runs side-by-side

Usage
cerberus compare --run-id <id> [--other-run-id <id>]
Options
--run-id <id>: Run to compare (the 'after' run)
--other-run-id <id>: Run to compare against (auto-selects previous if omitted)
--json: Output as JSON

cerberus trends

Analyze flaky test trends across runs

Usage
cerberus trends [--branch <name>] [-n <depth>]
Options
--branch <name>: Branch to analyze (default: main)
-n, --depth <count>: Number of runs to analyze (default: 50)
--json: Output as JSON

cerberus history

Show pass/fail history for a specific test

Usage
cerberus history --test "<name>" [--branch <name>]
Options
--test <name>: Full test name
--branch <name>: Branch to check (default: main)
-n, --depth <count>: Number of runs (default: 20)

cerberus status

Show classification status for a run

Usage
cerberus status --run-id <id>
Options
--run-id <id>: CI run identifier

cerberus baseline

Manage performance baselines from known-good runs

Usage
cerberus baseline <set|list|clear>
Options
set --run-id <id> [--label <text>]: Mark a run as baseline
list: List all baselines
clear [--run-id <id>]: Remove baseline(s)

Config Reference

Cerberus uses cerberus.config.yml for configuration. Generate a default config with cerberus init.

AI Provider

1ai:
2  provider: claude              # claude | openai-compatible | mock
3  model: claude-sonnet-4-6      # provider-specific model
4  base_url: null                # required for openai-compatible
5  api_key_env: ANTHROPIC_API_KEY # env var holding the API key

Classifier

1classifier:
2  consecutive_failures_threshold: 3  # auto-regression after N fails
3  history_depth: 5                    # recent runs to check
4  cache_ttl_days: 30                  # cache duration

Performance

1perf:
2  baseline_branch: main     # branch to compare against
3  baseline_runs: 10         # rolling median window
4  threshold_pct: 20         # regression threshold %
5  thresholds: {}            # per-metric overrides
6  exclude: []               # metrics to ignore

Gate

1gate:
2  fail_on_regression: true
3  fail_on_unknown: false
4  fail_on_perf_regression: true
5  max_new_flaky_tests: 3

Storage

1storage:
2  db_path: .cerberus/data.db

Supported Formats

Cerberus parses test results from any framework that outputs JUnit XML or Playwright JSON.

Playwright JSON

npx playwright test --reporter=json
Pass / fail / skip / timeout
Retry count (deduplication)
Duration per test
Error messages & stack traces

JUnit XML

Works with Jest, Cypress, pytest, vitest, and more
Standard JUnit XML format
Test suites & test cases
Failure messages
Time attributes