Docs

Set up BonaFident

BonaFident makes small changes to your code and checks whether your tests fail. It is mutation testing, scoped to the pull request.

Start with a local audit, then add the check to your CI. The CI files below are generated by the same code the console uses.

Try it on your machine

Terminal
npx bonafident@latest audit --local

No account. Nothing leaves your machine. Needs Node 22+ and Jest or Vitest.

Runs on macOS and Linux. The one network request is the first-run download of the engine, pinned by version and checked against a SHA-256 compiled into the CLI. After that the CLI blocks its own network calls, and BonaFident credentials in your shell are removed from the engine’s environment.

The audit runs in shadow mode, so it never fails on a verdict. It prints the summary in the terminal and writes these files to .bonafident/audit/ in the repository (or to --out):

  • bonafident-audit.md and bonafident-audit.html: the readable report
  • bonafident-report.json: the engine’s report. A local audit quotes the changed code in it, because it stays on this machine.
  • bonafident.draft.yml: a drafted policy with inferred critical paths, when the engine finds any

--write also creates bonafident.yml in shadow mode from the draft. It never overwrites an existing one.

GitHub Actions, with OIDC

Commit this file as .github/workflows/bonafident.yml. It is the same file the console and the setup pull request write.

.github/workflows/bonafident.yml
name: BonaFident
on:
  pull_request:
    types: [opened, synchronize, reopened]
permissions:
  contents: read
  id-token: write
concurrency:
  group: bonafident-${{ github.event.pull_request.number }}
  cancel-in-progress: true
jobs:
  bonafident:
    runs-on: ubuntu-latest
    timeout-minutes: 25 # run.budget 10m + 15m for install and baseline
    steps:
      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
        with:
          fetch-depth: 0
          persist-credentials: false
      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
        with:
          node-version: 22
      - run: if [ -f pnpm-lock.yaml ]; then corepack enable && pnpm install --frozen-lockfile; elif [ -f yarn.lock ]; then corepack enable && yarn install; else npm ci; fi
      - uses: bonafident/run@c10fd0fa75341f80d24a29a6da0805697f620359 # untagged
        with:
          budget: 10m
  • pull_request only: one run per head commit. A push trigger is opt-in and runs every change twice.
  • permissions grants contents: read and id-token: write and nothing else. The job authenticates with a short-lived OIDC token, so no API key is stored on GitHub. The BonaFident GitHub App posts the check and the comment, so the job needs no write access.
  • Every action is pinned to a full 40-character commit SHA. The tag after # is a comment for humans and is never resolved.
  • timeout-minutes is the budget plus 15 minutes for checkout, install, the engine download and the baseline run. This file’s budget is 10m.
  • Pull requests from forks get no OIDC token from GitHub. The action skips them with a warning by default, so they are not audited.

bonafident/run is pinned to an untagged commit (c10fd0fa7534). The public v1.0.0 tag does not exist yet.

Make the check required

In the repository, open Settings > Rules and add a ruleset that requires status checks. Require the check named exactly BonaFident from the BonaFident GitHub App. Do not require the workflow job by its name: the App’s check is the one that carries the verdict.

GitLab CI

Merge this job into .gitlab-ci.yml. It runs on merge requests (merge_request_event) with the CLI pinned to exactly 0.3.0.

.gitlab-ci.yml
# Merge this job into .gitlab-ci.yml. Keep your existing jobs.
# Set BONAFIDENT_API_KEY as a masked, protected CI/CD variable. It is not written in this file.
bonafident:
  stage: test
  image: node:22-bookworm
  timeout: 25m
  id_tokens:
    BONAFIDENT_ID_TOKEN:
      aud: bonafident
  variables:
    GIT_DEPTH: "0"
    BONAFIDENT_BUDGET: "10m"
  script:
    - if [ -f pnpm-lock.yaml ]; then corepack enable && pnpm install --frozen-lockfile; elif [ -f yarn.lock ]; then corepack enable && yarn install; else npm ci; fi
    - npx --yes [email protected] run
  artifacts:
    when: always
    reports:
      codequality: .bonafident/out/gl-code-quality-report.json
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
  • The job signs in with a project API key. Add BONAFIDENT_API_KEY under Settings > CI/CD > Variables as a masked, protected variable. It is never written in the file.
  • To attest the run (the run page and the Evidence Ledger then record it as GitLab OIDC), also give the job a GitLab ID token named BONAFIDENT_ID_TOKEN with the audience bonafident, as below. The CLI sends it together with the key; without it the run is recorded as not attested.
  • The job also needs the project id as BONAFIDENT_PROJECT_ID. The console writes it into the job’s variables when you copy the job from there; otherwise add it as a CI/CD variable.
  • For a sticky merge request note, add GITLAB_API_TOKEN, a Project Access Token. Without it the job still reports through its exit code.
  • Revoke a key at any time in the console under API keys.
.gitlab-ci.yml
bonafident:
  id_tokens:
    BONAFIDENT_ID_TOKEN:
      aud: bonafident

Bitbucket Pipelines (report only)

Bitbucket gets a report, not a pull-request check. The CLI derives a pull-request diff only on GitHub and GitLab, so on Bitbucket it runs a full sweep within the budget. Set BONAFIDENT_API_KEY as a secured repository variable.

bitbucket-pipelines.yml
# Merge this pipeline into bitbucket-pipelines.yml if that file already exists.
# Set BONAFIDENT_API_KEY as a secured repository variable. It is not written in this file.
# Bitbucket runs a full sweep within the budget: the CLI derives a pull-request diff only on GitHub and GitLab.
image: node:22-bookworm
clone:
  depth: full
pipelines:
  pull-requests:
    '**':
      - step:
          name: BonaFident
          max-time: 25
          script:
            - export BONAFIDENT_BUDGET="10m"
            - if [ -f pnpm-lock.yaml ]; then corepack enable && pnpm install --frozen-lockfile; elif [ -f yarn.lock ]; then corepack enable && yarn install; else npm ci; fi
            - npx --yes [email protected] run

What is supported

Support matrix
AreaEntryStatus
Language or frameworkTypeScriptSupported
Language or frameworkJavaScript (ESM, CJS)Supported
Language or frameworkReact / JSXSupported
Language or frameworkNext.jsSupported
Language or frameworkVue SFCPartial
Language or frameworkSveltePartial
Test runnerJestSupported
Test runnerVitestSupported
CIGitHub Actions (PR check)Supported
CIGitLab CI (merge request report)Supported
CIBitbucket PipelinesReport only
LanguagePythonNot supported yet
LanguageGoNot supported yet
LanguageRustNot supported yet
LanguageJavaNot supported yet

Node 22 or newer: the engine’s test runtime requires it.

bonafident.yml

The file is optional. This is the smallest useful one: report only, an 80% floor, and one critical path.

bonafident.yml
version: 1
gate:
  mode: shadow
  global: 80
  strict_paths:
    - src/billing/**

Keys

bonafident.yml keys
KeyType and allowed valuesWhat it does
versionexactly 1 (required)Schema version. Only 1 exists.
gate.modeshadow or enforceshadow reports a would-block verdict and never fails the job; enforce blocks. The file can tighten the operator's mode to enforce, never loosen it.
gate.globalnumber, 0 to 100Minimum mutation score, in percent (80 means 80%). 0 sets no floor. A value above 0 and at most 1 is rejected as ambiguous: write 80, not 0.8.
gate.max_unprotectedinteger, 0 or moreMost survived mutants a pull request may add on the lines it changes. Never applies to a full sweep.
gate.strict_pathslist of stringsCritical paths (globs): a single survived mutant inside one blocks the merge.
gate.ignore_pathslist of stringsGlobs whose survived mutants never count toward the gate.
gate.on_incompleteblock or passWhat an enforce-mode gate does with an audit that stopped early. Default block.
scope.includelist of stringsGlobs to audit; empty audits everything not excluded.
scope.excludelist of stringsGlobs never audited.
scope.defaultsbooleanKeep the built-in exclusions (dependencies, build output, tooling, tests, generated files, dot-directories). Default true.
tests.commandstringThe test command. Detected from the repository when absent.
tests.envlist of strings, matching ^[A-Za-z_][A-Za-z0-9_]*$Names of job environment variables mutation workers may see. Exact names, never patterns; credentials are refused.
run.budgetstring, matching ^(?:[1-9][0-9]*h(?:[0-9]+m)?(?:[0-9]+s)?|[1-9][0-9]*m(?:[0-9]+s)?|[1-9][0-9]*s)$Time budget for evaluation, as a duration in hours, minutes and seconds: 10m, 1h30m, 90s. An audit that runs past it is incomplete, never a pass.
run.workersexactly "auto" or integer, 1 or moreParallel test workers: auto, or a positive integer.
report.sourcenone or fullnone (default): paths, lines, operators and digests only. full: also the changed expressions and their scopes.
mutation.profilestandard or strictstandard or strict (default). Scores from different profiles are not comparable.

Where it is read from

  • bonafident.yml at the repository root, or the path given to --config.
  • Older locations are still read, with a deprecation warning that names the git mv to run: .bonafident.yml, .gitlab/bonafident.yml, .github/bonafident.yml, .github/bonafident.yaml. When the root file exists, they are ignored with a warning.
  • An invalid file never falls through to another location.

Strictness

  • version: 1 is required.
  • An unknown key exits 2 in every mode, shadow included. The message names the key, suggests the closest valid one and lists the valid keys at that level.
  • The pre-v1 spellings gate.score, gate.strict and gate.ignore are rejected with their replacements (gate.global, gate.strict_paths, gate.ignore_paths). tests.coverage is rejected: no engine ever read it.

Units and precedence

  • gate.global is a percentage: 80 means 80%. A value above 0 and at most 1 is rejected as ambiguous.
  • A command-line flag beats the file, the file beats BONAFIDENT_* environment variables, and those beat the defaults.
  • The cloud policy is applied after that and may only tighten gate.*. It never changes run.*.

Suppress a mutant

Suppress with a reason, in the diff, where reviewers see it. Every honoured ignore is listed in the report, so an ignore cannot become a silent bypass.

Inline directives
// bonafident-ignore-next-line log text, asserted by a snapshot elsewhere
logger.info(`charged ${amount}`)

// bonafident-ignore-next-line SLR,NLR: retry banner copy is not behaviour
const banner = retries > 0 ? 'Retrying' : 'Ready'

// bonafident-ignore-file generated by openapi-codegen on every release
  • // bonafident-ignore-next-line <reason> covers every mutant on the next line; // bonafident-ignore-next-line SLR,NLR: <reason> only those operators.
  • // bonafident-ignore-file <reason> covers the whole file.
  • The reason is required and must be at least 8 characters. A directive without one is not honoured, and the report says why.
  • Your Stryker comments work as they are: // Stryker disable next-line all: <reason> and // Stryker disable all … // Stryker restore all.
  • Inside gate.strict_paths no directive is honoured. Suppressing a critical path is a policy change (gate.ignore_paths in bonafident.yml, reviewed like any other), or a risk acceptance on the pull request.
  • In the console, "Not useful" on a survivor hides it and never changes a verdict.

Exit codes

Exit codes
CodeMeaningSet by
0Passed, shadow mode, or nothing in scope. EXIT_OKengine, CLI
1The gate blocked the change in enforce mode. EXIT_GATE_BLOCKEDengine, CLI
2Usage or configuration error (unknown command, flag or bonafident.yml key). EXIT_USAGEengine, CLI
3The engine finished with every report written and the gate blocked. EXIT_ENGINE_GATE_BLOCKEDengine only
4The audit is incomplete (CLI: and gate.on_incomplete blocks in enforce mode). EXIT_AUDIT_INCOMPLETEengine, CLI
5Nothing BonaFident can audit: no supported language or test runner. EXIT_UNSUPPORTED_REPOSITORYengine, CLI
70The engine stopped on an internal error and produced no verdict. EXIT_ENGINE_INTERNALengine only
  • The CLI passes the engine’s code through, except 3, which it reports as 1, and 4, which it reports as 0 unless the incomplete audit blocks.
  • Shadow mode exits 0 for a would-block verdict and for an incomplete audit. Exit 2 still applies: a broken policy file is not a verdict.
  • A billing response (402) never sets the exit code.

CLI commands

Install nothing: run every command with npx bonafident@latest. --version and --help need no credentials and no network.

bonafident audit [options] [-- engine flags]

A local audit in shadow mode: survived mutants are reported, never enforced. No account and no credentials: the CLI removes BonaFident credentials from the engine’s environment and blocks its own network calls once the engine is installed. On a branch ahead of the default branch it audits the changes since the merge-base; otherwise the whole repository within the budget. bonafident audit --local still works: every audit is local.

--repo <path>
Repository to audit. Default: the current directory.
--base <ref>
Audit only the changes since this ref.
--all
Audit everything in scope, not only the branch’s changes.
--budget <duration>
Evaluation budget, for example 10m.
--out <dir>
Where the reports go. Default: <repo>/.bonafident/audit.
--write
Also write bonafident.yml in shadow mode from the drafted policy. Never overwrites an existing file.

bonafident run [--] [engine flags]

For CI. Runs the pinned engine on the runner, uploads bonafident-report.json and applies the gate. bonafident with no command does the same. Every flag the CLI does not own goes to the engine; a -- before them is optional.

--base <ref>
Engine flag: the ref the pull request is diffed against, for example origin/main. Without it the CLI uses BONAFIDENT_BASE, then the CI’s pull-request target.
--budget <duration>
Engine flag: the evaluation budget, for example 10m. Beats run.budget in bonafident.yml.
--workers <n>
Engine flag: parallel test workers.

bonafident init [options]

Writes bonafident.yml (v1, shadow mode) and the BonaFident CI workflow from the same template the console uses. The provider is detected from existing CI files and the origin remote; the package manager from the lockfile. Existing files are kept; .gitlab-ci.yml and bitbucket-pipelines.yml are never overwritten: the job is written to .bonafident/ci/ for you to merge.

--repo <path>
Repository root. Default: the current directory.
--provider <name>
github, gitlab or bitbucket, instead of detecting it.
--package-manager <name>
npm, pnpm or yarn, instead of detecting it.
--budget <duration>
The budget written into the workflow.
--force
Overwrite existing bonafident.yml and GitHub workflow files.
--dry-run
Print what would be written; write nothing.

bonafident doctor [--repo <path>] [--cloud]

Checks Node.js, git, the test runner, the pinned engine, bonafident.yml and the diff scope, and prints a fix for every problem. Makes no network request unless BonaFident Cloud is checked. Exits 0 when nothing failed (warnings allowed), 2 when a check failed.

--repo <path>
Repository root. Default: the current directory.
--cloud
Also check the credentials with BonaFident Cloud. Done anyway when run credentials are set.

bonafident debug <filepath> [--repo <path>]

Replays a source file’s tests (or the named test file) under Vitest with setupFiles and globalSetup cleared, isolate on and no file parallelism, to reproduce a test entry that fails to start. Nothing is uploaded.

--repo <path>
Repository root. Default: the current directory.

bonafident help [command]

Prints usage. Also bonafident --help, -h, and bonafident <command> --help. Needs no credentials and no network.

bonafident --version [--json]

Prints the CLI version and the pinned engine version with the SHA-256 prefix for this platform. Also bonafident version and -V. Needs no credentials and no network.

--json
One JSON object: cli, engine, platform, engine_sha256.

bonafident <command> [global options]

Valid on every command, before any --.

--json
One JSON result object on stdout; everything else goes to stderr.
--output <file>
Also write the JSON result to this file.
--quiet, -q
Only warnings, errors and the final line.
--no-color
No colour (also NO_COLOR).

Error ids

Every CLI error prints a stable id next to the message, so a log line can be searched. The CLI’s README lists the fix for each.

CLI error ids
IdCause
BF-E010Credential rejected or missing (401).
BF-E011Credential cannot access this project (403).
BF-E012Plan limit or billing (402). Nothing blocks your build.
BF-E020Usage or configuration.
BF-E021Git context (commit SHA or remote).
BF-E030Network, including the engine download.
BF-E031Timeout talking to BonaFident Cloud.
BF-E032BonaFident Cloud error or busy (5xx).
BF-E033Run or project not found (404).
BF-E040A run already exists for this commit (409).
BF-E041Report upload failed.
BF-E050Engine failure: baseline red, crash, or no certified score.
BF-E060Gate blocked.
BF-E099Unexpected API response.