Compliance Scoring¶

Codaris scores a pipeline against a versioned ruleset derived from the nf-core specifications, and applies rollback-safe fixes for the rules it fails. The score describes how closely a pipeline follows the nf-core and Nextflow DSL2 standards. It is not a certification under any regulatory framework, and the report says so in those words.

The ruleset¶

Every rule has an id (CR-0001 to CR-0249), a title, a description, the tier, the method, the files it applies to, and the source it was derived from. /rules in the chat and codaris review --json expose the catalogue; the YAML is in backend/app/rulesets/.

Automated nf-core lint is deliberately outside the ruleset. Lint runs as its own step (codaris lint, and inside every build), and rule CR-0063 (Pass lint tests) records its outcome.

Profiles¶

codaris review --profile full or CODARIS_RULE_PROFILE=full selects the second.

Examples¶

Running a review¶

codaris review ./my-pipeline              # core profile, evaluator model for the heuristic rules
codaris review ./my-pipeline --quick      # deterministic checkers and nf-core lint only, no model
codaris review ./my-pipeline --json       # one JSON object with the score, the grade, and the report paths

In the chat the same audit is /audit (the counterpart of codaris audit), or /review --compliance; /review --quick is the quick pass. A bare /review is a different, cheaper engine: the structural wiring review (review_pipeline: channels, params, containers, configs; free, deterministic, a few seconds). Which engine answers is decided by the text you type, not by the model: a bare /review binds review_pipeline and nothing else, and /audit binds the scorecard tool and nothing else, so the same message always runs the same engine. A /review that asks something (/review are the channels wired?), names a path, or carries --security or --lint stays in Review mode, where the model picks the engine by what you asked.

The audit is read-only: it writes nothing into the pipeline (the report comes back in the answer; nf-core lint runs on a throwaway copy), and /audit cannot apply fixes; only /remediate (or /autofix) does. Ask for the report to be saved and it writes docs/compliance-report.json, and nothing else.

Every scored result opens with one line that carries its numbers, and the chat repeats it word for word rather than summarising from the tables:

COMPLIANCE SUMMARY: score 52.9% (grade F, graded: yes) | findings: 38 unaddressed (94 open checks) | audit agreement: 5/12 = 41.7% | checks evaluated: 384/386 | written: nothing (read-only)

graded: no replaces the letter when fewer than half of the checks produced a verdict; findings is the fix directives not applied and, in brackets, the checks that failed or warned; audit agreement is the share of a sample of verdicts that the second model confirmed on re-derivation, or no audit run; written: lists the files this call wrote, so a read-only run says nothing (read-only) and a run that applies fixes lists each file. The chat does not say a report was generated or saved unless that field lists it.

The command prints the Markdown report and closes with one line:

Review — overall 91.4% (grade A)
Report: /path/to/my-pipeline/docs/compliance-report.json

The command asks for docs/compliance-report.json inside the pipeline, with a scorecard per category and the findings per rule; remediate also writes the interactive HTML report, the score history and the fix plan. The score is weighted: each applicable rule counts its ruleset weight, so a REQUIRED failure costs more than a RECOMMENDED one, and the overall figure is the ratio over every applicable rule rather than an average of the categories. The grade is a letter band on the score (A+ from 97, A from 93, down to F below 60). The grade is given only when at least half of the checks produced a verdict. If the evaluator could not answer most of them, the report says Not graded: 17 of 386 checks evaluated and labels the score as the share of the evaluated checks that pass, rather than printing a letter over a handful of checks.

Remediation¶

codaris remediate ./my-pipeline
codaris remediate ./my-pipeline --allow-destructive --max-iterations 3

remediate (and /remediate in the chat) proposes a fix for each failed rule that has a deterministic repair, applies it with a backup, re-validates the file, and keeps the change only when a verifier model agrees it addresses the rule without breaking anything else. Each attempt is recorded as applied, rejected and rolled back, withdrawn, or skipped, and the closing summary counts them:

Remediation — overall 94.8% (grade A)
Applied 6, rejected/rolled back 1, withdrawn 0, skipped 2, errors 0.
  applied:
    …one line per fix, naming the rule and the file…
  rolled back:
    …the fixes the verifier rejected, with its reason…
Report: /path/to/my-pipeline/docs/compliance-report.json

Fixes that are not rollback-safe run only with --allow-destructive.

Containers¶

Every process the compiler writes declares a quay.io/biocontainers image with an explicit version tag, taken from the module's own declaration. A latest tag is a CR-0002 violation; a tag that the registry does not know is reported by the container check (CODARIS_CONTAINER_LOOKUP=0 turns the network lookup off for air-gapped hosts). Pinning by image digest is a choice you make in the pipeline; Codaris verifies the tag it writes and leaves the digest form to you.

The provenance record¶

The build writes .codaris/provenance.json after its last write to the tree:

{
  "codaris_signature": "CODARIS-SIG-667b27218694e713",
  "sha256_merkle_root": "667b2721…",
  "pipeline_name": "rnaseq",
  "manifest": {
    "conf/base.config": "…",
    "main.nf": "…",
    "modules/nf-core/fastqc/main.nf": "…",
    "workflows/rnaseq.nf": "…"
  },
  "method": "sha256 of each file under PROVENANCE_GLOBS, then sha256 over '<path>\\0<sha256>\\n' for every entry in path order",
  "scope": ["main.nf", "nextflow.config", "nextflow_schema.json", "conf/*.config", "workflows/**/*.nf", "subworkflows/**/*.nf", "modules/**/*.nf"],
  "computed_at": "2026-10-11T03:40:12+00:00",
  "verify": "python -m app.engine.security_scanner verify <pipeline_dir>"
}

The record identifies the files a build report describes. python -m app.engine.security_scanner verify <pipeline> recomputes the manifest and lists the files changed, added, or removed since; the exit code is 0 when nothing changed. The record is a content fingerprint computed by the same process that wrote the files; it carries no signature from a key and no claim about the build platform.