Changelog

All notable SecHelix release changes are summarized here. Detailed release notes live in docs/releases/, and the Git history remains the authoritative development record.

[Unreleased]

• First measurement on real code: evals/cve_pairs.py reviewed the vulnerable and patched trees of 12 public CVEs blind. SecHelix found 2 of 12 known defects under the pre-registered rule and 3 with disclosed hand adjudication; no finding it marked VERIFIED was a known defect, and 10 of the 14 VERIFIED findings were in patched trees. Published as measured in docs/research/cve-pairs-2026-09-21.md, with the manifest, every adjudication and the scored result. Findings about other projects that do not match a published CVE are counted but withheld.
• README: the non-claims section now states that result, and corrects "refuses to report unproven claims" — unproven claims are labelled, not dropped (92 of 106 in that run).

[4.0.0-alpha.7] - 2026-09-21

• The installed skill now ships instructions and data only. sechelix_core, the scanner adapters, the report renderer and six scripts are no longer copied into skills/sechelix/: 52 Python files that no review step executes, and executable surface that every install-time security audit has to judge. The bundle drops from 160 files / 2.0 MB to 102 files / 1.46 MB with zero executable files.
• The optional helpers are unchanged and stay where they are installable: python -m pip install sechelix for the CLI and the sechelix_core modules SKILL.md names, and a repository clone for the release gate, renderer, adapters and validators. references/runtime.md and SKILL.md now state which source provides what, and that the workflow requires neither.
• scripts/validate_skill.py fails if any executable file returns to the bundle, with a regression test beside it. The "portable isolation" claim in the compatibility matrix is restated to match what now ships.

See release notes.

[4.0.0-alpha.6] - 2026-09-15

• Added distributions/awesome-copilot/, a curated runtime-free Agent Plugin with one skill, sechelix-lite: a 197-line SKILL.md and five references for evidence-first AppSec review with independent verification. It carries no SEO or cleanup workflow, catalog, runtime or product copy. The full skill is unchanged.
• Added scripts/validate_distribution.py to CI. It enforces the edition's line, file and byte budgets, Agent Plugins v1.0.0 manifest rules, release-version match, runtime independence, schema vocabularies, scope and secret checks, and a copy-install. Mutation tests cover each gate.
• Trimmed the canonical SKILL.md from 483 to 433 lines by moving runtime resource lists and commands to references/runtime.md; no methodology rule was removed. It also corrected a stale "fifteen" schema count (the tree has 22), and check_doc_consistency.py now checks that file.
• sync_portable_skill.py now prunes stale files. The portable bundle drops a stale agents/README.md, the deliberately vulnerable demo app and derived report renderings: 175 to 160 files.
• README: added what SecHelix does not claim, and moved the optional SEO/cleanup workflows out of the security usage path.
• CI: the commit-hygiene body-length rule no longer applies to Dependabot's generated release notes (#88); CodeQL and deploy-pages actions bumped (#87).

See release notes.

[4.0.0-alpha.5] - 2026-09-13

• Added agent-guided SEO Audit and Codebase Cleanup workflows, with evidence-backed findings, risk/impact estimates, staged remediation and verification.
• Added audit-only and combined audit/fix/verify prompts to the command cookbook and routed discovery adapters through the canonical Skill.
• Preserved security catalog/gate semantics and the optional runner at 0.3.0; no new CLI scanner commands.
• Corrected the canonical Skill path in AGENTS.md and synchronized the portable distribution.

See release notes.

[4.0.0-alpha.4] - 2026-09-09

• Shortened the action.yml description from 188 to 113 characters. GitHub Marketplace rejects a listing whose description is 125 characters or more, so the action could not be published; a folded YAML scalar hid the length from anyone counting lines rather than characters. No behaviour changed — inputs, outputs, steps and the fail-closed decision logic are exactly what 4.0.0-alpha.3 shipped.
• Added ActionMarketplaceMetadata tests: the description must be under 125 characters and a single line, branding.color must be one of the eight colours Marketplace accepts, and the outcome vocabulary dropped from the description must still appear in docs/github-action.md. Confirmed red against the old 188-character string.
• Moved the action pin in the README, docs/github-action.md and the example workflow to v4.0.0-alpha.4. The optional Python runner is unchanged at 0.3.0.

See docs/releases/4.0.0-alpha.4.md for the full notes.

[4.0.0-alpha.3] - 2026-09-09

• Removed the skills and agents top-level fields from plugin.json. Agent Plugins v1.0.0 sets additionalProperties: false and discovers both by directory convention, so the fields pointed at directories discovery already finds and failed schema validation. Flagged by GitHub's awesome-copilot intake on a real submission.
• Added tests asserting the manifest carries no key outside the v1.0.0 set, and that the conventional skills/ and agents/ directories are actually present — a pointer is only safe to remove while the convention it pointed at holds.
• Moved the action pin in the README, docs/github-action.md and the example workflow to v4.0.0-alpha.3. The optional Python runner is unchanged at 0.3.0.

See docs/releases/4.0.0-alpha.3.md for the full notes.

[4.0.0-alpha.2] - 2026-09-09

Distribution

• Moved the GitHub Action to the repository root so it is usable as uses: omarmohelal/SecHelix@<tag> and eligible for GitHub Marketplace. It reports PASS, PASS_WITH_KNOWN_RISK, BLOCKED or INCOMPLETE, decides INCOMPLETE before findings are examined, writes SARIF that says so explicitly rather than an empty results array, and passes every input through the environment instead of interpolating it into a shell.
• Published the MCP adapter to the official MCP Registry as io.github.omarmohelal/sechelix, verified by GitHub OIDC and a PyPI ownership token, with a publish workflow that refuses a dangling or mismatched entry.
• Added examples/expense-api: one reproducible 90-second demo of a real finding and a refuted false positive, driven by CI in the vulnerable, patched and reverted states so it cannot silently stop working.
• Added a separate SecHelix Challenge repository with ten cases, three of them decoys, and reproducible scoring.

Security

• Fixed two defects a SecHelix review of the new Action found before merge: the action uploaded the unredacted stdout projection of a run as a build artifact, routing around the redactor; and any output value could forge a second $GITHUB_OUTPUT entry, including an outcome=PASS that would win over a real outcome=BLOCKED. Both carry regression tests confirmed red first.
• Filed the underlying CLI behaviour separately rather than changing a published --json contract inside a distribution release (#76).

Unchanged

• No security semantics changed. The full SecHelix workflow remains NOT_MEASURED, and the 76-case blind-label result keeps its stated boundary.

See docs/releases/4.0.0-alpha.2.md for the full notes.

Runner 0.3.0 - 2026-09-09

• Kept sechelix_runner.RUNNER_VERSION, pyproject.toml, server.json and its package entry in agreement, and added a test that fails when they drift. The PyPI publish gate caught the first drift and refused the release; the test makes it visible before the push instead of after it.

• Added server.json and .github/workflows/publish-mcp.yml so the MCP adapter is registered at the official MCP Registry as io.github.omarmohelal/sechelix. Authentication is GitHub OIDC; no registry token is stored.

• Carried an mcp-name ownership token in the published package description so the registry can verify that the PyPI project and the registry namespace are the same publisher.

• The publish workflow refuses to run when server.json, pyproject.toml and the package entry disagree, or when the PyPI release it points at does not exist or lacks the ownership token. A registry entry naming a version nobody published is a dangling install instruction.

• tests/test_server_json.py asserts the published listing matches the code: tool count, stdio transport, sechelix mcp remaining a real subcommand, no shell reachable from the adapter, no credentials required, and path confinement actually refusing ../.., absolute paths and symlink-style escapes.

• No runtime behaviour changed.

Runner 0.2.0 - 2026-09-04

• Added Gemini CLI as a second provider-neutral reasoning executor using the official headless JSON interface. The adapter launches each node in a fresh empty working directory, applies system-level no-tool/MCP/extension/skill settings, suppresses normal context-file discovery, and rejects any reported tool call.
• Kept provider authentication owned by the official CLI; SecHelix does not harvest or reuse Gemini credentials.
• On Windows npm installs, bypasses the gemini.cmd shell boundary and invokes the official package's Node entry point directly. If that layout cannot be resolved, execution fails closed.
• sechelix doctor now reports Claude Code and Gemini CLI availability, while --executor accepts claude-code or gemini-cli.

Runner 0.1.1 - 2026-09-04

• Published the independently versioned Python runner to PyPI using GitHub OIDC Trusted Publishing; no long-lived PyPI token is stored in the repository.
• Fixed Claude Code 2.1.248 compatibility where an explicitly successful JSON envelope ending with stop_sequence could accompany a non-zero process status. The adapter accepts only the narrow successful-completion case; truncation, tool continuation, malformed output and explicit errors still fail closed.
• Added an explicit runner-release.json publication marker and CI guards binding the marker, pyproject.toml, and RUNNER_VERSION before release.

[4.0.0-alpha.1] - 2026-09-03

V4 evidence runtime

• Added the optional standard-library-only runner with deterministic DAG orchestration, least-context routing, budget/coverage state, replayable evidence, loopback API/MCP integration and fail-closed provider execution.
• Added bounded LOCAL proof execution for authorization/IDOR, traversal, race/idempotency, webhook and SSRF; hardened HTTP proofs to literal loopback with no DNS names, ambient proxies or automatic redirects. Proof behavior never auto-promotes a finding.
• Added graph-grounded threat modeling, conservative false-positive guidance, deep protocol packs and a candidate-only C/C++/Rust native source lane.
• Added Opengrep as deterministic candidate evidence.

Measurement and product surfaces

• Added the fail-closed Arena protocol for end-to-end applicability, verification, false-positive refutation, root-cause, regression-proof and release-gate measurement. The full workflow remains NOT_MEASURED; no competitor score is published in this release.
• Shipped Workbench V4 on sechelix.com for local recorded-run inspection without uploading artifacts.
• Preserved the existing uncontaminated 76-case blind-label result and its explicit boundary: precision 0.950, recall 1.000 and FP rate 0.053 describe the label task, not the complete V4 workflow.

Distribution

• Bumped the Agent Skill/plugin release to 4.0.0-alpha.1 while keeping the optional Python runner at its independent 0.1.0 version.
• Added release notes and automated release/SBOM publication safeguards. At the time 4.0.0-alpha.1 was cut, PyPI publication was still blocked on external publisher setup; runner 0.1.1 is now published through Trusted Publishing.

See docs/releases/4.0.0-alpha.1.md for the full notes.

[3.4.0-alpha.2] - 2026-09-02

Evaluation

• Published the first uncontaminated blind-label evaluation: precision 0.950, detection recall 1.000, false-positive rate 0.053, false-positive rejection rate 0.947, counts TP 38 / FP 2 / TN 36 / FN 0.
• Kept the two measurement layers separate everywhere: the blind label suite is MEASURED, the full SecHelix workflow remains NOT_MEASURED. verified_precision is 0.0 only because verification_status was NOT_RUN for every case.
• Corrected the blind packet's published digest, which had been computed on a CRLF working copy and therefore failed verification for anyone following the documented download.

Public record

• Synced the evidence boundary across README, ROADMAP, SUPPORT, evaluation and enterprise documents, the website, and the AI-readable llms.txt / llms-full.txt surfaces.
• Fixed two 404 links in llms.txt (SKILL.md, TROPHY_CASE.md).

Discovery

• Added an /appsec-agent product-category pillar and a second research piece on reviewing AI-generated code; declined a third page that would have duplicated the pillar.
• Recorded a search-intent baseline together with the Search Console state it was not derived from (zero impressions).
• Named Google-Extended explicitly in robots.txt with the same public-allow / admin-deny shape, covered by automated assertions. This governs Gemini training and grounding use, not Search ranking.
• Strengthened entity structured data with stable @id values. No rating schema was added.

[3.4.0-alpha.1] - 2026-09-02

Evidence platform

• Added versioned policy packs with fail-closed applicability and expiring accepted-risk records.
• Added blind verifier quorum outcomes for consensus, disagreement, and insufficient evidence.
• Added root-cause security campaigns and a controlled remediation loop that never applies a patch directly to the caller's primary working tree.
• Added LOCAL/STAGING runtime evidence correlation, dependency exploitability graphs, secret lifecycle tracking, MCP permission/data-flow graphs, and AI security inventory artifacts.
• Added an ablation-benchmark design for comparing the same evaluator with and without SecHelix. The scored public benchmark remains NOT_MEASURED until an uncontaminated evaluator runs the blind packet.

Integrity and safety

• Closed an integrity gap where an extra file injected after proof-bundle export could be absent from the manifest yet pass bundle verification.
• Kept runtime observations, dependency advisories, MCP declarations, and AI inventory declarations as evidence inputs rather than automatically promoting them to verified vulnerabilities.
• Preserved the open-core boundary: the local security engine remains open-source and unpaywalled.

Repository and distribution

• Enforced PR-only, squash-only public history with concise PR-title commits and CI checks against assistant trailers, session URLs, and diary-style commit bodies.
• Kept existing public history and immutable release references intact rather than rewriting provenance for cosmetic reasons.
• Aligned compatibility documentation around VERIFIED, DOCUMENTED, MODEL_COMPATIBLE, UNVERIFIED, and NOT_SHIPPED states.

Website and discovery

• Added automated XML sitemap validation, canonical discovery surfaces, RSS/Atom support, trust/early-access pages, and continued llms.txt / llms-full.txt support.
• Search and directory visibility remain measured separately from implementation; no ranking or recommendation claim is made by this release.

See docs/releases/3.4.0-alpha.1.md for the full notes.

V3.3 development milestone - 2026-09-01

• Added confidence calibration with withheld metrics until a sufficient uncontaminated sample exists.
• Added incremental evidence caching, proof bundles, authorization graph analysis, and PR security review.
• Added mutation/property tests that found and closed fail-open behavior in the release gate and revision freshness logic.
• Added a one-command blind-evaluation runbook; the public benchmark remained NOT_MEASURED.

[3.2.0-alpha.1] - 2026-09-01

• Fixed Agent Skills packaging so installs use the portable skill bundle rather than copying the whole development repository.
• Moved non-agent documentation out of the executable agent directory and split the Claude marketplace into its own repository.
• Added untrusted-repository mode, differential review, attack-chain correlation, revision binding, patch mode, and variant-rule generation.
• Expanded framework-aware and AI/MCP security guidance while preserving independent verification and fail-closed release semantics.
• Published the first authorized worked case study, including both a verified/fixed finding and a plausible candidate that was independently refuted.

See docs/releases/3.2.0-alpha.1.md.

[3.0.0-alpha.5] - 2026-09-01

• Reconciled the canonical report contract with renderers and release gates.
• Made missing required evidence fail closed as INCOMPLETE rather than silently passing.
• Reworked blind evaluation export/scoring and recorded the contaminated-evaluator blocker instead of publishing a contaminated score.
• Expanded the first Gold Check Packs, knowledge graph, and case-study/evidence program.

[3.0.0-alpha.4] - 2026-09-01

• Finished the V2 Pro private product surfaces for the verification console, Workbench, Attack Surface, Authorization Matrix, Variant Analysis, Benchmark Lab, and Command Center.
• Preserved the separation between severity and verification state and kept benchmark UI honest when no measured result exists.

[3.0.0-alpha.3] - 2026-09-01

• Added rights-aware source trust, knowledge graph, lesson-card, and live-research contracts.
• Added explicit human-only boundaries for restricted training curricula and provenance-backed security research states.

[3.0.0-alpha.2] - 2026-09-01

• Added the curated extension registry and extension safety/validation lifecycle.
• Prevented community extensions from self-promoting to official status without maintainer review and fixture evidence.

[3.0.0-alpha.1] - 2026-08-31

• Materialized the stable hypothesis catalog, versioned contracts, specialist roles, normalized evidence adapters, report formats, and fail-closed release-gate foundation.
• Established self-contained Agent Skills distribution and the public/private website-source boundary.

[2.2.0] - 2026-08-31

• Rebuilt the public README and repository contribution/security surfaces for open-source launch.
• Standardized Agent Skills installation and added evidence-only Trophy Case and benchmark methodology documentation.

[2.1.0] - 2026-08-31

• Introduced the portable Agent Skills architecture, composable security-family/verification-lens model, independent verifier methodology, company rollout documentation, and first public site/CI foundation.
• Renamed the early prototype to SecHelix and established evidence-based model/tool claims as a project rule.