Skip to content

glapsfun/sre-agent

v0.18.0

SRE investigation skills and local stdio tools

SRE Agent host setup and usage

The plugin supplies investigation instructions, generated investigator roles, local CLI/MCP tools, and Claude investigator hooks. The agent host follows the workflow and presents approval decisions; the Go executable collects evidence and maintains local records. It does not run an autonomous repair service. Start with the operator walkthrough, or consult the architecture for the implementation boundaries.

Investigation workflow

The skill starts at Phase 0: load the project memo and check its freshness. Phases 1 through 6 then scope the incident, discover the environment, collect evidence, analyze hypotheses and expected behavior, propose options for approval, and apply/validate the approved option. Failed validation returns to evidence collection. Each option names its target, preview steps, risk, impact, and rollback plan.

Seven playbooks cover Kubernetes, metrics, logs, changes, traces, EKS, and GKE. Symptom-based triage selects an initial wave and an escalation wave from the applicable investigators. Hosts with supported subagents can dispatch them; otherwise the skill executes their playbooks inline. State snapshots around waves compare selected fingerprints, not all possible mutations. Unreachable clusters or missing tools make that check inconclusive and must be recorded.

Installation

The host integration follows Claude Code plugin packaging and the Agent Skills format.

For Claude Code, run these commands in the host:

/plugin marketplace add glapsfun/sre-agent
/plugin install sre-agent@sre-agent

For noninteractive setup, the corresponding commands are claude plugin marketplace add glapsfun/sre-agent and claude plugin install sre-agent@sre-agent. A local marketplace installation uses the absolute checkout path containing .claude-plugin/marketplace.json. Update with /plugin marketplace update sre-agent.

The Claude plugin provides /sre-agent, investigator agents, MCP registration, and hooks. Skills-only installation copies the skill and leaves host registration to you:

npx skills add glapsfun/sre-agent --skill sre-agent --agent codex --global -y
npx skills add glapsfun/sre-agent --skill sre-agent --agent gemini-cli --global -y
npx skills add glapsfun/sre-agent --skill sre-agent --agent copilot --global -y

Choose the command for your host. Omit --global for a project installation. npx skills does not install plugin-level hooks, slash commands, MCP registration, or Claude agents. It can run the investigation playbooks inline.

For Codex subagents, locate the installed skill and run its bundled scripts/install-codex-agents.sh. The default target is ${CODEX_HOME:-~/.codex}/agents/; with --project, run it from the intended project because the target resolves against the current directory. Re-run it after updating the skill to keep the generated roles synchronized. Configure the parent-session MCP filter below before dispatching investigators.

Claude and Codex local fixtures were exercised. The Codex portable MCP fixture used version 0.167.1 and a locally built binary. Gemini was unavailable for runtime checks; Copilot CLI and VS Code packaging were not runtime-tested. Their configuration recipes are setup guidance, not a claim of verified host behavior. JSON validation alone does not prove host startup or permission filtering. Bash fallbacks still need their tools, network access, and a read-only identity.

MCP tools

The source-built sre mcp command serves stdio only. It exposes exactly discover, logs_fold, prom_query, timeline, recall, run_resume, ledger_add, and report. It has no HTTP listener, resources, or authentication layer. stdout carries only MCP messages; diagnostics go to stderr. The launcher pins v0.18.0. If that release or its mcp command is unavailable, use a verified source build or the CLI and script fallbacks below.

Prefer available typed read tools for covered operations. Supply the exact nonempty pinned cluster context to discover, logs_fold, prom_query, and timeline, including inline log folding, an explicit Prometheus URL, and automatic endpoint discovery. Preserve the random external-data boundaries in returned text. run_resume reads an explicit run without binding a session. report returns content and has no output-file argument. Use the guarded Bash fallback for uncovered operations or unavailable registration/server, and record the limitation.

The recommended Claude read permissions are individually named:

mcp__plugin_sre-agent_sre__discover
mcp__plugin_sre-agent_sre__logs_fold
mcp__plugin_sre-agent_sre__prom_query
mcp__plugin_sre-agent_sre__timeline
mcp__plugin_sre-agent_sre__recall
mcp__plugin_sre-agent_sre__run_resume
mcp__plugin_sre-agent_sre__report

Claude investigators receive only the subset relevant to their role: discovery for all seven; logs and timeline for Kubernetes/EKS/GKE; logs for the logs investigator; Prometheus queries for metrics; timeline for changes. Trace queries retain their guarded Bash recipes. Recall, resume, and report are available to the orchestrator. ledger_add is the sole write tool and requires a separate orchestrator permission decision. Never grant it to investigators. A server-wide wildcard such as mcp__plugin_sre-agent_sre__* includes ledger writes and is not the recommended read-only setup. readOnlyHint annotations describe tools; they do not enforce permissions or replace the guard. Semantic reads such as resume and recall can create or migrate the repository-local database when it opens. They do not append ledger entries. The report tool instead requires an existing compatible database and opens it read-only.

The final encoded MCP result is limited to 8000 bytes, including SDK-added fields. Oversized reads return result_too_large with a CLI route instead of silently truncating the result. A committed ledger write can return a compact receipt with stable run/entry IDs and a gap; retry an uncertain append only with the same entry ID and payload.

Claude loads plugin-root .mcp.json, whose command is ${CLAUDE_PLUGIN_ROOT}/skills/sre-agent/scripts/sre with the separate argument mcp. Codex uses the portable plugin.json and mcp.json: bash receives ${PLUGIN_ROOT}/skills/sre-agent/scripts/sre and mcp as separate arguments. That portable layout started and called discover in an installed Codex 0.167.1 fixture. The compatibility layout was recognized too, but needed an absolute launcher path because its root placeholders stayed literal in that runtime. These fixtures used a locally built binary, not a published release.

Gemini's gemini-extension.json declares server sre with ${extensionPath}/skills/sre-agent/scripts/sre and the separate mcp argument, following the extension reference. Gemini runtime expansion has not been tested locally because Gemini is absent. Keep the placeholders literal in manifest files; the named host must expand them. Parsing JSON alone does not verify startup or tool execution.

Required Codex investigator permissions

Before starting a Codex investigation, install the seven-read allowlist in its parent session. Custom agent TOML files inherit parent MCP servers; their instructions cannot override that policy. Keep ledger writes in a separate orchestrator-only session that never spawns investigators. Do not add ledger_add to an investigation session or resume investigators from a write-enabled parent.

For the bundled portable plugin, add this policy to your Codex config.toml. Replace sre-agent@sre-agent with the exact installed plugin ID, including its marketplace suffix, shown by /plugins:

[plugins."sre-agent@sre-agent".mcp_servers.sre]
enabled_tools = ["discover", "logs_fold", "prom_query", "timeline", "recall", "run_resume", "report"]

For manual registration, locate the installed skill's executable scripts/sre or a verified source-built sre, then use this complete configuration:

[mcp_servers.sre]
command = "/absolute/path/to/skills/sre-agent/scripts/sre"
args = ["mcp"]
enabled_tools = ["discover", "logs_fold", "prom_query", "timeline", "recall", "run_resume", "report"]

codex mcp add sre -- /absolute/path/to/skills/sre-agent/scripts/sre mcp registers the transport only. Add the allowlist before using investigators. Choose one registration path; disable duplicate SRE registrations that could expose unfiltered tools. These policies use the supported Codex MCP tool allowlist; subagents inherit parent MCP configuration.

Start a fresh investigation session after setup. Before dispatching any investigator, inspect /mcp and verify the effective SRE tool list contains exactly the seven reads above and excludes ledger_add. Call discover with the pinned context to check startup. If the effective filter cannot be verified, disable that MCP registration for the investigation and use the guarded CLI fallback, recording the limitation. A role's prose restriction or a tool's read-only annotation is not proof of enforcement.

In the separate orchestrator-only session, grant ledger_add only after the write-permission decision. Keep investigators in the filtered session and return their evidence to the orchestrator for recording. Retain the CLI ledger workflow when MCP write permission is unavailable.

For Copilot CLI, use /mcp add, name the server sre, and enter the absolute launcher path followed by mcp in Command. Select only the seven read tools above (using the host's displayed tool names), with ledger writes decided separately. For VS Code, open Chat: Open Customizations, select MCP Servers, and add a stdio server named sre with that executable and argument. Copilot's native configuration uses mcpServers; older editor-managed .vscode/mcp.json uses servers. Neither client was runtime-tested for this packaging change.

npx skills installs the skill without registering these plugin manifests. Use manual registration or keep the CLI and bundled scripts. When MCP is unavailable, locate the launcher and use sre discover --context <ctx>, sre logs fold --context <ctx> --namespace <ns> --pod <pod>, sre prom query '<PromQL>' --context <ctx>, or sre timeline --context <ctx> --namespace <ns> with the operation's remaining arguments. CLI stdin folding uses sre logs fold < app.log without cluster flags. Local reads use sre recall, sre run resume <run-id>, and sre report <run-id>. If the launcher or command is unavailable too, retain the existing discovery/evidence scripts and report the gap. Investigator Bash fallback remains subject to the read-only guard where installed and the read-only identity on every host; a denied call is not a reason to bypass it.

CLI launcher

Run skills/sre-agent/scripts/sre version from this plugin directory. The launcher downloads the pinned v0.18.0 release on first use for Linux or macOS, on amd64 or arm64. It requires Bash, curl, and a SHA-256 tool: OpenSSL, sha256sum, or shasum. It prefers OpenSSL when available and verifies all binary bytes on every launch. The binary prints one JSON object with ok, data, gaps, and errors. Use --format text for a readable text view. The CLI provides version, discover, logs fold, prom query, timeline, run, ledger, incident add, recall, report, and process-log log; the existing investigation scripts remain available. If the installed binary lacks a command, use a source build or the existing scripts for that operation.

sre logs fold reads stdin and returns redacted patterns, counts, first/last timestamps, levels, and samples. For example:

sre logs fold --max-patterns 20 < app.log
sre logs fold --context prod --namespace payments --pod checkout-abc \
  --container app --previous --since 1h --format text

Errors sort first, then warnings, then other levels; rarer patterns sort first within each group. The default limit is 20 patterns. omitted_patterns and omitted_lines account for everything outside the limit. Samples and templates carry fresh external-data markers. Pod collection streams a finite snapshot, enables timestamps, and uses a 30-second deadline. It selects explicit context, then SRE_KUBE_CONTEXT, then the current context without changing it. See log folding details for masking, timestamp handling, and limits.

sre timeline returns every matching retained Kubernetes event, ReplicaSet revision, Flux/Argo revision, local git commit and currently firing Alertmanager alert in one sorted list. Namespace is required; workload accepts an exact name or KIND/NAME. For example:

sre timeline --context prod --namespace payments --workload deployment/checkout \
  --since 2h --repo ./checkout --alertmanager-url http://localhost:9093 \
  --timeout 30s --format json

The default window is the last two hours, captured once in UTC. Context follows --context, SRE_KUBE_CONTEXT, then one current-context lookup. Setup and all collectors share the timeout. Alertmanager uses --alertmanager-url, then SRE_ALERTMANAGER_URL; supply an already reachable HTTP(S) endpoint. The command does not start a port-forward or use discovery hints as endpoints. Missing repo or endpoint produces a gap. A failed source sets ok=false and exits 1 while retaining independent successes; absence and historical-coverage gaps alone exit 0. An unresolved requested workload never falls back to namespace scope.

JSON keeps the complete typed list at data.entries, plus all seven source summaries, undated revision context and structured gaps. One fresh external-data boundary marks the report; text presents the same records and coverage. There is no entry limit or pagination. Events and ReplicaSets cover retained objects, not deleted history; aggregate event counts do not date each occurrence. ReplicaSet creation does not prove rollout or revision application time. Safe pod-template comparisons omit literal environment values, annotations, commands and other excluded fields while preserving references such as Secret names and keys. Git covers local HEAD ancestry by committer time, not deployment time. GitOps ownership must be verifiable; ambiguous remote associations and unknown timestamps remain gaps or undated context. Alertmanager covers current firing alerts, including suppressed alerts, not resolved alert history. Cloud audits, GitHub activity, Helm history and mesh evidence still use the existing helpers.

The cache lives under CLAUDE_PLUGIN_DATA/sre-agent when set, otherwise XDG_CACHE_HOME/sre-agent, otherwise HOME/.cache/sre-agent. Each version and platform has its own directory. Downloads use HTTPS, bounded timeouts, and a release SHA256SUMS manifest. The launcher verifies the checksum before publishing a cache entry and before every execution. Checksums detect changed downloads and cached binaries; they trust the release manifest and local cache.

See release operations for version alignment, publication, and recovery. The launcher does not silently execute fallback scripts.

A valid cached release works offline. With no cache, a failed download exits nonzero and points to the existing scripts, such as sre-env-discovery.sh and sre-evidence.sh, including when the pinned release is unavailable. A checksum mismatch refuses execution; remove the affected version cache and retry online. A concurrent download exits with a retry message. After an interrupted download, remove its stale .lock directory only after confirming no launcher is running.

Release tags build and run the version command on all four native platforms before publishing binaries and SHA256SUMS. Updating the launcher pin is a separate source change when adopting a later release.

Local database

The CLI storage package creates .sre-agent/sre.db at the repository root on first use. Linked Git worktrees get their own database. Keep .sre-agent/ out of version control in repositories where you use the agent. The directory and database are private to the current user.

The store uses a pure Go SQLite driver, WAL, and a five-second busy timeout for concurrent local processes. Use a local filesystem; SQLite WAL does not support sharing a database between machines over a network filesystem. Schema upgrades run in a transaction when the database opens. A database from a newer binary is refused. After all users close it, a deleted database is recreated empty on next use.

The investigation ledger stores append-only typed updates and redacts secret values before each committed write. Existing incident Markdown and project memos stay in place.

sre run start "checkout errors" --id checkout-run --repo ./checkout
sre ledger add evidence --run checkout-run --id evidence-1 --file evidence.json --repo ./checkout
sre run resume checkout-run --repo ./checkout
sre run show checkout-run --offset 1 --repo ./checkout --format text

For example, evidence.json contains one concise observation:

{"fact":"checkout returned HTTP 503","source":"provided incident observation"}

The accepted kinds are metadata, evidence, hypothesis, expected-behavior, option, approval, action, validation, and safety-violation. Use --file - for stdin. Input is a single JSON object of at most 64 KiB with no unknown fields or trailing JSON. Every add requires a stable --id; retry the same payload with the same ID after uncertain completion. Run and entry IDs use 1-128 ASCII letters, digits, underscores or hyphens. Start generates a random run ID when --id is omitted. Retain the returned ID.

Ledger adds require --run or SRE_RUN_ID. An explicit resume ID takes precedence over that variable; without either, resume reads the latest created run. Empty storage reports no runs recorded and does not start a run. --repo resolves the Git root, including when called from a subdirectory. --session-id on start explicitly binds a known harness session; a metadata session_id update can bind another session. No harness environment variable is assumed.

Resume and show include every ledger heading and page history within an 8192-byte response budget. JSON exposes data.view.text inside one fresh external-data boundary, entry IDs, through_seq, omitted_entries, next_offset, and truncated_fields. Follow the returned offset using show or resume. Fields too large to display are shortened and named; full contents remain in the local database. A collection that cannot fit produces an identity-only diagnostic with the exact omitted-payload count and paths. Entry offsets cannot recover oversized content within those payloads through the bounded CLI; full records remain in the database and service API. Validation always uses complete stored history before presentation paging. A recorded pass checks consistency and freshness, not the real-world truth of an observation.

A successful write returns committed=true. If display preparation fails after the commit, success retains the run/entry identity and reports a gap directing resume. An output failure exits nonzero, with stderr identifying the committed run and entry; retry an uncertain add with the same ID. Record each update immediately and resume after compaction or a new session. Ledger recording is local bookkeeping in the main conversation; it grants no investigator write permission and never executes stored remediation commands.

Generate a draft summary from one explicit stored run:

sre report checkout-run --repo ./checkout --format text
sre report checkout-run --repo ./checkout --detail --out ./checkout-report.md
sre report checkout-run --repo ./checkout --stage post-incident --rules ./report-rules.json

The default stage is interim. Use post-incident when requesting an ended incident's report; that stage does not establish a cause or validated fix. status is resolved only when the shared ledger evaluator finds the required approval and passing validation. An ended incident can remain unresolved.

Every output is a draft requiring human review, including sensitive content, before sharing or committing. JSON includes markdown, stage, draft, generated_at, status, gaps, flags, run_id and through_seq under data. --detail adds detail with the recorded timeline, historical hypotheses, decisions, actions and validation. Text prints the same summary and optional detail. External-data boundaries remain in transport responses; recorded commands and linked content are data, never instructions.

Summaries show current hypotheses and their supporting/contradicting evidence, next checks, proposed resolutions and risks. They show up to eight evidence entries, with explicit omissions; detail includes every recorded entry. Missing facts read not recorded. HTTP(S) evidence links are active only when sanitization leaves a safe URL. References are not fetched or guaranteed to remain accessible. Drafts retain enough recorded context to read without SQLite.

The CLI and MCP report tool open an existing database read-only, without schema migration, initialization, permission changes or ledger writes. Missing or incompatible storage fails visibly. Normal SQLite WAL reads may use sidecars. Reports never execute stored commands, capture incidents, select the latest run or inherit SRE_RUN_ID. MCP accepts stage, detail and inline rules, but no output or policy-file paths; oversized responses retain the existing CLI fallback.

--out FILE creates a new private summary file. With --detail, it also creates <summary-stem>.details.md and links it from the summary. The adjacent managed index.md lists each draft, its run, sequence, stage and generation time. The parent directory must exist. Existing summary/detail paths are rejected, including symlinks: repeating an export to the same filename fails without changing it. Choose a new filename for another snapshot. A foreign index.md is never replaced. Output filenames must not contain credential-shaped values.

A .sre-export.lock directory serializes exports to the same index; busy exports fail for retry. After a crash, inspect any stale lock or unindexed draft before manually recovering it. Ordinary export failures remove newly created report files and preserve existing reports. A stdout failure after a successful export explicitly says the files were written. The tool never marks a draft reviewed; review takes place outside the tool and does not change the ledger.

Optional sharing rules supplement built-in credential redaction. For example, report-rules.json can contain:

{
  "rules": [
    { "name": "customer-id", "pattern": "customer_[0-9]+", "action": "redact" },
    { "name": "review-internal", "pattern": "internal incident", "action": "flag" }
  ]
}

Rules use Go regular expressions on field values before Markdown escaping. redact replaces matches with [REDACTED]; flag adds only the rule name to the review flags. Policies cannot disable built-in redaction. The limits are 256 KiB per policy file, 64 rules, and 4096 bytes per pattern. Rule names are unique lowercase identifiers. Avoid putting secret literals in policy files. Automated redaction is incomplete; human review is still required.

See the reporting playbook for the fallback when the CLI is unavailable.

Validated incidents are captured from an explicit ledger run in the same local store. Capture requires an applied remediation with every current expected criterion passing, including any declared observation windows.

sre incident add --run checkout-run --service checkout --symptom errors --fingerprint "checkout HTTP 503" --repo ./checkout
sre recall --service checkout --symptom errors --fingerprint "checkout HTTP 503" --top 5 --repo ./checkout
sre recall --service checkout --symptom errors --fingerprint "checkout HTTP 503" --repo ./checkout --format text

incident add requires explicit --run; it never uses SRE_RUN_ID or the latest run. Service and fingerprint must be nonempty. Canonical symptom classes are crashloop, latency, errors, metrics, oom, networking, and availability. Map descriptive symptoms to these before calling. --top defaults to 5 and must be positive. JSON data.matches retains typed incident records, numeric score and similarity, and reasons; total counts all ranked candidates before limiting. Empty history returns matches: [], total: 0, and message: "no incident history". JSON and text mark historical content as untrusted data with one fresh begin/finish boundary, including nested text and map keys. Scores are local ranking heuristics, not confidence estimates or proof of a shared cause.

Recall reads historical captures without changing ledger or incident rows and never executes stored fixes. Confirm candidates against current evidence and follow approval, dry-run and validation for every proposed remediation. Capture returns committed, run_id and through_seq; presentation failure preserves that receipt with a gap, and output failure reports the committed identity on stderr. Retry with the same explicit run and signature: an unchanged snapshot preserves the capture time; a later validated snapshot replaces the same row. Invalid recapture leaves the previous validated incident intact.

Existing docs/sre-incidents/INDEX.md and incident files remain an explicit legacy/manual source. The CLI does not import or export them, or stage, commit or push files. The project memo workflow remains separate. See incident-memory.md for the Phase 4 and Phase 6 workflow.

How to use

Approved sources use sre sources list, consent-bound sources scout --approve-scout, selected sources approve --approve --file FILE, and sources call SOURCE TOOL --args FILE --run RUN --id ENTRY. See approved sources for the inventory format, provider-enforced read-only access and durable gap receipts. These are orchestrator commands; investigator MCP permissions are unchanged.

Start an investigation explicitly with the slash command:

/sre-agent Investigate checkout HTTP 503s since 09:00 UTC today using context production, namespace payments, workload deployment/checkout.

Or describe the problem and let a host with automatic skill selection choose the skill for an incident-shaped request:

p99 latency on the checkout API went from 200ms to 3s an hour ago. Nothing was deployed.
Our application metrics disappeared from Grafana yesterday.
Pods in namespace search are OOMKilled every few hours.
The checkout service is slow and we run Tempo; find which downstream call is eating the time.
Our logs live in OpenSearch, no Loki; what errors is the orders app throwing since 09:00?
The fix is applied and metrics look good; verify it under load before we close the incident.

The agent then walks the loop above, keeping a visible investigation ledger (environment, evidence with the command that produced each fact, hypotheses with confidence, actions, validation results) updated at every phase. When it reaches the approval gate it presents the options and stops - pick one, ask for more evidence, or reject them all.

After resolution you get a final incident report (summary, impact, timeline, root cause, actions, validation, rollback info, follow-ups) that you can save to your repo.

What's inside

ComponentPurpose
skills/sre-agent/SKILL.mdThe orchestrator: loop, phase gates, safety rules, investigation ledger
skills/sre-agent/references/investigators/The seven investigator playbooks (k8s, metrics, logs, changes, traces, eks, gke); single source of truth; executed inline when subagents are unavailable
agents/; sre-k8s-investigator, sre-metrics-analyst, sre-logs-investigator, sre-change-historian, sre-trace-analyst, sre-eks-investigator, sre-gke-investigatorRead-only Claude Code subagents dispatched in parallel for evidence collection; generated from references/investigators/ by scripts/gen-sre-agent-artifacts.sh
skills/sre-agent/agents/codex/The same seven subagents in Codex TOML format (generated); installed by install-codex-agents.sh
skills/sre-agent/references/Deep knowledge loaded on demand: discovery, PromQL (golden signals, kube-state, burn rates), LogQL and error taxonomy, Elasticsearch/OpenSearch query DSL, TraceQL/Jaeger tracing, deep Kubernetes evidence (nodes, NetworkPolicy, DNS, storage), service mesh (Istio/Linkerd), Grafana API discovery, root-cause analysis method, remediation templates, Terraform remediation (locating IaC-managed resources, safe plan reading, plan-classified apply/rollback), Pulumi remediation (locating IaC-managed resources, safe preview reading, preview-classified apply/rollback), Crossplane remediation (ownership detection, Ready/Synced condition and trace interpretation, status-check-verified apply/rollback), validation checklist and report format, k6 load validation, GitHub investigation via gh (repo discovery, change timeline, prior-incident and code search), sibling-skill deference (argocd/fluxcd/helm/bash-scripting/kubernetes-operator/karpenter/gh-guru/glab-guru), pinned official sources
skills/sre-agent/scripts/Evidence, preview, and setup helpers: sre-env-discovery.sh (tools, cluster, GitOps, cloud), sre-obs-discovery.sh (Prometheus/Alertmanager/Grafana/Loki/Mimir/Tempo/Jaeger/Elasticsearch endpoints, service-mesh and k6 detection), sre-evidence.sh <ns> <workload> (one-shot evidence pack), sre-snapshot.sh (Phase 3 mutation verification: snapshot/diff spec-hash fingerprints around each wave's dispatch), sre-gh-discovery.sh (read-only GitHub discovery/search via gh: repo/timeline/incidents/code subcommands), terraform-plan-check.sh <tf-dir> (Phase 5/6 Terraform plan classification: no-op/create/update/delete/replace counts, flags any destroy/replace), pulumi-preview-check.sh <stack-dir> (Phase 5/6 Pulumi preview classification: same/create/update/delete/replace counts, flags any destroy/replace), crossplane-status-check.sh [TYPE[.VERSION][.GROUP][/NAME]] (Phase 2-6 Crossplane evidence and verification: classifies Provider/Managed/Composite Resource Installed/Healthy/Ready/Synced conditions), install-codex-agents.sh (Codex subagent install)
commands/sre-agent.mdThe /sre-agent <problem> entry point (Claude Code)
hooks/The read-only guard for the seven investigator subagents (Claude Code)

Safety model

  • Read-only until approval: the skill instructs investigators to collect evidence only. The main conversation may apply changes only after you select an option. These instructions depend on host permissions and credentials.
  • Dry-run first: kubectl diff/--dry-run=server, helm --dry-run, flux diff, terraform plan, pulumi preview before every apply.
  • Secrets: metadata only: names, ages, revisions; never values.
  • GitOps- and IaC-aware: fixes to Flux/Argo-managed workloads, and to Terraform/Pulumi/CDK/eksctl-managed EKS infrastructure (node groups, IRSA roles, ALB config), are directed to the source repository/IaC, not hand-edited directly.
  • Gentlest effective action: restart over delete, scale over replace; blast radius stated before anything destructive.
  • Load generation is mutation-class: the optional k6 validation step has its own approval gate, states target environment/RPS/duration/blast radius up front, and never targets production unless you explicitly say so.

Read-only enforcement

In a Claude Code plugin install, a hook (hooks/readonly-guard.sh) checks matching tool calls from the seven configured investigators and allows only known read operations. Anything it doesn't recognize is blocked and recorded as a gap. It also blocks reads of common credential files.

It stops mistakes and simple prompt injection, but it's not a sandbox. It can't see inside a script it allows, a GET request can still carry data out, and it doesn't cover the main conversation, which is where approved fixes run. The packaged Go checker runs through the checksum-verified launcher. If it cannot start, investigator calls fail closed; the main conversation and other agents remain outside the guard.

The pre-hook records each recognized investigator call in the repository's .sre-agent/sre.db. Post success/failure hooks reconcile the same (session_id, tool_use_id) row. guard=allowed only means the guard allowed the request; outcome=unknown does not establish execution or success. Logging errors produce a nonblocking diagnostic and never change the guard's decision.

Bind the harness session when starting a run, or set SRE_RUN_ID to an existing run before launching the harness. An explicit run ID takes precedence over session binding; hooks never select the latest run.

sre run start 'Investigate API latency' --id incident-1 --session-id SESSION_ID
sre log incident-1 --format text
sre log incident-1 --slowest 10 --by-investigator
sre log incident-1 --blocked --repo /path/to/repository

log defaults to JSON. Filters combine; investigator counts describe the selected rows after limiting. Measured durations sort before unknown durations, with stable observation/session/call ordering for ties. The timestamp records the first observation, not a tool start time. Absent duration and numeric exit status remain null. The hook adapter does not infer process exit status from tool output or failure text.

Metadata contains sanitized command/path text, never complete inputs, responses, or failure messages. Credential patterns, sensitive shell assignments, and user credential flags, including curl clusters, are redacted before writing. Ambiguous credential arguments cause the entire command text to be stored as [REDACTED]; simple values retain sanitized command context. Identity fields that require redaction are rejected to avoid identity collisions. Do not place arbitrary secret values in commands or identifiers. The first observation pins the validated run and call metadata. Contradictory events or metadata produce an absorbing conflict outcome and clear numeric metrics; a run-binding conflict leaves the original row unchanged and emits a diagnostic. Repeated delivery cannot create another row for the same call.

The matchers cover every declared investigator tool: Bash, Read, Grep, Glob and WebFetch, plus mcp__plugin_sre-agent_sre__.*. Glob records only its optional sanitized path; WebFetch records its sanitized URL in the existing path field. Glob patterns, WebFetch prompts and responses are omitted. Only the seven planned MCP read operations are allowed: discover, logs_fold, prom_query, timeline, recall, run_resume, and report. ledger_add and unknown operations in this namespace are blocked for investigators.

Calls with no delivered hook cannot be logged. Permission denial can omit a post event, and schema rejection can omit both events. Legacy pre events with all of session_id, tool_use_id, and cwd absent remain guard-only; partially supplied or invalid metadata emits a logging diagnostic. Live MCP and harness acceptance remain separate from the local hook-shaped integration tests.

The hard limit is a read-only identity. Have an administrator provision Kubernetes RBAC and AWS/GCP read permissions appropriate to the evidence you need, then investigate using that identity. Unreadable resources become gaps. Each remediation option names its target context and needs separate approval and credentials that permit the selected change.

The other documented install paths do not register this Claude investigator hook. Codex documents that custom agent instructions don't create a read-only sandbox and that subagents inherit the session's sandbox policy, and npx skills installs only the skill. On those paths, the read-only identity is the only hard limit.

Degraded environments

Missing tooling is the normal case, not an error. The playbooks describe fallbacks for common missing capabilities: no Prometheus → kubectl top + events; no Loki → Elasticsearch/OpenSearch, else kubectl logs; no trace backend → latency RCA proceeds metrics/logs-only; mesh present but no istioctl/linkerd CLI → kubectl-only mesh evidence; no k6 → passive validation plus a script you can run yourself; no cluster access at all → static analysis of repo manifests with the limitation stated. Every gap is recorded in the ledger; the skill requires live evidence before claiming a fix is verified. Permission denials remain gaps and must never be bypassed through a fallback.

Works best with

Install alongside the cloud-native plugins from the cnative-skills marketplace; the orchestrator defers to them for deep work when they are available, and falls back to its own playbooks when they are not: kubernetes-operator, helm, fluxcd, argocd, karpenter.