amiable-dev/stdtel
Telemetry for standards-as-skills: attributes token cost and policy outcomes to individual skills across Claude Code and GitHub Copilot.
Changelog
Versions are shared by the Python package and the plugin manifests, and a test asserts they agree. A version bump is what makes clients pick up a new copy — both marketplaces serve the cached version until this number changes — so bump it for anything a user would receive.
Unreleased
Added
stdtel-uilinks into Grafana (ADR-017 W2, #163).- Findings with an aggregate behind them link to the provisioned dashboard by uid.
- A session that
native coveragenames opens the operational board filtered to that session, from its start. The board has a newsessionvariable (a regex, default.*), and every Loki panel filters on it. - Links are tested against the provisioned uids and variables. A live test also checks that every
Loki panel parses with a linked session substituted. The first version used
re.escape, whose\-LogQL rejects. - A check added to doctor appears on the page with no page code.
stdtel-ui, a local health page beside Grafana (ADR-017 W1, #162). It serves doctor's own checks on 127.0.0.1, by default on berth'swebport (11000here), and polls them every 30 s. It never picks another port.- Security, following berth's rules:
- GET and HEAD only, with a per-response CSP (
frame-ancestors 'none'); - the Host header must be
127.0.0.1:<port>orlocalhost:<port>, or the request gets 421, which defends against DNS rebinding; - its one JSON endpoint refuses cross-site requests.
- GET and HEAD only, with a per-response CSP (
- Checks: each runs under its own timeout, is cached for 15 s or 60 s, and concurrent requests
share one run. A slow or broken check shows as
unknownwhile the rest render. - What it renders: text only. Home paths show as
~. A setting's value renders as its name, for example$STDTEL_OTLP_ENDPOINT. - The page follows berth's design pack.
- Security, following berth's rules:
Changed
- A tested rollback for the metrics path (ADR-016 W5, #161).
STDTEL_METRICS_OVERLAY=overlay-remote-write.yamlmoves the collector's metrics back to remote write. It sits in its own--configslot, so it combines with Langfuse. Prometheus keeps its remote-write receiver on, so this one variable is the whole switch.docs/local-stack.mdhas the new section "Rolling back the metrics path". It covers the last-passing-digest rule, the collector floor and what the fallback costs.tests/test_rollback.pyruns the fallback on the shipped images.
- The fallback was measured, and a correction came out of it.
- Collector 0.124+ starts each new
calls_totalseries at 0 itself (contrib #38537). Count panels therefore count a first occurrence on either path. Only the duration panels undercount on the fallback, and each rate panel's description says which of the two it is. - The same measurement shows the Prometheus
created-timestamp-zero-ingestionflag is load-bearing, for the duration histograms. #158 had called it unneeded, because its tests read onlycalls_total. The new testtest_a_first_duration_is_countedfails without the flag. - ADR-016's amendment now cites contrib 0.124 (#35994, #38537), not 0.125 (#38990).
- Collector 0.124+ starts each new
- Empty dashboard panels say what is empty, and nothing is coloured as an alarm by default
(ADR-016 decision 4, #160). This applies to both dashboards.
- "Error rate by skill" reads 0 for a skill with no errors. It used to show nothing, because the error series does not exist until an error happens. Now the numerator falls back to zero.
- The spend and request stats no longer turn red above 80. They had no thresholds, so Grafana's default applied.
- Every timeseries legend names its series, such as the skill or the kind, not the expression.
- Every panel's empty state says what is missing and, where it helps, what to check.
tests/test_empty_states.pyenforces each rule on every panel. The error-rate case runs on the shipped images.
- "Skills invoked" counts skills typed as
/name(ADR-016 decision 3, #159). It now reads Claude Code'sskill_activatedevents, which fire once per activation whether the skill was typed (user-slash), chosen by Claude (claude-proactive) or loaded by another skill (nested-skill).- The panel used to count Skill tool calls, and a typed skill makes none. Over six days, 30 of 46 activations were typed: the panel showed 16.
- It now breaks down by how the skill was invoked. Names no longer need the detailed view.
- New "Spend by scope" panel, from the warehouse view
llm_request_scope(ADR-016 decision 3).- Each Claude Code request is joined to its stdtel turn by prompt id. Requests whose prompt joins no turn get their own row, never folded into "(no scope)".
- Each row shows its share of the spend and
data_through, the warehouse's age. - A prompt whose turns name two scopes is not guessed.
- A skill that ran once now counts on the dashboards (ADR-016 decision 1, #158). Before this, a
series' first occurrence was invisible to
rate()andincrease(), so most skills read as never run.- Prometheus is now 3.15.0 and receives metrics on its own OTLP endpoint with
created-timestamp-zero-ingestion, which writes a zero at each series' start. - The collector is now contrib 0.126.0. That is the first published image whose spanmetrics connector keeps a start time per series. 0.118 kept one per resource, so a series born later got a zero hours before its first sample, outside any recent window. Prometheus 3.15 alone was measured to still read 0.
- Both images are pinned by digest.
prometheusremotewriteis gone. tests/test_counting_first_occurrences.pyruns the shipped images and configs through the W1 matrix: a first occurrence, three increments, a collector restart, the start, middle and end of the window, a retried delayed batch, and every dashboard expression.- Restart the stack to apply it:
make down && make up. Data already in Prometheus keeps its undercount.
- Prometheus is now 3.15.0 and receives metrics on its own OTLP endpoint with
std.scope.keyis no longer a metrics label, and metric series expire after a day (ADR-016 decision 2, #157). The key is a branch hash: bounded per branch, unbounded over time. Per-key figures read the warehouse;std.scope.namestays. Spanmetrics setsmetrics_expiration: 24h. The operational board gains a "Prometheus active series" panel with a 10,000 line, and Prometheus now scrapes itself so that panel has data.- The skills and the analyst check the warehouse is current before quoting it (ADR-015 decision 6,
#155).
stdtel-query(1.3.0) and the scorecard analyst (1.2.0) runstdtel-doctor --jsonfirst, and state a non-passwarehouse freshness, with how stale, before the first number.stdtel-setup(1.2.0) offersstdtel-install loaderand ends withstdtel-doctor --json, reading back every check that is notpass. - The status line warns a session that will not export (ADR-015 decision 4, #154). SessionStart
records whether
CLAUDE_CODE_ENABLE_TELEMETRYis on in the session's environment and what the settings resolve it to (enabled,opted-outorunknown). Settings on with the variable absent showsusage not exported: started before telemetry was on; restart to fix, and a restart clears it. Opted out or unreadable settings show nothing. New doctor checkclaude settingsnames a settings file that is not valid JSON. stdtel-doctorchecks have four outcomes:pass,fail,pendingandunknown(ADR-015 decision 1, #150). Onlypassis a pass. New--jsonprints one object per check (name,outcome,observed,remedy,observed_at) and nothing else. The exit code is0when every check passed,1when any failed, and2when none failed but some are pending or unknown. That is a change for anyone gating on1: a check that used to fail can now exit2.artefact eventsnot yet seen ispending, not a failure, so a healthy machine no longer fails until it happens to spawn a sub-agent.hooks runningwith state no Claude Code transcript owns isunknown; it used to pass.- An unreadable state directory, a hook probe that could not run, a content probe that never
arrived or could not be sent, and a check that raised are each
unknown. The detailed view still refuses to turn on unless the content probe passes.
stdtel-install loaderkeeps the warehouse current (ADR-015 decision 5, #171). It schedulesstdtel-loadas a launchd agent on macOS or a systemd user timer on Linux. The job definition carries your STDTEL_* settings and a PATH, is mode 0600, and is never printed. It refuses without the warehouse extras, and refuses (naming the setting, never its value) a setting a systemd unit cannot carry.loader livenessnow points at it.stdtel-install detailed-view offwith no settings file now says it is already off instead of creating one.stdtel-loadruns every loader once, under a lock (ADR-015 decision 5, #170). One command for a scheduler, or for you. Each loader's window reaches back to its last successful watermark (24 h at least, 720 h at most), so an outage longer than a day is repaired. Only one run at a time, by an OS lock the kernel releases when its holder exits: an overlapping run exits without writing. The log is kept to 1 MB. Without the warehouse extras it refuses and says how to install them. Thewarehouseloaders now ship in the wheel; install withuv tool install 'stdtel[warehouse]'.load_tracescatches up after a week away (ADR-015 decision 5, #153). A window wider than Tempo's search maximum (168 h here, read from/status/config) used to fail with HTTP 400. It is now split into chunks, each written as it loads. A failing chunk leaves the others loaded and records the run asok = false, naming the chunk. A search that fills Tempo's 1,000-trace limit is split in half until it doesn't, and fails the chunk rather than lose traces. Every Tempo call now has a timeout and checks its HTTP status, and--sincemust be whole hours, such as24h.session_costkeeps the newest snapshot, not the first (#172).std.session.costis sent at every Stop with the transcript's running total, and the loader kept whichever snapshot it wrote first, which is usually the oldest and smallest. A newobserved_atcolumn (the Stop that sent the snapshot) now decides, whatever order the snapshots load in. Re-applywarehouse/schema.sqlto an existing warehouse (docs/local-stack.md, "Resetting"); it only adds the column. A load whoseloader_runrow cannot be written now exits non-zero.stdtel-doctorsays how far the warehouse is behind, and whether the loader runs (warehouse freshnessandloader liveness, ADR-015 decision 3, #152). Freshness compares each loader's watermark (the newest source timestamp a successful run loaded) with whether Tempo or Loki now holds anything newer, beyond twiceSTDTEL_LOAD_INTERVAL(default 900 s). Liveness reads the latest attempted run of each loader, names a failed run's error, and needs only Postgres. Both readSTDTEL_DSNand needstdtel[warehouse]. On 2026-10-09 this is what said the warehouse was a week behind.stdtel-doctornames the sessions Claude Code is not exporting for (native coverage, ADR-015 decision 2, #151). A session reads its telemetry settings once, at start, so one begun before native telemetry was on never exports, and resuming it changes nothing. The check correlates session state with Loki'sapi_requestevents by session id, with a 10-minute grace period and a 24-hour window, and names each silent session with its repository and start time. Session state gainsfirst_turn_at, set once at Stop. Each session is judged by its first completed turn, or by its latest when the first is beyond reach, so sessions already running are judged at once. Loki is asked about the last 7 days at most (less if it keeps less), whatever a session's age. A session with no turn inside that reach isunknown, not missing. Retention is Loki's globalretention_period;retention_streamrules and per-tenant overrides are not evaluated. Malformed state files are skipped one at a time, session ids are queried in batches of 50, and the Loki URL is shown without credentials.content droppedlooks at every Claude Code content setting, not just the detailed view (#164).OTEL_LOG_USER_PROMPTSorOTEL_LOG_ASSISTANT_RESPONSESon now runs the probe, which also carriesresponse.OTEL_LOG_RAW_API_BODIESon fails outright: nothing can prove a raw body is dropped. Before this, prompt logging on with the detailed view off passed as "nothing sends content".
Fixed
- A compaction's reason and token counts are recorded (#156). Every compaction said
reason=unknownwith no tokens: the PostCompact hook readcompaction_reasonandtoken_count_estimate_*, which came from the documentation and are not in a real payload. A real one (Claude Code 2.1.296) is now in the replayed fixtures. The reason comes from itstrigger, and the counts from the transcript'scompact_boundaryentry. Itscompact_summaryis content and is never read.
0.9.1 — 2026-10-03
Fixes found watching a live session: usage on the dashboard, MCP and skill names through the collector, and
skills started by slash command. Restart the collector and Grafana after upgrading (docs/local-stack.md).
Fixed
-
A skill started by slash command is recorded, and a loop skill started that way opens its scope (#144). stdtel captured skills only through the Skill tool. A skill the user types as
/namenever goes through that tool, so it was invisible. Verified:/probe-echoproduced a turn and no skill activation. A loop skill declaringtelemetry.scopetherefore never opened its container when started the usual way. The Stop hook now reads typed commands from the transcript:- A command is a skill when the harness loaded one for it (its "Base directory for this skill:"
entry), or when the catalogue knows the name. Built-ins such as
/modeland/compactare ignored. A typed skill is recorded with triggeruser-slash. /loop /epic-loop …: the harness activates onlyloop, andepic-loopis its argument. By decision, a catalogued skill named first in/loop's argument is recorded as invoked with triggerloop, so the inference is labelled and can be filtered out.- Both are
std.artefact.source=transcript. Only the command name and that first argument word are read. Replayed against the real sightline transcript, it finds the session's five/loop /epic-loopstarts, the first on 31 August, and ignores/model,/compactand/advisor.
- A command is a skill when the harness loaded one for it (its "Base directory for this skill:"
entry), or when the catalogue knows the name. Built-ins such as
-
MCP server, MCP tool and skill names survive the collector (#142). With the detailed view on, Claude Code puts them only inside
tool_parameters, beside shell commands. Every user-configured MCP call'stool_nameis the literal"mcp_tool". The collector deletedtool_parameterswhole, so the names a user opted in to never reached storage.- A new
transform/tool_nameslifts exactly four keys out first:mcp_server_name,mcp_tool_name,skill_nameandsubagent_type. Only values shaped like a name are lifted (short, no spaces); the content is then dropped as before. It runs on logs and traces, under both overlays, and is tested on the real image with a content marker beside the names. - Verified live: the first MCP call after the collector restart arrived as
llm-council, and no content key appeared in any of 66 real events. - The operational dashboard gains tables for MCP calls by server and tool, skills invoked, and
tool calls by tool and success. A hidden name reads
(hidden: detailed view off). - Tables now show their value instead of a timestamp, and cache-read tokens sit on their own axis.
- A new
-
The operational dashboard shows usage (#140). During a live session that spent $15.43 over 167 model requests, it showed none of it. Three causes:
- Nothing read Loki, where Claude Code's own per-request records land. A new top row shows spend and requests in the range, spend per hour, tokens per hour by type, requests per hour by model, and spend by skill, agent and repo. All of it is live from Loki, with no loader in between.
- "Claude Code tokens by type" read a native metric the collector drops by design since #110,
so it could never show anything. It is removed. A test now fails any panel that queries a metric
prefix in the collector's
filter/native_metrics. - Seven "/ h" panels plotted per-second rates. "0.003 turns/h" was about 11. They are scaled to per hour, and a test holds every "/ h" rate to it.
Skill panels now say "No skill ran in this range" instead of "No data". Every Loki panel's query is run against Loki in tests.
-
Grafana reads datasource provisioning only at startup, so a stack running since before 0.8.0 has no Loki datasource and every usage panel fails with "Data source not found".
docs/local-stack.mdgives the one-line fix: restart Grafana alone.
0.9.0 — 2026-10-02
A skill's cost is now the harness's own requests named for it, and stdtel's tail-rule estimate is
retired (ADR-014 decision 10). Breaking on the wire and in the warehouse, as listed below; a minor
bump, as with 0.4.0 and 0.7.0, because this is pre-1.0 and the changelog says plainly what breaks.
Upgrade both halves: the package and the plugin (stdtel-query 1.2.0 and the analyst 1.1.0 ship in
it).
Changed — breaking, on the wire and in the warehouse
- A skill's cost is the harness's own requests, and stdtel's estimate is retired (ADR-014
decision 10, #117). Native records were verified live in 0.8.0, which started the clock the ADR
set. A user will see three things:
- Skill spans carry no tokens and no model.
std.skill.load_tokens,std.skill.tail_tokens,std.skill.tail_tokens_first_only,std.skill.llm_requestsand thegen_ai.usage.*andgen_ai.request.modelattributes are gone fromkind=skillactivations, and the allowlist refuses them. Turn, sub-agent, compaction and session spans keep their counts: those are what the transcript observed, not an attribution. - Nine
skill_invocationcolumns are dropped: the three estimates,llm_requests, the four token columns andmodel. They defaulted to 0, which for every new row would have read as "measured, and free". As at the ADR-013 cutover, the old values (estimates) are not migrated. - Per-skill cost is NULL for any session without Claude Code's own telemetry on: every
machine that has not run
stdtel-install settings(or--native-onlywith the plugin) since 0.8.0. Such runs are counted asunmeasured, never as free.
- Skill spans carry no tokens and no model.
- New views:
skill_request_cost, one row per (session, prompt, skill) overllm_request_attributed;skill_activation_cost, the same with stdtel's version, plusabandoned, which is NULL when the prompt has no native record.
- Rebuilt on the new views:
- the scorecard:
total_cost_usd,total_tokens,p50_tokens,requests_derived,abandonment_rateandunmeasured_activationsreplacep50_load_tokens,p50_tail_tokensand the oldtotal_tokens; - Q5 (
skill_tokens,n_prompts,n_unmeasured); - Q6, which counts each request in a scoped prompt once, plus external spend, and reports
prompts_without_usage; - the loop and token-by-kind dashboard panels;
- Q1, whose skill row now reports the harness's requests named for a skill in the session, and counts unmeasured runs, rather than showing every skill as usage-free;
- the demo fleet, which now seeds
llm_request. Every value it seeded before is unchanged: a full dump ofgenerate(42)before and after, minus the retired fields, is byte-identical.
- the scorecard:
- The scorecard panel is now held verbatim to
scorecard.sqlby a test, and every Warehouse panel is executed against the schema in tests. Before, a panel selecting a dropped column passed all of them. stdtel-query1.2.0 and the scorecard analyst 1.1.0 read cost from the views.telemetry.emit: falsemeans less than it did. It still suppresses stdtel's named span. It cannot stop Claude Code naming the skill on its own requests, which are now where a skill's cost is read from.docs/for-developers.mdsays so.- Copilot loses nothing it had. Copilot's own hook dialect sends
transcriptPath, which the Stop hook never read, so the tail rule never produced Copilot data. Copilot per-skill tokens arrive with #115.
Privacy
- The transcript reader no longer reads a Skill tool result's content. chars/4 was the only reason it ever did.
Fixed
stdtel-install settingsno longer redirects Claude Code's telemetry without saying so (#133). It used to overwrite anOTEL_EXPORTER_OTLP_ENDPOINTalready in the file. Run from a shell withoutSTDTEL_OTLP_ENDPOINT, that pointed Claude Code at localhost. A different endpoint is now kept, with its protocol (the two only work as a pair), and reported.--replace-endpointoverwrites both.- The content check and the detailed-view gate judge the collector Claude Code's events actually
go to (#133). An
OTEL_EXPORTER_OTLP_LOGS_ENDPOINTtakes precedence, as the harness resolves it: each variable comes from the highest-precedence settings file that sets it, then from the environment. - A session open across an upgrade no longer loses its skill windows. Saved state is loaded with
unknown keys skipped. Without that, the removed
load_tokensfield would have made every 0.8 state file raise inside a hook that exits 0, so the loss would have been silent.
0.8.1 — 2026-10-01
One fix for plugin installs, which are the common case.
Added
stdtel-install settings --native-only(#134). It writes Claude Code's own telemetry settings and no hooks. A plugin install needs exactly that: the plugin registers the hooks, and 0.8.0'ssettingswrote them into the settings file as well, so each hook fired twice. The command needs nostdtel-hook, and it cannot be combined with--no-native. Thestdtel-setupskill (1.1.0) and the README now send plugin users to it.
0.8.0 — 2026-09-30
Harness-native telemetry, ADR-014. Claude Code's own per-request records become the source of cost
and tokens, loaded into llm_request. stdtel-install settings now switches Claude Code's own
telemetry on as well as registering the hooks. Use --no-native for hooks only.
docs/for-developers.md says what that sends and how to stop it. Nothing on
the wire or in the warehouse breaks: every schema change is additive.
Changed
- The collector enforces privacy on every pipeline, for both harnesses (ADR-014 decision 3, #109).
It had to be done before any native telemetry is switched on.
- Identity is removed from spans, events and metric data points. The email becomes
std.user.hash; account and organisation ids are deleted. Verified live, Claude Code puts these on every record rather than on the resource, so the old resource-level rule would not have seen them. - Content is deleted for both harnesses' native keys: Claude Code's
tool_input,tool_parameters,full_command,prompt,responseand similar; Copilot'sgithub.copilot.tool.parameters.commandand.file_path, and its GenAI message and tool-call keys. - Copilot's git remote and branch become
std.branch.hash, computed exactly as the capture side computes it, and the plain values are deleted. A remote URL can carry a token. - All of this is tested against the pinned collector image, under both the default and the Langfuse configuration. Before this change, 22 of those 24 checks failed.
- Identity is removed from spans, events and metric data points. The email becomes
Added
- Third-party skills are named from stdtel's own record (ADR-014 decision 10, #117). Without the
detailed view, Claude Code names a third-party plugin's skill
"third-party". The new viewllm_request_attributednames such a request from the one skill that stdtel saw run on the same prompt and that the harness did not name itself, labelledderived. With more than one candidate it stays"third-party". It is read-time, so loader order cannot matter. Removing the tail rule and chars/4load_tokensremains scheduled for one release after native records are verified live. skills/stdtel-setup: interactive setup for both harnesses (ADR-014 decision 12, #114). An agent runs it with the user.- It installs the hooks and switches on Claude Code's and Copilot's own telemetry without content.
- It offers Claude Code's detailed view and says plainly what it costs: shell commands, file paths and written file contents travel to the collector and are deleted there.
- It enables the view only on an explicit yes, through the new
stdtel-install detailed-view on. That command enforces the rest in code: the collector must be local or vouched for, and the content probe must pass on the spot. Otherwise it writes nothing.offremoves the flag. - The gate and
stdtel-doctor's content check now probe the endpoint Claude Code will send to, read from its settings files in its own precedence order (project local, then project, then user). Until now they probed stdtel's own endpoint. That is usually the same collector, but not after a hand edit, a reinstall with another endpoint, or a project override, and then the check proved something about a collector Claude Code would never use.
stdtel-installswitches on both harnesses' own telemetry (ADR-014 decision 2, #113).settingsnow writes Claude Code'senvblock alongside the hooks: events on, exported to the collector stdtel already uses, with repository attributes on and account uuid off.- It never writes
OTEL_LOG_TOOL_DETAILSor a metrics exporter. A detailed view the user already chose survives a reinstall.--no-nativewrites the hooks only. - New
stdtel-install copilotprints, or merges with--vscode-settings, the VS Codegithub.copilot.chat.otel.*settings withcaptureContent: false, and the Copilot CLI'sCOPILOT_OTEL_*variables. The keys come from the VS Code documentation; they have not yet been seen in a live trace (#115). docs/for-developers.mdnow says what Claude Code sends and how to stop it:STDTEL_DISABLEDdoes not. It also drops two stale ticket references left over from ADR-013.
- One table of model requests, loaded from the harness's own records (ADR-014 decisions 5 and 7,
#112).
warehouse/load_requests.pyreads Claude Code'sapi_requestevents from Loki intollm_request: tokens, exact cost, model, and the skill, agent, plugin and MCP server each request served. A live capture on Claude Code 2.1.285 confirmed thatapi_requestcarries the real skill name (probe-echo) even whereskill_activatedredacts it. That capture, scrubbed, is the replayed fixture.llm_request_change_requestjoins each request to its change request through the stdtel turn with the sameprompt_id, reusingactivation_change_requestrather than restating ADR-013's rule. Only turns bridge: a sub-agent in its own worktree shares the prompt id but not the branch.session_cost_reconciliationcompares the harness's session total with the sum of its requests. The two are never added together.- Cost is stored as unbounded
NUMERIC. The harness sends seven decimals and more, whichNUMERIC(12,4)would have rounded. - Loki's
query_rangestops at its limit silently, so the loader pages until a short page comes back. A request with no id fails the run and is counted, not dropped. make loadand the composeloaderservice now run it after the traces load.make loadis an&&chain, so a Loki outage stops the delivery half there, exactly as a Tempo outage already did.
stdtel-doctorproves the collector drops content (ADR-014 decision 13, #111). A newcontent droppedcheck sends a probe span and event holding a marker in content-shaped keys and a control id, then reads Tempo and Loki back. It passes only when the control arrived and the marker did not. "Not found" alone would also pass when nothing arrived, so a missing control fails as inconclusive. It runs by default only whenOTEL_LOG_TOOL_DETAILSis on;stdtel-doctor --content-checkruns it on demand and is whatstdtel-setupwill call before offering the detailed view. The trace loader ignoresservice.name=stdtel-probe.- Loki stores harness-native events (ADR-014 decision 4, #110). Claude Code's per-request cost,
tokens and skill attribution exist only as events, and the logs pipeline used to export them to
debugand nowhere else. Loki runs on host port 11010, this project's berth extra, and Grafana has it as a datasource. Native metrics are dropped at the collector: each carriessession.idas a label, so it opens a series per session.
Fixed
-
Coding-agent PRs were being discarded as bots; they are now agent work for their harness (ADR-014 decision 9, #116). The fix was verified with the loader's own
gh pr listcall on real public PRs:- the Copilot coding agent is
app/copilot-swe-agentand the Claude GitHub app isapp/claude, bothis_bot: true, so the #62 filter dropped every one; - both are now kept, including the
gh searchspellingsCopilotandclaude[bot]; - dependency bots are still excluded.
assisted_bynow combines every signal: the arm labels, Anthropic'sclaude-code-assistedlabel, and an agent author.- One harness evidenced is that harness, even under
no-ai, because a control an agent touched is not a control. - Two harnesses is
mixed. - The absence of evidence is still
unknown, nevernone.
- the Copilot coding agent is
-
A credential in Claude Code's repository URL no longer reaches Loki (#128). With
OTEL_METRICS_INCLUDE_REPOSITORYon, every native event carriesvcs.repository.url.full, and a remote such ashttps://x-access-token:...@github.com/...would have been stored verbatim. The collector now strips the userinfo, on the resource and on the record, for every signal. It is tested against the real image under both overlays. -
A session with no repository now has
branch_hashNULL, not''(#122). The capture side sends an empty hash on purpose; the loader stored it verbatim, sobranch_hash IS NULLundercounted sessions that cannot join a change request (ADR-005).schema.sqlconverts existing''rows, touching nothing else. -
The compose
loaderservice's traces load had never run (#121). It calledpython -m warehouse.load_traceswith no arguments while--tempoand--dsnwere required, so every cycle ended in a usage error. That happened before aloader_runrow could be written, and|| echo "traces failed"was the only sign. Both flags now fall back toSTDTEL_TEMPOandSTDTEL_DSN, which the service sets, and a test parses the service's own invocation.
0.7.0 — 2026-09-30
Changed — breaking, on the wire and in the warehouse
- Work joins change requests by branch identity, not a ticket parsed from the branch name
(ADR-013). The regex requirement was never communicated. Measured, it missed 2 of 9 of this
repository's own branches and every branch of a repository that puts the issue number last, and
it invented tickets:
RELEASE-0on both releases,ANALYZE-4andINIT-4on Dependabot branches.- On the wire: spans carry
std.branch.hash, a hash of the normalised repository and branch, never the branch name.std.ticket.idis gone.std.repois nowowner/name. - Commit evidence: the Stop hook records the
git patch-idof each commit the session made, asstd.artefact.commit_patch_ids. Patch-ids survive a rebase; the SHAs of the rebased PR #87 did not. - The join:
activation_change_request(a view) assigns each activation to the change request its branch became, and records how:branch,branch+commitorcommit. The scorecard's with-arm requires commit evidence, and a branch-only link is excluded from both arms. - Delivery data: it loads into a forge-neutral
change_requestrecord, with commits and ticket links, through a GitHub adapter. GitLab is #103. Closed-unmerged PRs are loaded. - Tickets: they are now enrichment, taken from the forge's issue links first and a fixed regex second. In a PR description, a key counts only after a linking word ("Fixes PLAT-9"). The first real run read ADR numbers cited in descriptions as tickets.
- Scope unit:
ticketis renamedbranch. - Warehouse:
pull_requestand theticket_idcolumns are dropped at cutover. Prior data is not migrated.
- On the wire: spans carry
Fixed
- The demo fleet's skill invocations never shared a span id with their activations, as the real loader's always do, so any join from one to the other found nothing.
stdtel-querygave a tool-failure query againstsession_cost.tool_failures, a column that has never existed.
0.6.0 — 2026-09-28
Added
- External contract version 3 (ADR-012).
std.external.tool_use_idjoins an external run to the exact turn that made the MCP call. The Stop hook now recordsmcp__*tool-use ids from the transcript, including sub-agent transcripts, asstd.artefact.mcp_tool_use_ids, and the loader writes them tomcp_tool_call. An MCP server'ssession.idgoes stale after/clear, so the join is what attributes its spend correctly. Through the joined turn, external spend also gains a scope.std.external.requests_unpricedkeeps a partly priced run's observed cost as a lower bound instead of dropping it.- Query 7 version 3 reports partial runs apart from complete ones, and reports the join rate.
stdtel-conform --print-contractreportscontract_version: 3. (#94)
Fixed
<synthetic>is no longer counted as a model request. Claude Code uses that name for placeholder assistant messages, such as interrupted or failed responses, which carry zero usage. Counting them inflatedllm_requests, and when one came first in a turn it became the span's model: one Prometheus series was already labelledgen_ai_request_model="<synthetic>". In 1,157 real turns, every turn that appeared to use more than one model was really one model plus<synthetic>.- A span names a model only when its run used exactly one. This applies ADR-012 decision 5 to our own turns, sub-agents and skill tails. Genuinely multi-model runs were sub-agents, 5 of 200 sampled, and naming the first of two models said the run used one.
scrub()dropped every list attribute without a word. It now passes a list of strings and nothing richer.stdtel-conformread an OTLPintValueas a string, and would have refused every correctly sent count.- Collector: spanmetrics now counts across resources instead of per resource, so dashboards no longer read zero after a branch switch, a project change or an SDK upgrade. (#92)
mise run loadworks again, having failed since #68, and the loader no longer reports keys that our own Langfuse overlay adds. (#93)
0.5.0 — 2026-09-28
Changed
- Activations are children of their turn, not flat roots (ADR-010). A skill or sub-agent
activation now carries the span id of the turn it ran under, so a trace shows the turn and what happened
inside it. Compactions, and activations whose turn is not in the batch, stay roots, because that is
what was observed. Anyone querying Tempo for root spans will see fewer of them. The warehouse
gains
parent_span_idand is otherwise unaffected. A session is deliberately never a parent: one real session spans seven weeks, which no trace store will hold. (#75, #80) - The ticket follows the branch, not the session. The ticket key was read once, at session start, so a session that switched branches joined every later activation to the wrong PR. It is now refreshed on each event, and an empty read keeps the last known ticket rather than blanking it. (#77, #81)
- A concurrent state write no longer ends a session's telemetry. Sub-agent hooks fire while the parent is still calling tools, so two hook processes could write the state file at once and leave it unparseable. The hook swallowed the error and exited 0, and that session emitted nothing more. Writes are now atomic (a finished file is renamed into place). A torn file is salvaged from its intact leading document, which keeps the transcript offset, and the salvage is reported. Sub-agent and compaction hooks update the state file under a lock, so parallel writers no longer drop each other's updates. (#73, #74)
Added
- Containment across turns: scopes (ADR-010, ADR-011). A skill that drives many turns (a loop
over tickets) can declare
metadata.telemetry.scope: ticket. Everything recorded while its container is open carriesstd.scope.name/key/id, andwarehouse/efficiency/06_scope_self_vs_inclusive.sqlreports its own cost beside everything incurred under it. Self cost is stored; inclusive cost is derived — no row holds a total including its children. A rollup answers containment, never causation, and the analyst brief says so.stdtel-hook scope-closeends a scope explicitly. Nothing is required of any author: an undeclared skill is turn-scoped. (#78, #79, #82) - Decorated sub-agents. An agent file's
metadata:block (version,owner,standard_id,policy_ids) is now captured on its activations. This works by tolerance, not contract: verified on Claude Code 2.1.277, undocumented upstream. Agents cannot declare a scope. MCP servers cannot be decorated at all and are overlay-only, which ADR-011 records as a non-goal.stdtel-onboardcovers all three artefact types. (#83) - A fifth kind,
external, for spend the harness cannot see (ADR-010 decision 7). A process an agent calls, such as an MCP server that pays for its own model calls, reports its spend asstd.artefact.activationspans withkind=external.warehouse/efficiency/07_external_spend_and_coverage.sqlreports the spend and, separately, how much of it is known. (#86, #87) stdtel-conform, a new console script. It checks a foreign emitter's OTLP JSON against the external contract in their CI: span name, allowlist, no content, a named system, a well-formed session id, and costs that are numbers or absent. It reports cost coverage separately, because a shape-only check passes a file whose costs are missing.--print-contractprints the contract as JSON with acontract_version, for an emitter to diff its own copy against. (#87, #89)- External contract version 2: cost provenance.
std.external.cost_usdkeeps meaning observed;std.external.cost_source(provider|local) says how;std.external.cost_estimated_usdcarries the part priced from a list, beside it and never inside it. One run can mix billed and estimated calls, so one label over one figure had no honest value. Additive: every version-1 span stays valid. (#88, #89) - The loader names what it cannot load. Tempo keeps an attribute nobody declared; the loader was
where it vanished, without a word. It now names every unread key on an external span, with counts,
and records them in
loader_run.unknown_attr_keys. A renamed attribute therefore arrives as a named key rather than a column of NULLs. (#88, #89) skills/stdtel-instrument, a new skill. It adapts an external process to the contract and starts by reconciling the process's own records against the provider's bill. It records the verified fact that Claude Code strips an inheritedOTEL_*variable before starting an MCP server, so the endpoint must be set in the server'senvblock. (#87, #89)- The primary metric has an input. CI has uploaded a policy-results artefact on every PR run, and
nothing ever collected it, so first-time OPA pass rate had no data while CI stayed green.
warehouse/fetch_policy_results.pycollects it. (#21, #72)
Fixed
- The loader window is guarded. A sub-agent span is back-dated to the sub-agent's own start and is
not searchable until Tempo's ingester flushes. A
--sincenarrower than that delay stepped over such spans and never came back for them. It now warns. (#67, #71) - The release checks every console script. The wheel check named seven scripts by hand and missed
stdtel-conform, which would have shipped without ever running in the build. The list now has to coverpyproject.toml, and a test enforces it. - The exporter timeout is tested by behaviour.
opentelemetry-exporter-otlp-proto-http1.45.0 removed the private attribute the test read, so every CI run failed while the timeout still worked: 1.00s against a collector that never answers. The test now measures exactly that. (#90)
0.4.0 — 2026-09-16
Changed
- The unit of capture is an artefact activation, not a skill invocation (ADR-009). The span
std.skill.invocationis renamed tostd.artefact.activation, discriminated bystd.artefact.kind∈skill,subagent,compaction,turn. This is a wire-level rename and anyone querying Tempo or Prometheus for the old name must update; the shipped dashboards anddeploy/smoke.shalready have. There is deliberately no dual-emission period — two spans would have doubled everytraces_span_metrics_calls_totalseries and both dashboards. Compatibility lives one layer down:load_tracesstill writesskill_invocationfrom kindskill, soscorecard.sql, every existing query and every row recorded before the rename keep working unchanged, and spans still in Tempo under the old name keep loading. (#63)
Added
- Sub-agent, compaction and turn capture. A real session of this project's own spent $366, spawned
42 sub-agents and compacted twice, and recorded one skill row and session totals — the numbers being
collected could not answer "where did it go". New hooks
SubagentStart,SubagentStopandPostCompact; where a hook has never fired the value is read from the session directory instead and flaggedstd.artefact.source=transcript, so an inference is never mistaken for an observation. - The session's real cost.
session_cost.cost_usdhas been NULL for every row since the schema was written, because nothing read the harness's owncost-stateentry. It and the harness's API and tool wall-clock totals are now loaded. - Per-hook latency on each turn, as
std.turn.hook.<basename>.ms. The basename only — the payload carries an absolute path, which is filesystem layout rather than data. warehouse/efficiency/: five versioned queries answering the named efficiency questions, and a session-efficiency brief inagents/skill-scorecard-analystthat runs them rather than writing its own. Efficiency is answerable for a single session at any volume; the primary effectiveness metric and its 30-merged-PRs-per-arm floor are unchanged.- Scheduled loaders.
mise run load,mise run load-watch, and an opt-inloadercompose profile. Tempo held four days of spans the warehouse had never seen becauseload_traces.pyis run by hand. Every run writes aloader_runrow, including failures, and a dashboard panel shows how stale everything else on the page is. stdtel-doctorreports which of the new hook events have actually fired on this machine. Their payload shapes are transcribed from documentation, not captured, because hook configuration is read at session start and a session cannot observe a hook it just registered.
Fixed
- Dependabot branches minted phantom tickets (
upload-artifact-7→ARTIFACT-7), on spans as well as in thetickettable, putting work no developer did into the without-skill cohort. Both the capture and delivery parsers now anchor a ticket key to the start of a branch segment, bot-authored PRs are skipped, and a test pins the two parsers together because ADR-002 joins them on equality. (#62) span_id()silently mangled hex ids. A 16-character hex span id from Tempo's search API is also valid base64, so it was decoded into 12 bytes of a different id without raising. It now decodes only when the result is the length an id actually is.
Known issue
-
A sub-agent span stamped in the past is not searchable in Tempo until the ingester flushes its block, up to about half an hour later. Nothing is lost, but a loader window narrower than that delay would step over those rows permanently. (#67)
Correction, 2026-09-16. As shipped, this entry said such a span "never becomes searchable and never reaches the warehouse". That was wrong: it was measured before the block had flushed and not re-checked afterwards. The span appears, and the row loads. The real requirement is a loader window wider than the flush delay, which the default already is; a guard and a test were added afterwards.
0.3.3 — 2026-09-14
Changed
- Every GitHub Action is pinned to a commit SHA, with the version in a trailing comment. A tag is a
pointer somebody else can move, and the job that publishes holds
id-token: write— so whoever controlledpypa/gh-action-pypi-publish@release/v1controlled what shipped asstdtel, permanently. Two tests enforce it: no mutable reference anywhere, and every pin carrying the version it came from. (#34) - The published artefact is now verified after it lands. A new
verifyjob installsstdtel==<tag>from PyPI, runs its console scripts, checks the reported version, and verifies the PEP 740 attestation. Everything before it proved the wheel we built worked; nothing proved the one on the index did, and the two have differed before. It is not grantedid-token: write— reading an attestation does not require minting a credential. - A release must have a matching section in
CHANGELOG.mdor the build fails. Release notes are written from there, and a missing section means they are being improvised at the moment of shipping. - The sdist no longer ships
tests/. It carriedtests/*.pywithoutconftest.pyor the fixtures, so the suite could not even be collected — worse than shipping none, because it reads as a broken package.
0.3.2 — 2026-09-14
Added
- Drift between the plugin and the package is now reported, in both directions. They are separate
installs and neither updates the other, so a developer updates one and assumes the other followed.
Both failures are quiet: a missing package leaves the launcher exiting 0 in silence, and a stale
plugin keeps working while the skills it ships lag the release.
stdtel-doctorgains a plugin in step check. Not having the plugin passes —stdtel-install settingsis a supported install — but a version mismatch fails, naming the command that fixes it, which differs by direction.session_startprints the same line once per session, on the only hook whose output a developer reads.- The plugin's launcher announces a missing package at session start. This closes a
circularity: with the package absent nothing of ours runs,
stdtel-doctorincluded, so the launcher is the only part of the install that can report it. Every other event stays silent — the same line on every tool call is noise, not information.
Fixed
stdtel.__version__said0.1.0. Hardcoded in the first commit and wrong for every release since; nothing read it, so nothing noticed. It is now derived from the installed package metadata, and the version-agreement test covers it — the skew check above depends on it being true.
0.3.1 — 2026-09-14
Fixed
stdtel-doctorreported0 attributed by overlaywhile two skills were being attributed by one. It counted manifests whose file sat in an overlay root, butfill_gapsreturns a manifest based on the skill's own file, so a filled skill never matched. It now counts what the overlay actually contributed. Found by reading the number against a real overlay rather than trusting it.- Test isolation now clears every
STDTEL_*variable rather than a list of known ones. OnceSTDTEL_SKILLS_OVERLAYexisted, a developer who had configured one saw two unrelated tests fail, because their real overlay leaked into assertions about an empty catalogue. Enumerating the environment cannot go stale; a list can.
0.3.0 — 2026-09-14
A minor rather than a patch: a new span attribute, a new environment variable and a new warehouse column. Nothing is removed and no existing field changes meaning.
Added
std.skill.content_hash— SHA-256 of the SKILL.md body, truncated. Every version this system records is asserted by whoever wrote the front-matter; nothing ever checked it against the instruction the model was given, which ADR-005 forbids. The hash is observed: same version with two hashes is an edit that skipped the bump, and that scorecard row is aggregating two populations. The data-quality panel counts them. Deliberately not a spanmetrics dimension — a series per edit is #42 again. (#52)STDTEL_SKILLS_OVERLAY— roots of front-matter-only stubs that attribute skills you do not own without editing them. Editing a third-partySKILL.mdsurvives until the next upstream release, plugin reinstall or new machine, and fails silently each time. Overlays fill gaps and never override, the reverse ofSTDTEL_SKILLS_ROOT: an overlay that overrode would keep asserting its pinned version after upstream declared a real one, with nothing to notice it by. A stub may omitversion, and never supplies a content hash — its body is the operator's note, not the skill's instruction. (#51)
Changed
- A manifest that fails the contract is kept in degraded form rather than dropped by lenient loading, carrying its name and content hash. Dropping it cost us the only thing about an un-onboarded skill that can be observed rather than asserted.
- Empty contract fields are omitted from spans instead of emitted blank. A blank
std.standard_idreads as a value in every dashboard that groups by it. stdtel-doctorreports unversioned and overlay-attributed counts.
Fixed
warehouse/schema.sqlnow migrates an existing database.CREATE TABLE IF NOT EXISTSdoes nothing to a table that already exists, socontent_hashwould have reached new warehouses only and the loader would have failed on its first INSERT after the upgrade.
0.2.5 — 2026-09-14
Fixed
stdtel-validatesaid a manifest was invalid without saying which one, and stopped at the first failure. Onboarding a directory of skills meant one run per problem, each run's output identical to the last. The path was available where the error was raised and was being thrown away. Every failure now names its file, all of them are reported in one run, and a manifest with nometadata:block at all is told that the contract fields live there — the thing "missing required field: version" does not convey to someone seeing it for the first time. (#49)
Changed
policy_idsmay be empty when the skill does not claim a policy signal. It was required to be non-empty on every skill, which is a rule nobody can satisfy honestly: plenty of useful skills have no deterministic check, and the gate pushed people to cite an unrelated policy to get past it — worse than an empty list, because the scorecard would then score the skill against a rule it has nothing to do with. The rule now binds the pair: emptypolicy_idsis accepted only withtelemetry.success_signal: testormanual, and the signal still defaults topolicy. ADR-004 carries the reasoning;skills/stdtel-onboardis at 1.1.0 because its guidance changed.
0.2.4 — 2026-09-13
Fixed
- Span-metric counters could never increase, so three dashboard panels read zero at any volume.
Unset, the OpenTelemetry SDK generates
service.instance.idper process, andprometheusremotewritemaps it to theinstancelabel — so every hook, being its own process, opened a brand-new series which received exactly one increment and was abandoned.rate()needs two points in one series and always had one. Measured before the fix:max(traces_span_metrics_calls_total)= 1 across 563 samples, with cardinality growing by a series set per hook invocation. Identity is now derived rather than generated (stdtel/identity.py), and the metrics pipeline dropsservice.instance.idregardless of who sent it.deploy/smoke.shemits from two separate processes and fails if the counter does not reach 2. (#42) distinct_usersread 0 for every skill, at every volume. The collector derivedstd.user.hashfromuser.email, which nothing ever set, so the scorecard'sreview-single-userverdict could never fire and the "5+ developers" half of the reporting floor was measured by a column pinned at zero. The hook now emits a pseudonymous per-developer id — SHA-256 of uid and hostname, hashed before it leaves the process, never reversible to a name. (#43)stdtel-doctorreported "hooks running" when the current project was recording nothing. Every state file on the machine belonged to other projects; the session being watched had none. The check now says which project its newest data came from, and fails when none of it is from here. Claude Code reads hook configuration at session start, so a session already open when stdtel was installed runs no hooks for its entire life —stdtel-installnow says so, and it is the first entry under "empty dashboards" in SUPPORT.md. (#44)
0.2.3 — 2026-09-12
First release published to PyPI: uv tool install stdtel.
Changed
- Publishing to the real index is release-only. A manual dispatch skipped the version-vs-tag check
(gated on
github.event_name == 'release'), so any build could have gone out under any version — permanently, since PyPI never allows re-uploading a version. Manual dispatch now targets Test PyPI only. Added a pre-flight check that the version is free on the target index, so a collision names the version instead of surfacing as an opaque 400 at the final step.
Fixed
- Hooks failed with
stdtel-hook: command not foundon every tool call. The shipped manifests used the bare command name, which does not resolve in a hook's non-loginsh -c. They now invoke${CLAUDE_PLUGIN_ROOT}/bin/stdtel-hook, a launcher that finds the real CLI ($STDTEL_HOOK_BIN,~/.local/bin, the uv-tool and pipx locations, then PATH) and exits 0 in silence when it finds none — an uninstalled plugin should record nothing, not error on every keystroke. Amplified by 0.2.0's unmatchedPostToolUse, which fires on every tool rather than only Skill.
0.2.2 — 2026-09-12
Fixed
- Plugin failed to load its hooks.
hooks/hooks.jsonat the plugin root is discovered automatically, so"hooks": "./hooks/hooks.json"in the manifest registered it twice: "Duplicate hooks file detected ... manifest.hooks should only reference additional hook files." The declaration is removed; the file still ships and is still loaded.claude plugin validateaccepts the duplicate, so this was only observable by installing.
0.2.1 — 2026-09-12
Fixed
- Plugin installation.
agentsin the plugin manifest takes a list of agent files; 0.2.0 shipped["./agents/"], a directory, and every install failed withagents.0: Invalid input. The manifests are now checked byclaude plugin validateitself — 0.2.0's tests only compared them against our owninstall.py::EVENTStable, and one test asserted the broken value, pinning it. make downleft six containers running.docker compose downignores profiled services unless named, so the Langfuse containers survived and the network could not be removed (Resource is still in use).downnow passes--profile langfuse.
Changed
- A test that claimed to verify the scorecard agent's refusal behaviour was reading markdown for a
substring; renamed to say what it checks. ADR-007 (proposed) records that no test here verifies any
behaviour of a skill or agent, and issue #10 tracks adopting
claude plugin evalfor that.
0.2.0 — 2026-09-10
Added
std.session.costspans: total session spend, emitted even when no skill loads, giving cost-per-PR a complete denominator. Loaded intosession_cost.- Tool-call counts and failures across every tool, carried on the session span.
warehouse/load_delivery.py: GitHub →ticket/pull_request/defect, andpolicy_resultfrom a CI artefact.stdtel-install, which binds hooks to an absolute path at install time.- Plugin layout for three loaders: root
plugin.json(Agent Plugins v1),.claude-plugin/,com.github.copilot/. policies/:logging.required_fields,logging.no_pii,telemetry.manifest_valid— the primary metric computes for real, where before--dry-runreturnedTruefor every policy.skills/stdtel-onboard,skills/stdtel-query,agents/skill-scorecard-analyst.mise run smoke: eight-hop verification ladder for the local stack.- Docs: reference, skills, evaluation-power, insight-walkthroughs, local-stack, ADRs 002–005.
Changed
SKILL.mdcontract fields moved undermetadata:for Agent Skills conformance. The flat form still parses, so existing catalogues keep validating.stop()returns the total number of spans emitted, which now includes the session-cost span.PostToolUse/PostToolUseFailureare no longer matched toSkill.- Configuration moved to
STDTEL_OTLP_ENDPOINT/STDTEL_OTLP_TIMEOUT;OTEL_*cannot reach a hook. stdtel-validateexits 2 on an unreadable root, where it previously exited 0.
Fixed
- Empty
transcript_pathresolved to"."and killed all telemetry silently. - The configured collector endpoint never reached the exporter, which fell back to localhost.
- A dead collector stalled the Stop hook ~7.3s per turn.
- Copilot events arriving via
~/.claude/settings.jsonwere stampedclaude-code. - Namespaced (
plugin:skill) names matched no catalogue entry — 71% of real invocations. std.skill.triggerwas a constant; it now comes from the transcript or readsunknown.SessionState.save()dropped fields it did not list, silently, across hook processes.- Three never-executed bugs in
warehouse/load_traces.py(OTLP id encoding, missing end time).
0.1.0
Initial scaffold: hooks, tail attribution, collector config, warehouse schema, eval runner.