llm-observability
Monitor and observe LLM applications in production. Covers token usage tracking and cost monitoring, latency monitoring (TTFT, TPS, E2E), LangSmith tracing, LangFuse integration, Phoenix/Arize for LLM observability, prompt/completion logging, conversation tracking, quality metrics over time, error rate monitoring, rate limit tracking, model comparison dashboards, feedback collection, A/B test analysis, hallucination rate monitoring, and LLM-specific alerting. Use when monitoring LLM applications, tracking costs, debugging quality issues, or setting up LLM observability infrastructure.
- Version
- 1.0
- License
- Apache-2.0
Pinned to revision 45cf0fa3c5e7, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/llm-observability/SKILL.md
- skills/llm-observability/references/REFERENCE.md
- skills/llm-observability/scripts/llm_monitor.py
- skills/llm-observability/scripts/quality_tracker.py
Every link opens the file at its source, pinned to the revision this page describes.