long-context
Extend context windows of transformer models using RoPE, YaRN, ALiBi, and position interpolation techniques. Use when processing long documents (32k-128k+ tokens), extending pre-trained models beyond original context limits, or implementing efficient positional encodings. Covers rotary embeddings, attention biases, interpolation methods, and extrapolation strategies for LLMs.
- Version
- 1.0.0
- License
- MIT
Pinned to revision df088027ff23, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/long-context/SKILL.md
- skills/long-context/references/extension_methods.md
- skills/long-context/references/fine_tuning.md
- skills/long-context/references/implementation-patterns.md
- skills/long-context/references/rope.md
Every link opens the file at its source, pinned to the revision this page describes.