Skip to content

xberg-io/crawlberg

v1.10.3MIT

Web crawling and scraping with HTML→Markdown and headless-Chrome fallback.

crawlberg-headless-fallback

Use when a static fetch returns nothing useful and the page needs a real browser. Covers --browser-mode auto|always|never, external CDP via --browser-endpoint, symptoms of JS-only pages and WAF blocks, and the performance cost.

Read SKILL.md at the source

Pinned to revision c7ccb675ee70, so it is the text this page describes rather than whatever the author pushed since.

Files

Every link opens the file at its source, pinned to the revision this page describes.