zytedata/zyte-web-data

Build a full-site or multi-page Scrapy spider to get structured data from a website, in a new project or added to an existing one. Do not use for analyzing locally saved HTML, fixing or debugging an existing spider, schema-only or existing-spec work, Scrapy Cloud operations, or advisory questions.

Extract structured data (all available fields with values) from a page saved locally as an HTML file, optionally following a schema. Do not invoke when the user provides a URL. When invoking, pass the user's full request verbatim as args — do not pre-parse file paths and don't rephrase it.

Plan a web scrape and author a validated extraction spec — explore a detail page, discover fields, confirm the schema, then validate it with more pages, HTML-variant comparison, extracted values, and an optional browser review, and present the plan for approval.

Read this BEFORE writing, modifying, debugging, or explaining any Scrapy or web-poet code — spiders, page objects, pipelines, settings, selectors — down to a one-line fix, or scaffolding/configuring a Scrapy project, scrapy-poet, scrapy-zyte-api, or browser rendering. These libraries moved on since your training cutoff; invoke this FIRST. NOT for building a scraper for a given site (scrape), a saved HTML file (scrape-analyze-page), or Scrapy Cloud deploy/jobs (zyte).

Zyte APIs and cloud services: credentials (set up, log in, get an API key, or a blocked site); Scrapy Cloud (deploy, schedule or run a spider, manage cron jobs, list/stop jobs, inspect items and logs); recorded API usage (spend, requests, response times, status codes) and pricing (plans, tiers, discounts, cost estimates); how-to and docs questions about any Zyte product or account/billing, from docs.zyte.com. NOT for writing/debugging spiders locally (scrapy-extra) or estimating spend from code instead of recorded stats.