rag-pipeline
Build a RAG (retrieval-augmented generation) pipeline or a custom search engine
on top of Bright Data's Discover API — using intent-ranked web results + parsed
page content as the retrieval/ingestion layer for an LLM or vector store. Use
when the user wants to "build a RAG pipeline", "add web search to my LLM/agent",
"ground my model in live web data", "build a search engine over the web",
"ingest web content into a vector DB / knowledge base", or "give my chatbot
retrieval". Covers both live retrieval (Discover at query time as a web-grounded
retriever) and ingestion (Discover → chunk → embed → vector store → retrieve).
Built on the discover-api skill. For a one-off written report use
live-research; for raw markdown of specific known URLs use scrape.
- Version
- 1.0
Pinned to revision 4e8964e24a5a, so it is the text this page describes rather than whatever the author pushed since.
Files
Every link opens the file at its source, pinned to the revision this page describes.