datastore-selection
Decide where data should live: a relational OLTP database, a columnar warehouse, a document store, a key-value cache, a graph database, a data lake or lakehouse, or a combination. Use whenever someone asks which database or storage to use for an application, feature or data product; SQL or NoSQL; Postgres vs MongoDB vs Cassandra vs Redis vs Neo4j vs ClickHouse vs BigQuery; data lake vs warehouse vs lakehouse or Delta Lake; OLTP vs OLAP; whether a system needs ACID transactions or can live with eventual consistency; how the CAP theorem applies; or how to design polyglot persistence for a platform with very different kinds of data. Also use to review a storage choice that is struggling with scale, latency or cost. Not for picking distribution or partition keys inside a chosen warehouse, and not for pipeline design or data quality rules.
Pinned to revision a18d88341e79, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/datastore-selection/SKILL.md
- skills/datastore-selection/references/analytical-columnar.md
- skills/datastore-selection/references/document.md
- skills/datastore-selection/references/graph.md
- skills/datastore-selection/references/key-value.md
- skills/datastore-selection/references/lake-and-lakehouse.md
- skills/datastore-selection/references/relational.md
Every link opens the file at its source, pinned to the revision this page describes.