stevenacoffman/data-systems

Use when choosing between batch and stream processing for a data pipeline that builds derived outputs (search indexes, ML features, analytics), deciding how often a job runs, fixing stale results, or recovering derived data after a pipeline bug. Not for OLTP request latency.

Use when a distributed system orders events, resolves conflicts, or deduplicates by wall-clock timestamps from multiple nodes: last-write-wins (LWW) in Cassandra or ScyllaDB, audit logs or ledgers sorted by timestamp, or claims that NTP is good enough for ordering.

Use when deciding whether an operation on replicated storage needs linearizability or can accept eventual consistency: uniqueness constraints, quotas, rate limits, distributed locks, concurrent counters, or check-then-act reads from a replica. Not for single-node databases.

Use when designing or reviewing a distributed lock, lease, or leader election (Redis SETNX, ZooKeeper, etcd) that guards writes to shared storage or a job, or when a post-mortem shows duplicate writes, double job runs, or split-brain despite locking. Not for single-transaction row locks.

Use when read replicas make users see wrong or confusing data, not merely slow data: a just-saved change shows the old value (read-your-writes), data moves backward on refresh (monotonic reads), or a reply appears before its question (consistent prefix).

Use when choosing single-leader, multi-leader, or leaderless database replication, or when a topology causes trouble: multi-region writes, failover data loss or split-brain, write conflicts needing resolution, or assuming Cassandra quorum reads give strong consistency.

Use when data volume or write throughput outgrows one node and you must shard: choosing key-range vs hash partitioning, fixing a hot shard, designing compound or time-series keys. Not for read-only scaling (use read replicas) or single-machine Postgres table partitioning.

Use when selecting a storage engine or database for a new system, or when performance problems stem from the workload rather than queries or schema: write/read ratio, point vs range access, OLTP vs analytical column scans, LSM-tree vs B-tree.

Use when an application writes to several stores (database plus search index, cache, or warehouse) and you must decide which is authoritative, when two stores have diverged, when keeping a new downstream system in sync, or when a migration must reach multiple stores.

Use when someone proposes relaxing consistency, accepting stale reads, or using eventual consistency or async pipelines, to decide which operations tolerate delay (timeliness) and which risk permanent corruption (integrity), or when data loss was misdiagnosed as stale reads.

Use when choosing or verifying a database isolation level, or debugging intermittent concurrency anomalies (lost updates, dirty reads, write skew, phantoms): check-then-act bookings or quotas, ORM read-modify-write, or assuming ACID, serializable, or Postgres or MySQL defaults make it safe.