Building a serverless database replica with Carl Sverre

Database School

Want to learn more SQLite? Check out my SQLite course: https://highperformancesqlite.com

In this episode, Carl Sverre and I discuss why syncing everything is a bad idea and how his new project, Graft, makes edge-native, partially replicated databases possible. We dig into SQLite, object storage, transactional guarantees, and why Graft might be the foundation for serverless database replicas.

SQLSync: https://sqlsync.dev
Stop syncing everything blog post: https://sqlsync.dev/posts/stop-syncing-everything
Graft: https://github.com/orbitinghail/graft

Follow Carl:
Twitter: https://twitter.com/carlsverre
LinkedIn: https://www.linkedin.com/in/carlsverre
Website: https://carlsverre.com/

Follow Aaron: Twitter: https://twitter.com/aarondfrancis
LinkedIn: https://www.linkedin.com/in/aarondfrancis
Website: https://aaronfrancis.com - find articles, podcasts, courses, and more.

Database school on YouTube: https://www.youtube.com/playlist?list=PLI72dgeNJtzqElnNB6sQoAn2R-F3Vqm15
Database school audio only: https://databaseschool.transistor.fm

Chapters: 00:00 - Intro and Carl’s controversial blog title
01:00 - Why “stop syncing everything” doesn't mean stop syncing
02:30 - The problem with full database syncs
03:20 - Quick recap of SQL Sync and multiplayer SQLite
04:45 - How SQL Sync works using physical replication
06:00 - The limitations that led to building Graft
09:00 - What is Graft? A high-level overview
16:30 - Syncing architecture: how Graft scales
18:00 - Graft's stateless design and Fly.io integration
20:00 - S3 compatibility and using Tigris as backend
22:00 - Latency tuning and express zone support
24:00 - Can Graft run locally or with Minio?
27:00 - Page store vs meta store in Graft
36:00 - Index-aware prefetching in SQLite
38:00 - Prefetching intelligence: Graft vs driver
40:00 - The benefits of Graft's architectural simplicity
48:00 - Three use cases: apps, web apps, and replicas
50:00 - Sync timing and perceived latency
59:00 - Replaying transactions vs logical conflict resolution
1:03:00 - What’s next for Graft and how to get involved
1:05:00 - Hacker News reception and blog post feedback
1:06:30 - Closing thoughts and where to find Carl