Leoirc's Blogs

原文 https://www.cockroachlabs.com/blog/how-we-built-cockroachdb-serverless/

We recently announced general availability (GA) for Serverless, with support for change data capture (CDC), backup and restore, and a 99.99% uptime SLA. Read on to learn how CockroachDB Serverless works from the inside out, and why we can give it away for free – not free for some limited period, but free. It required some significant and fascinating engineering to get us there. I think you’ll enjoy reading about it in this blog or watching the recent presentation I gave with my colleague Emily Horing:

我们最近宣布了无服务器的全面可用性 (GA)，支持变更数据捕获 (CDC)、备份和恢复以及 99.99% 的正常运行时间 SLA。 请继续阅读，了解 CockroachDB Serverless 从内到外的工作原理，以及为什么我们可以免费赠送它——不是在有限的时间内免费，而是免费。 我们需要一些重要且令人着迷的工程才能实现这一目标。 我想您会喜欢在这个博客中阅读相关内容或观看我最近与同事 Emily Horing 进行的演示：

Raft 是一种用来管理日志复制的一致性算法。

列举资料：

论文：https://pdos.csail.mit.edu/6.824/papers/raft-extended.pdf

动画：https://thesecretlivesofdata.com/raft/

go库：https://pkg.go.dev/go.etcd.io/etcd@v3.3.27+incompatible/raft

go版实现：https://github.com/etcd-io/raft

https://raft.github.io/

This paper describes a new extensible query optimization framework that resolves many of the short- comings of the EXODUS and Volcano optimizer generators. In addition to extensibility, dynamic pro- gramming, and memorization based on and extended from the EXODUS and Volcano prototypes, this new optimizer provides (i) manipulation of operator arguments using rules or functions, (ii) operators that are both logical and physical for predicates etc., (iii) schema-specific rules for materialized views, (iv) rules to insert ”enforcers” or ”glue operators,” (v) rule-specific guidance, permitting grouping of rules, (vi) basic facilities that will later permit parallel search, partially ordered cost measures, and dy- namic plans, (vii) extensive tracing support, and (viii) a clean interface and implementation making full use of the abstraction mechanisms of C++. We describe and justify our design choices for each of these issues. The optimizer system described here is operational and will serve as the foundation for new query optimizers in Tandem’s NonStop SQL product and in Microsoft’s SQL Server product.

本文描述了一种新的可扩展查询优化框架，它解决了 EXODUS 和 Volcano 优化器生成器的许多缺点。 除了基于 EXODUS 和 Volcano 原型并从其扩展的可扩展性、动态编程和记忆之外，这个新的优化器还提供（i）使用规则或函数操作运算符参数，（ii）逻辑和物理运算符 谓词等，(iii) 物化视图的特定于模式的规则，(iv) 插入“执行者”或“粘合操作符”的规则，(v) 特定于规则的指导，允许对规则进行分组，(vi) 将 后来允许并行搜索、部分有序成本测量和动态计划，(vii) 广泛的跟踪支持，以及 (viii) 充分利用 C++ 抽象机制的干净接口和实现。 我们针对每个问题描述并证明我们的设计选择。 这里描述的优化器系统是可操作的，并将作为 Tandem NonStop SQL 产品和 Microsoft SQL Server 产品中新查询优化器的基础。

Commercial databases support different isolation levels to allow programmers to trade off consistency for a poten- tial gain in performance. The isolation levels are defined in the current ANSI standard, but the definitions are ambigu- ous and revised definitions proposed to correct the problem are too constrained since they allow only pessimistic (lock- ing) implementations. This paper presents new specifica- tions for the ANSI levels. Our specifications are portable*; they apply not only to locking implementations, but also to optimistic and multi-version concurrency control schemes. Furthermore, unlike earlier definitions, our new specifica- tions handle predicates in a correct and flexible manner at all levels.*

商业数据库支持不同的隔离级别，使程序员可以牺牲一致性来获得潜在的性能提升。 隔离级别在当前的 ANSI 标准中进行了定义，但这些定义是不明确的，并且为纠正该问题而提出的修订定义过于受限，因为它们只允许悲观（锁定）实现。 本文提出了 ANSI 级别的新规范。 我们的规格是便携式的； 它们不仅适用于锁定实现，还适用于乐观和多版本并发控制方案。 此外，与早期的定义不同，我们的新规范在各个级别以正确且灵活的方式处理谓词。

CockroachDB supports ACID transactions across arbitrary data in a distributed database. A discussion on how this works was first published on our blog three years ago. Since then, a lot has changed. Perhaps most notably, CockroachDB has transitioned from a key-value store to a full SQL database that can be plugged in as a scalable, highly-available replacement for PostgreSQL. It did so by introducing a SQL execution engine which maps SQL tables onto its distributed key-value architecture. However, over this period of time, the fundamentals of the distributed, atomic transaction protocol at the core of CockroachDB have remained untouched 1.

CockroachDB 支持分布式数据库中任意数据的 ACID 事务。 三年前，我们的博客首次发表了关于其工作原理的讨论。 从那时起，很多事情都发生了变化。 也许最值得注意的是，CockroachDB 已经从键值存储转变为完整的 SQL 数据库，可以作为 PostgreSQL 的可扩展、高可用性替代品插入。 它通过引入 SQL 执行引擎来实现这一点，该引擎将 SQL 表映射到其分布式键值架构上。 然而，在这段时间里，CockroachDB 核心的分布式原子事务协议的基本原理并未受到影响1。

This article was written in 2015 when CockroachDB was pre-beta. The product has evolved significantly since then. We will be updating this post to reflect the current status of CockroachDB. In the meantime, the transaction section of the Architecture Document provides a more current description of CockroachDB’s transaction model.

本文写于 2015 年，当时 CockroachDB 还处于预测试阶段。 从那时起，该产品已经发生了显着的发展。 我们将更新这篇文章以反映 CockroachDB 的当前状态。 同时，架构文档的事务部分提供了 CockroachDB 事务模型的更新描述。

原文：https://www.cockroachlabs.com/blog/serializable-lockless-distributed-isolation-cockroachdb/

Editor’s Note*: This post was originally authored when CockroachDB was pre-1.0. CockroachDB’s architecture has undergone many changes since then. One of the most significant, as it relates to this post which focuses on our previous “lockless” design, is that we now use more locking and lock-like structures to provide SERIALIZABLE isolation. For more current details about CockroachDB’s transaction model, read our transaction layer architecture documentation.*

编者注：这篇文章最初是在 CockroachDB 1.0 版本之前撰写的。 从那时起，CockroachDB 的架构经历了许多变化。 最重要的一个，因为它与这篇文章相关，重点关注我们之前的“无锁”设计，我们现在使用更多的锁定和类似锁的结构来提供可串行化的隔离。 有关 CockroachDB 事务模型的更多最新详细信息，请阅读我们的事务层架构文档。

Abstract: ANSI SQL-92 [MS, ANSI] defines Isolation Levels in terms of phenomena: Dirty Reads, Non-Re- peatable Reads, and Phantoms. This paper shows that these phenomena and the ANSI SQL definitions fail to characterize several popular isolation levels, including the standard locking implementations of the levels. Investigating the ambiguities of the phenomena leads to clearer definitions; in addition new phenomena that better characterize isolation types are introduced. An important multiversion isolation type, Snapshot Isolation, is defined.

摘要：ANSI SQL-92 [MS, ANSI] 根据现象定义了隔离级别：脏读、不可重复读和幻像。 本文表明，这些现象和 ANSI SQL 定义无法表征几种流行的隔离级别，包括这些级别的标准锁定实现。 研究现象的模糊性可以得出更清晰的定义； 此外，还引入了更好地表征隔离类型的新现象。 定义了一种重要的多版本隔离类型：快照隔离。

The notion of consistency is used across different computer science disciplines from distributed systems to database systems to computer architecture. It turns out that consistency can mean quite different things across these disciplines, depending on who uses it and in what context it appears. We identify two broad types of consistency, state consistency and operation consistency*, which differ fundamentally in meaning and scope. We explain how these types map to the many examples of consistency in each discipline.*

一致性的概念用于不同的计算机科学学科，从分布式系统到数据库系统再到计算机体系结构。 事实证明，一致性在这些学科中可能意味着完全不同的事情，具体取决于谁使用它以及它出现在什么背景下。 我们确定了两大类一致性：状态一致性和操作一致性，它们在含义和范围上有根本的不同。 我们解释这些类型如何映射到每个学科中的许多一致性示例。

原文链接：https://jepsen.io/consistency

This clickable map (adapted from Bailis, Davidson, Fekete et al and Viotti & Vukolic) shows the relationships between common consistency models for concurrent systems. Arrows show the relationship between consistency models. For instance, strict serializable implies both serializability and linearizability, linearizability implies sequential consistency, and so on. Colors show how available each model is, for a distributed system on an asynchronous network.

这个可点击的地图（改编自 Bailis、Davidson、Fekete 等人和 Viotti & Vukolic）显示了并发系统的常见一致性模型之间的关系。 箭头显示一致性模型之间的关系。 例如，严格可串行化意味着可串行化和可线性化，可线性化意味着顺序一致性，等等。 颜色显示每个模型对于异步网络上的分布式系统的可用性。