etcd raft library设计原理和使用
早在2013年11月份，在raft论文还只能在网上下载到草稿版时，我曾经写过一篇blog对其进行简要分析。4年过去了，各种raft协议的讲解铺天盖地，raft也确实得到了广泛的应用。其中最知名的应用莫过于etcd。etcd将raft协议本身实现为一个library，位于https://github.com/coreos/etcd/tree/master/raft，然后本身作为一个应用使用它。本文不讲解raft协议核心内容，而是站在一个etcd raft library使用者的角度，讲解要用上这个library需要了解的东西。
这个library使用起来相对来说还是有点麻烦。官方有一个使用示例在 https://github.com/coreos/etcd/tree/master/contrib/raftexample。整体来说，这个库实现了raft协议核心的内容，比如append log的逻辑，选主逻辑，snapshot，成员变更等逻辑。需要明确的是：library没有实现消息的网络传输和接收，库只会把一些待发送的消息保存在内存中，用户自定义的网络传输层取出消息并发送出去，并且在网络接收端，需要调一个library的函数，用于将收到的消息传入library，后面会详细说明。同时，library定义了一个Storage接口，需要library的使用者自行实现。
Storage接口如下:
// Storage is an interface that may be implemented by the application// to retrieve log entries from storage.//// If any Storage method returns an error, the raft instance will// become inoperable and refuse to participate in elections; the// application is responsible for cleanup and recovery in this case.type Storage interface { // InitialState returns the saved HardState and ConfState information.
InitialState() (pb.HardState, pb.ConfState, error) // Entries returns a slice of log entries in the range [lo,hi).
// MaxSize limits the total size of the log entries returned, but
// Entries returns at least one entry if any.
Entries(lo, hi, maxSize uint64) ([]pb.Entry, error) // Term returns the term of entry i, which must be in the range
// . The term of the entry before
// FirstIndex is retained for matching purposes even though the
// rest of that entry may not be available.
Term(i uint64) (uint64, error) // LastIndex returns the index of the last entry in the log.
LastIndex() (uint64, error) // FirstIndex returns the index of the first log entry that is
// possibly available via Entries (older entries have been incorporated
// into the latest Snapshot; if storage only contains the dummy entry the
// first log entry is not available).
FirstIndex() (uint64, error) // Snapshot returns the most recent snapshot.
// If snapshot is temporarily unavailable, it should return ErrSnapshotTemporarilyUnavailable,
// so raft state machine could know that Storage needs some time to prepare
// snapshot and call Snapshot later.
Snapshot() (pb.Snapshot, error)
}这些接口在library中会被用到。熟悉raft协议的人不难理解。上面提到的官方示例https://github.com/coreos/etcd/tree/master/contrib/raftexample中使用了library自带的MemoryStorage，和etcd的wal和snap包做持久化，重启的时候从wal和snap中获取日志恢复MemoryStorage。
要提供这种IO/网络密集型的东西，提高吞吐最好的手段就是batch加批处理了。etcd raft library正是这么做的。
下面看一下为了做这事，etcd提供的核心抽象Ready结构体：
// Ready encapsulates the entries and messages that are ready to read,// be saved to stable storage, committed or sent to other peers.// All fields in Ready are read-only.type Ready struct { // The current volatile state of a Node.
// SoftState will be nil if there is no update.
// It is not required to consume or store SoftState.
*SoftState // The current state of a Node to be saved to stable storage BEFORE
// Messages are sent.
// HardState will be equal to empty state if there is no update.
pb.HardState // ReadStates can be used for node to serve linearizable read requests locally
// when its applied index is greater than the index in ReadState.
// Note that the readState will be returned when raft receives msgReadIndex.
// The returned is only valid for the request that requested to read.
ReadStates []ReadState // Entries specifies entries to be saved to stable storage BEFORE
// Messages are sent.
Entries []pb.Entry // Snapshot specifies the snapshot to be saved to stable storage.
Snapshot pb.Snapshot // CommittedEntries specifies entries to be committed to a
// store/state-machine. These have previously been committed to stable
// store.
CommittedEntries []pb.Entry // Messages specifies outbound messages to be sent AFTER Entries are
// committed to stable storage.
// If it contains a MsgSnap message, the application MUST report back to raft
// when the snapshot has been received or has failed by calling ReportSnapshot.
Messages []pb.Message // MustSync indicates whether the HardState and Entries must be synchronously
// written to disk or if an asynchronous write is permissible.
MustSync bool}可以说，这个Ready结构体封装了一批更新，这些更新包括：
[*] pb.HardState: 包含当前节点见过的最大的term，以及在这个term给谁投过票，已经当前节点知道的commit index
[*] Messages: 需要广播给所有peers的消息
[*] CommittedEntries:已经commit了，还没有apply到状态机的日志
[*] Snapshot:需要持久化的快照
库的使用者从node结构体提供的一个ready channel中不断的pop出一个个的Ready进行处理，库使用者通过如下方法拿到Ready channel：
func (n *node) Ready()
页:
[1]