RAFT algorithm/protocol

 Based on notes made upon: http://thesecretlivesofdata.com/raft/

Consensus - agreement on a value:

1. non-distributed - DB client send data to single DB server
2. distributed - DB client send data to single DB server and this server must replicate data to the other DB servers

In Raft protocol node can be in 3 states:

1. follower
2. candidate
3. leader

Leader Election Process:

1. Each node begins in follower state and starts election timeout (random between 150ms-300ms)
2. If the follower doesn't hear from leader (during election timeout), it becomes candidate and starts new election term
3. The candidate votes for itself and sends Request-Vote message to the other nodes and starts election timeout
4. Other nodes reply with their vote if they didn't vote already in this term
5. If:
1. The candidate gets votes from the majority of nodes - it becomes leader and stops timeout
2. Election term is not properly held (no leader is elected) during election timeout (possibly with even number of nodes, when two nodes become candidate at the same time and can't collect majority of votes) - new election term is started because of election timeout
6. Majority of votes forces only one leader to be selected per term
7. All changes to the system now goes through the leader and heartbeat timeout is started on followers
8. Election term lasts until leader manages to send Append-Entry message during heartbeat timeout, if so - followers restart heartbeat timeout, otherwise new election term starts
9. If node goes down and then resumes, it firstly checks it's own election term number with the number of the election term in the Append-Entry message. If term number is equal or higher than it's own term number - current leader is accepted and all values are updated according to the new leader's values

Log Replication:

1. Each change is added as an uncommitted entry (value is not updated) to the leader's log
2. Leader sends replication to the followers
3. Leader waits until the majority of followers made uncommitted entry to their own log
4. Leader commits entry and value changes
5. Leader sends Append-Entry messages to the followers during heartbeat timeout (Append-Entry messages sent to the followers even no changes are really made - in order to keep-alive)
6. Followers commit entry and respond to each Append-Entry message