Saturday, July 17, 2021

RAFT algorithm/protocol

 Based on notes made upon: http://thesecretlivesofdata.com/raft/

Consensus - agreement on a value:

  1. non-distributed - DB client send data to single DB server
  2. distributed - DB client send data to single DB server and this server must replicate data to the other DB servers

In Raft protocol node can be in 3 states:

  1. follower
  2. candidate
  3. leader
Leader Election Process:
  1. Each node begins in follower state and starts election timeout (random between 150ms-300ms)
  2. If the follower doesn't hear from leader (during election timeout), it becomes candidate and starts new election term
  3. The candidate votes for itself and sends Request-Vote message to the other nodes and starts election timeout
  4. Other nodes reply with their vote if they didn't vote already in this term
  5. If:
    1. The candidate gets votes from the majority of nodes - it becomes leader and stops timeout
    2. Election term is not properly held (no leader is elected) during election timeout (possibly with even number of nodes, when two nodes become candidate at the same time and can't collect majority of votes) - new election term is started because of election timeout
  6. Majority of votes forces only one leader to be selected per term
  7. All changes to the system now goes through the leader and heartbeat timeout is started on followers
  8. Election term lasts until leader manages to send Append-Entry message during heartbeat timeout, if so - followers restart heartbeat timeout, otherwise new election term starts
  9. If node goes down and then resumes, it firstly checks it's own election term number with the number of the election term in the Append-Entry message. If term number is equal or higher than it's own term number - current leader is accepted and all values are updated according to the new leader's values

Log Replication:
  1. Each change is added as an uncommitted entry (value is not updated) to the leader's log
  2. Leader sends replication to the followers
  3. Leader waits until the majority of followers made uncommitted entry to their own log
  4. Leader commits entry and value changes
  5. Leader sends Append-Entry messages to the followers during heartbeat timeout (Append-Entry messages sent to the followers even no changes are really made - in order to keep-alive)
  6. Followers commit entry and respond to each Append-Entry message



    Thursday, July 1, 2021

    Linux file/directory permissions and extended permissions/ACL (getfacl/setfacl)

     [admino@mwg www]$ getfacl SystemFiles/

    # file: SystemFiles/

    # owner: www-data

    # group: www-data

    user::rwx

    group::rwx

    other::r-x


    #Give access to admino user:

    #-m is modify ACL (add or change)

    #-x is delete ACL (setfacl -x u:sales:r test/)

    #-d is to put a 'default' user and/or group to new generated files (can be given only to directory).

    # u: / g: / o: / m: / d: are user, group, others, effective mask, default (the same as -d)

    #

    #read (r or 4) – read file; list directory content

    #write (w or 2) – modify file; if x is also set - modify dir contents (rm, cp, mv files and subdirectories), otherwise - no effect

    #execute (x or 1) – execute file, enter directory

    #The setuid (s instead of u:x) bit simply indicates that when running the executable, it will set its permissions to that of the user who created it (owner), instead of setting it to the user who launched it. Similarly, there is a setgid (s instead of g:x) bit which does the same for the gid. (-rwsr-xr-x. 1 root root 27856 Apr  1  2020 /usr/bin/passwd).

    #When a directory has the sticky (t instead of o:x) bit set, its files can be deleted or renamed only by the file owner, directory owner and the root user (drwxrwxrwt.   6 root root  4096 Jul  1 09:59 tmp). 

    #ls shows + sign after permissions to indicate extended ACL (like: drwxr-xr-x+) s


    ###Remove all existing extended ACL:

    sudo setfacl -b /var/www/SystemFiles

    ###Get list of extended and standard ACL:

    getfacl /var/www/SystemFiles

    getfacl: Removing leading '/' from absolute path names

    # file: var/www/SystemFiles

    # owner: www-data

    # group: www-data

    user::rwx

    group::rwx

    other::r-x

    ###give admino ability to upload files:

    sudo setfacl -m u:admino:rwx /var/www/SystemFiles

    ###

    getfacl: Removing leading '/' from absolute path names

    # file: var/www/SystemFiles

    # owner: www-data

    # group: www-data

    user::rwx

    user:admino:rwx

    group::rwx

    mask::rwx

    ###to read non-default ACL:

    getfacl SystemFiles | grep -Ev "(::)|^#|^$"

    ###set defaults for user and group:

    sudo setfacl -m d:u:www-data:rw SystemFiles

    sudo setfacl -m d:g:www-data:rw SystemFiles

    ###check ACL (effective means group::rwx applies mask::rw and we have union - effective:rw):

    [admino@mwg www]$ getfacl SystemFiles/u34jf\ o.png

    # file: SystemFiles/math.png

    # owner: admino

    # group: admino

    user::rw-

    user:www-data:rw-

    group::rwx                      #effective:rw-

    group:www-data:rw-

    mask::rw-

    other::r--