DHCP on Docker

DHCP (Dynamic Host Configuration Protocol) helps us to address dynamically our hosts on the network. In fact, when a Host is configured to get its IP address dynamically, it will broadcast a DHCP REQUEST on the network searching for a DHCP server. DHCP server has to be on the same broadcast domain as the CLIENTS since routers do not forward broadcast packets.

1. create macvlan network:
1. docker network create -d macvlan -o parent=enp3s0 --subnet 172.16.3.0/24 --gateway 172.16.3.4  --aux-address 'host=172.16.3.250' mynet
2. add macvlan-aux to the docker-host (to ping directly from docker-host) - including ip link, route etc:
1. ip link add mynet-aux link enp3s0 type macvlan mode bridge
2. ip addr add 172.16.3.250/32 dev mynet-aux 
3. run container with macvlan driver (assign static IP) and run /bin/bash:
1. docker run --name='ctr0' --hostname='ctr0' --net=mynet --ip=172.16.3.249 -it centos /bin/bash
4. Ping container IP from the Docker-host:
1. ping 172.16.3.249
5. on container (all of these can be done with docker-file):
1. yum install net-tools -y
2. yum install dhcp -y
3. 1.1.6.1 is DHCP relay IP address
4. dhcpd listens *only* on interfaces for which it finds subnet declaration in dhcpd.conf

vi dhcp.conf:

# this server is primary and thus - authorative server on that network

authoritative;
subnet 172.16.3.0 netmask 255.255.255.0 {
           range 172.16.3.1 172.16.3.3;
           option routers 172.16.3.4;
           option domain-name-servers 172.16.3.6;
}     

Run dhcp service with specified file: dhcpd -cf dhcp.conf

to kill process on container:

top > k > PID > Enter

YAML

YAML Ain't Markup Language:

1. YAML uses indentation to distinguish layers (same indentation - same layer) - use spaces, because tabs are not allowed.
2. YAML start with three dashes: ---
3. YAML disctionary in key-value pair in one of two forms:
1. colon separated key (like Python dictionary):
1. key: value
2. indentation separated key:
1. key:
2.       value
3. Also one key can contain nested dictionary:
1. Method 1:
1. first_level_key:
2.    second_level_key_under_the_first_level: second_level_value
2. Method 2:
1. first_level_key: {second_level_key_under_the_first_level: second_level_value}
4. YAML uses dashes as indentation to represent list of items (use lists when you want key to have more than one values which are not keys themselves):
1. Methos 1:
1. this_is_a_list:
2.  - element_1
3.  - element_2
2. Method 2:
1. this_is_a_list : [element_1, elemenet_2]

Relational Algebra

RA is a formal language that forms conventions used in implemented languages like SQL.
RA operates on relations and produces relations as result.

Query (expression) on set of relations produces relation as result. 

Key is an attribute or a set of attributes whose value is guaranteed to be unique.
Tuple  - row of a relation.
Attribute  - column of a relation.
Relation - table consisting of attribute (column) and tuples (rows).
Schema  - relation header (attribute names)

We'll use simple college admission database with three relations (keys are in bold):

1. 1st relation: College(schema: cName, state, enrollment)
2. 2nd relation: Student(schema: sID, sName, GPA, sizeHS) # sizeHS = size of High School a student attended
3. 3rd relation: Apply(schema: sID, cName, major, decision)

Simplest query in a RA is simply the relation name, for example Student is valid expression (query) in RA and it's returning copy of Student relation.

Use operators to filter, slice, combine relations:

1. Select - picks certain rows out of a relation - σ (sigma):
1. Students with GPA > 3.7 (GPA - Grade Point Average)
1. σ _{GPA > 3.7} Student
2. Students with GPA > 3.7 and sizeHS < 1000 (caret ^ is logical and operator)
1. σ _{GPA > 3.7  ^ sizeHS < 1000} Student
3. Application for Stanford for CS major
1. σ _{cName="Stanford" ^ major="CS"} Apply
4. Select operator general case:
1. σ _condition Relation
2. Project operator - picks certain columns:
1. Apply relation with sID and decision columns only
1. П _sID,decision Apply
2. Project operator general case:
1. П _{col1,col2,col3...} Relation
3. To Select and Project at the same time:
1. ID and name of students with GPA>3.7:
1. П _sID,sName (σ _GPA>3.7 Student)
4. Duplicates:
1. SQL is based on multisets/bags, so duplicates are also shown
2. RA is based on sets, so duplicates are eliminated automatically
5. Cross-product operator (a.k.a. cross-join) - combine two relations (a.k.a. Cartesian product) horizontally:
1. Student x Apply as result we get big relation which going to have eight (8) attributes
2. as a convention when cross-product is done and we get the same attributes for both cross-producted relations we preface their name with the name of the relation they came from: Student.sID / Apply.sID
3. cross-product gives as relation where for every row of Student relation you get all rows of Apply relation
4. Names and GPA's of students with HS>1000 who applied to CS and were rejected:
1.  П _{sName, GPA} (σ _{Student.sID=Apply.sID ^ HS>1000 ^ major="CS" ^ decision="Reject"} (Student x Apply))
6. Difference operator A\B or A-B returns elements of A that not in B:
1. if A = {a, b, c} and B = {b, c, d} then A - B = {a}
2. IDs of students who didn't apply anywhere:
1. (П _sID Student) - (П _sID Apply)
3. IDs and names of students who didn't apply anywhere (we can't just add sName to the project from Student relation because Apply relation has not this attribute and we might not use difference operator on sets with different quantity of attributes):
1. П _sID,sName ( ( (П _sID Student) - (П _sID Apply) ) ⋈ Student )
7. Union (denoted by ∪) operator is the set of all elements in the collection. It is one of the fundamental operations through which sets can be combined and related to each other. Union combines vertically:
1. A ∪ B means all members of A and also all members of B that not in A (RA is set based and removes duplicates):
1. if A = {a, b, c} and B = {b, c, d} then:
1. all members of A are {a, b, c} 
2. all members of B that not in A is B\A = {d}
3. combine both {a, b, c} and {d} => {a, b, c, d}
4. so: A ∪ B = {a, b, c, d}
2. List of college and student names:
1. (П _cName College) ∪ (П _sName Student)
8. Natural join operator performs cross-product operator and then enforces equality on all of the attributes with the same name (as in above example: Student.sID=Apply.sID) also natural join eliminates one copy of duplicate attributes:
1. Names and GPA's of students with HS>1000 who applied to CS and were rejected:
1. П _{sName, GPA} (σ _{HS>1000 ^ major="CS" ^ decision="Reject"}(Student ⋈ Apply))
2. Names and GPA's of students with HS>1000 who applied to CS and were rejected to colleges with the enrollment greater than 20000:
1. П _{sName, GPA} (σ _{HS>1000 ^ major="CS" ^ decision="Reject" ^ enrollment>20000}(Student ⋈ (Apply ⋈ College)))
3. Relation between ⋈ and x:
1. E1 ⋈ E2 => П _{schema(E1) U schema(E2)} (σ _{E1.a1= E2.a1 ^ E1.a2= E2.a2 ^ ...} (E1 x E2))
9. Theta Join - operator takes two expressions/relations and combines them with bow tie looking operator (⋈) but with a subscript theta (_θ) which means select condition (any select condition you want):
1. E1 ⋈_θE2 = σ_θ (E1 ⋈ E2 )
2. Theta join is the basic operation implemented in RDBMS so the term "join" often means theta-join
10. Intersection operator -  A ∩ B of two sets A and B is the set that contains all elements of A that also belong to B (or equivalently, all elements of B that also belong to A), but no other elements:
1. if A = {1, 2, 3} and B = {2, 3, 4} then A ∩ B = {2, 3}
2. Names that are both college name and student name:
1. (П _sName Student) ∩ (П _cName College)
3. Expressing intersection via difference
1. A ∩ B => A - ( A - B ) 
4. Expressing intersection via natural join:
1. A ∩ B => A ⋈ B
11. Rename operator uses ρ (rho), it reassigns schema in the result of expression (relation). Above (in 7.2.1 and 10.2.1 we used operators on relations with different attribute names - different schemas, in practice RA doesn't allow that, so we need to use rename operator):
1. General form:
1. ρ _{R(A1,A2,...An)} Relation E # call result of the relation E - R with attributes A1 to An
2. Unify schemas for set operators:
1. List of college and student names (7.2.1):
1. ρ _C1(name) (П _cName College) ∪ ρ _C2(name) (П _sName Student)
2. Names that are both college name and student name (10.2.1):
1. ρ _C1(name) (П _sName Student) ∩ ρ _C2(name) (П _cName College)
3. for disambiguation in self-joins:
1. pairs of colleges in same state:
1. With cross-join
1. σ _{s1=s2 ^ n1!=n2} ( ρ _C1(n1,s1,e1) College x ρ _C2(n2,s2,e2) College )
2. With natural-join:
1. σ _n1!=n2 ( ρ _C1(n1,s,e1) College x ρ _C2(n2,s,e2) College )
12. Select, Project, Cross-join, union, difference, rename are RA basic operators
13. Natural join, theta join, intersection are not RA basic operators they can be expressed with use of basic operators, so there are actually are abbreviations

These materials were used to write this synopsis-post:

1. https://www.youtube.com/watch?v=tii7xcFilOA
2. https://www.youtube.com/watch?v=GkBf2dZAES0

Hash Functions, Binary Tree, O(n)

Hash Function

A hash function is any function that can be used to map data of arbitrary size to data of fixed size (N).  Example of simple hash function is h(x) = x mod N (mod is modulus or remainder of division, for example 5 / 5 = 1 with no reminder => mod=0, 6 / 5 = 1 and 1 in remainder => mod = 1)The values returned by a hash function are called hash values, hash codes, digests, or simply hashes. Hash functions are often used in combination with a hash table (consists of hash function h and array, also called table, of size N), a common data structure used in computer software for rapid data lookup. Hashing is done for indexing and locating items in databases because it is easier to find the shorter hash value than the longer string. Hashing is also used in encryption.This term is also known as a hashing algorithm or message digest function. No sorting and no searching required. When you compute the hash function you know where to store the data and you know where to find the data. Hash functions are just one-way they cannot be reversed. The main idea is to store key-element pairs (k, e) as index h(k):
Example:

1. phone book with 5 numbers in it (N = 5)
2. h (name) = (lenght of name) mod 5
3. This function is ok if all names in phonebook have different lengths, if some lengths are the same, then collision is occurred (collision - when pairs of input to hash function are mapped to the same hash value): 
1. h(John) = 3 mod 5 = 3
2. h(Jack) = 3 mod 5
3. So h(John) = h(Jack)
4. so because of many collision this is example of the bad hash-function
5. actually collision can be in every hash function but hash function must be designed in the way minimizing collision possibility. To do so hash functions produce long enough hash-values and this values are hold smaller enough to be computed quickly.

Binary Tree

In computer science, a binary tree is a tree data structure in which each node (узел) has at most two children, which are referred to as the left child and the right child. Topmost node called root and this is L-0 (level zero) and height of 0. Each child node in binary tree defines a sub-tree, the root of which it is.

Big O notation

In computer science, big O notation is used to classify algorithms according to how their running time or space requirements grow as the input size grows.Actual formula is O(f(n)) meaning: with an increase in the parameter n (amount of input to the algorithm) the running time of the algorithm will increase no faster than some constant multiplied by f(n). How to find big-O of some operation (as Example 3n^2+ n^5 + 4n + 5 + 2^n + log8(n) ):

1. omit constants and constant multipliers (3n^2+ n^5 + 4n + 5 + 2^n + log8(n) => n^2+ n^5 + n + 2^n + log8(n))
2. n^a grows faster than n^b for a > b. In other words if you have n^3 - omit n^2 (n^2+ n^5 + n + 2^n + log8(n) => n^5 + 2^n + log8(n) )
3. any polynomial grows faster than any logarithm, so n or even sqrt(n), grows faster than log3(n)  ( n^5 + 2^n + log8(n) => n^5 + 2^n )
4. any exponential grows faster than any polynomial, so 3^n grows faster than n^5 ( n^5 + 2^n  => 2^n )
5. So O(3n^2+ n^5 + 4n + 5 + 2^n + log8(n)) = 2^n
6. All of the above doesn't mean that nobody cares constant - in practice speeding-up algorithm twice can be very hard but efficient, but it's much more reasonable to find approximate values first

Draft: Relax-and-Recover

Backup to USB and restore from USB

sudo yum install git syslinux syslinux-extlinux kernel-devel

git clone https://github.com/rear/rear.git

cd rear/

insert USB stick to the backed-up computer

lsblk # to find name of the USB flash card

umount /dev/sdb1 # umount if USB flash is automatically mounted

sudo usr/sbin/rear format /dev/sdb

type 'Yes' to format USB flash

rear will format that flash as REAR-000

edit rear configuration:

vi etc/rear/local.conf

### write the rescue initramfs to USB and update the USB bootloader

OUTPUT=USB

### create a backup using the internal NETFS method, using 'tar'

BACKUP=NETFS

### write both rescue image and backup to the device labeled REAR-000

BACKUP_URL=usb:///dev/disk/by-label/REAR-000

Create rescue image  (it's without OS backup and used to restore OS in case of failure) with verbose output:

sudo usr/sbin/rear -v mkrescue
Now reboot your system and try to boot from the USB device. If it's ok, then rescue image is ok and you can do OS data backup alonh with creating rescue media:
sudo usr/sbin/rear -v mkbackup

Access CentOS from the other machine (via VNC)

On target machine

yum groupinstall "GNOME Desktop"

Set default target to GUI:

systemctl set-default graphical.target

yum install tigervnc-server xorg-x11-fonts-Type1

Choose a VNC display to use for accessing this particular computer (here we'll use display 5 which equals to the port 5905)

cp /lib/systemd/system/vncserver@.service /etc/systemd/system/vncserver@:5.service

Add user (if you need one) whose Desktop will be accessed:

adduser admin

passwd admin

Access file:

vi /etc/systemd/system/vncserver@:5.service

Then:

1. Go to [Service]
2. ExecStart  - here - change <USER> with admin
3. PIDFile - here - change <USER> with admin

firewall-cmd --permanent --zone=public --add-port=5905/tcp 

firewall-cmd --reload

su - admin

Execute vncserver command to assign VNC password for needed user (admin in our case)

systemctl daemon-reload 

systemctl start vncserver@:5.service

systemctl enable vncserver@:5.service

systemctl status vncserver@:5.service
Check that vncserver is listening to the ports other that localhost:
lsof -i -P | grep -i "listen" | grep Xvnc

If some kind of problem appears do below:

Remove not needed VNC display sessions (if any):

rm -f /tmp/.X11-unix/*

rm -f /tmp/.X*-lock

systemctl restart vncserver@:5.service

ps aux | grep vnc

On client machine

yum install tigervnc

Open TigerVNC Viewer (Applications > Internet) or vncviewer (from bash)

Specify serverNameOrIp:5095
Specify VNC password
then login using needed login and password (admin in our case)

Use F8 to make full-screen and to change other settings.

Copying multiple files with scp

You can copy all files in directory but problem is when you are trying to copy several files from huge directory. Below is the method I use to achieve needed:

1. save list of names of needed files with their paths into the test file (ex: scpList.txt) - full path is needed!
2. assign this list to the variable: scpList=$(echo \";for file in $(cat scpLoad.txt); do printf $file ; printf " "; done; echo \")
3. manually copy variable content: echo $scpList
4. mkdir ~/Desktop/scpFiles
5. scp root@srvIP:insertCopiedVariableContentHere ~/Desktop/scpFiles