Linux tar incremental backup & restore

1. Backup

In this post I'll describe steps to use tar for incremental backup of files.

Incremental backups needed when it's lack of free space on your backup device so you have full-backup and incremental backup which contains changes/differences.

To make incremental backups tar uses `.snar` meta-files which contains changes since last backup. We'll use following tar options for backup (-vcSpzg):

1. -c OR --create - tells that we are creating new archive.
2. -v | --verbose - useful if you want log files.
3. -S OR --sparse - sparse files are files having holes (the section of the files' contents which was never written, the contents of a hole reads as all zeroes). Many file systems not actually allocating storage for hole but saving length of the hole so without that option tar archive will be longer than the original. Using this option makes tar to recognize sparse files and save only actual date. When restoring tar will add holes to such files at the needed places.
4. -p OR --preserve-permissions - we tell tar to save all permissions for every file.
5. -z OR --gzip - use compression.
6. -g OR --listed-incremental - show which meta-data file to use, this options show that we make backup and show's it's level. 
7. -f OR --file - backup file name.

The core concept is:

1. if snar file in `-g` option is not existing - make full backup and create snar file containing list of files backuped
2. if snar file exists - make incremental backup not backuping files listed in snar file

We'll have 7 files:

1.  for Monday - dir-dump1.sh
2.  for Tuesday - dir-dump2.sh
3.  for Wednesday - dir-dump3.sh
4.  for Thursday - dir-dump4.sh
5.  for Friday - dir-dump5.sh
6.  for Saturday - dir-dump6.sh
7.  for Sunday - dir-dump7.sh

So for example if we want to make full backup on Monday:
cat dir-dump1.sh
#! /bin/bash
#remember core concept (1) - we removing snar file for full-backup day
/bin/rm -f /mnt/backup/snar-dir/monday0.snar

#monday0.snar backup file will be created during full-backup
#$(date +%W) returns number of the week of the year
/bin/tar -cvSpzg /mnt/backup/snar-dir/monday0.snar -f /mnt/backup/$(date +%W)-monday.gz   /var/lib/recordings

For Tuesday we make incremental backup with file differing from monday backup:
cat dir-dump2.sh
#! /bin/bash
#core concept (2) - specified snar file will be used for incremental backup
/bin/cp -f /mnt/backup/snar-dir/monday0.snar /mnt/backup/snar-dir/tuesday.snar
/bin/tar -cvSpzg /mnt/backup/snar-dir/tuesday.snar -f /mnt/backup/$(date +%W)-tuesday.gz   /var/lib/recordings


For Wednesday, Thursday, Friday, Saturday & Sunday we make the same - renaming snar file of the previous day:

cat dir-dump3.sh
#! /bin/bash
/bin/mv -f /mnt/backup/snar-dir/tuesday.snar /mnt/backup/snar-dir/wednesday.snar
/bin/tar -cvSpzg /mnt/backup/snar-dir/wednesday.snar -f /mnt/backup/$(date +%W)-wednesday.gz   /var/lib/recordings

cat dir-dump4.sh
#! /bin/bash
/bin/cp -f /mnt/backup/snar-dir/wednesday.snar /mnt/backup/snar-dir/thursday.snar
/bin/tar -cvSpzg /mnt/backup/snar-dir/thursday.snar -f /mnt/backup/$(date +%W)-thursday.gz   /var/lib/recordings

cat dir-dump5.sh
#! /bin/bash
/bin/cp -f /mnt/backup/snar-dir/thursday.snar /mnt/backup/snar-dir/friday.snar
/bin/tar -cvSpzg /mnt/backup/snar-dir/friday.snar -f /mnt/backup/$(date +%W)-friday.gz   /var/lib/recordings

cat dir-dump6.sh
#! /bin/bash
/bin/cp -f /mnt/backup/snar-dir/friday.snar /mnt/backup/snar-dir/saturday.snar
/bin/tar -cvSpzg /mnt/backup/snar-dir/saturday.snar -f /mnt/backup/$(date +%W)-saturday.gz   /var/lib/recordings

cat dir-dump7.sh
#! /bin/bash
/bin/cp -f /mnt/backup/snar-dir/saturday.snar /mnt/backup/snar-dir/sunday.snar
/bin/tar -cvSpzg /mnt/backup/snar-dir/sunday.snar -f /mnt/backup/$(date +%W)-sunday.gz /var/lib/recordings

Then add backup for every day to the crontab (redirecting all errors and output to the log file):
0 5 * * 1 /mnt/backup/dir-dump1.sh 2>&1>> /mnt/backup/dir-dump.log
0 5 * * 2 /mnt/backup/dir-dump2.sh 2>&1>> /mnt/backup/dir-dump.log
0 5 * * 3 /mnt/backup/dir-dump3.sh 2>&1>> /mnt/backup/dir-dump.log
0 5 * * 4 /mnt/backup/dir-dump4.sh 2>&1>> /mnt/backup/dir-dump.log
0 5 * * 5 /mnt/backup/dir-dump5.sh 2>&1>> /mnt/backup/dir-dump.log
0 5 * * 6 /mnt/backup/dir-dump6.sh 2>&1>> /mnt/backup/dir-dump.log
0 5 * * 7 /mnt/backup/dir-dump7.sh 2>&1>> /mnt/backup/dir-dump.log

If you understood core concepts you can do any mix of full and incremental backups, using snar files.

 

2. Restore

Meta-files not needed for restoring. They needed only for the incremental backup process.  To restore we first restore last full backup and then all incremental backups. If use our backup scheme as example, restore will be as (i.e. we restore files for 10th week of the year). We'll ise `/` in `--directory` option because tar saves full-path and showing `/var/lib/recordings` will result in having `/var/lib/recordings/var/lib/recordings/file_name`. For snar file we specify `/dev/null` it is necessary because option `-g` must be specified in order to keep-track on operations over files during incremental backup (deletion, creation, changing) and snar file option can't be empty if you specify `-g` option:

/bin/tar --extract -vSpzg /dev/null -f /mnt/backup/10-monday.gz  --directory=/

/bin/tar --extract -vSpzg /dev/null -f /mnt/backup/10-tuesday.gz  --directory=/

/bin/tar --extract -vSpzg /dev/null -f /mnt/backup/10-wednesday.gz  --directory=/

/bin/tar --extract -vSpzg /dev/null -f /mnt/backup/10-thursday.gz  --directory=/

/bin/tar --extract -vSpzg /dev/null -f /mnt/backup/10-friday.gz  --directory=/

/bin/tar --extract -vSpzg /dev/null -f /mnt/backup/10-saturday.gz  --directory=/

/bin/tar --extract -vSpzg /dev/null -f /mnt/backup/10-sunday.gz  --directory=/

 

Time-Management & Productivity 1

Last month I realised how much time we loose doing things we don't need to do, trying to procrastinate (do anything but task we need to accomplish).  Many times I tried to begin using to-do lists, many different approaches  like Kanban, GTD, Eisenhower Box, The Pomodoro Technique etc.  But all this approaches failed for me (to speak frankly may be I'm too lazy and trying to find bad sides in everything). So I decided to find/set-up my own approach, based on other approaches learnt by me. I'll try to write new posts every time I used all things mentioned in the previous posts.

Lets start...
First of all you need to write-down (I'll use the Google Docs spreadsheets - for easiness of synchronisation across devices) all things you are planning to do. Write everything - you just want to empty your brain (as David Allen's GTD methodology suggests). Write even if it's not actually a todo-item, it may be just your thought on something - to the `backlog` (Kanban term) column.

Next make processing off items in this column (using Eisenhower Box) and make note in the `type` column (select B column then right click > Data validation...

• Cell range: > Add: 
• Sheet1!B2:B 
• Criteria > List of Items > Add:
• DoFirst,Schedule,Delegate,MayBeSomeDay,Done)

DoFirst (important and urgent) - must be done in day or two
Schedule (important but not urgent - things that you really need to do)  - write deadline in `note` column
Delegate (urgent but not important) - delegate this if possible (write name of person in `note` column)
MayBeSomeDay (not important and not urgent - at least for now) - if you can just delete this item from the backlog or it may be plan for future spare-time (i.e. make ASA config more readable - good but not too important or urgent to put it into schedule).
Done - mark all done items

Daily review your list and try to do all items with DoFirst and Schedule type, also check Delegate items. View MayBeSomeDay items when you have spare time.

PS I'll try to do things in this post for a period of one month and then do changes to this work-flow.

Update 1:
decided to use range for `Data validation...`
Created sheet `REF` with `DoFirst,Schedule,Delegate,MayBeSomeDay,Done` in an A column (each item in it's own cell).
Then removed data validation, and:
select B column then right click > Data validation...

• Cell range: > Add: 
• Sheet1!B2:B 
• Criteria > List from a range > Add:
• REF!A:A

Update 2:
have no more than 5 items in `DoFirst`, if you have more, then prioritise this items and move less important to the `Schedule`. Try to have as less items in `Schedule` as possible (move them to `MayBeSomeDay` if you can). When you full-fill `DoFirst` you can review `Schedule` items and choose most important to be new `DoFirst`. 

Update 3:
Added `DailyRoutine` to the `REF!A:A` for now I'll have only one routine - daily review of the items in the list.

Update 4:
Don't do daily review if you didn't do all `DoFirst` items. For now I have 3 `DailyRoutine`s (daily review, Amosov exercies and CCNA 100-105 cert prepare) and 1 `DoFirst` and 1 `WeeklyRoutine` (Choose DoFirst for weekedns and holidays and then take a photo of this. I do this because don't want to spend additional time on day-offs) item. This all helps me to avoid distraction

Update 5:
Added `PreSelected` to the `REF` using it for quickly selecting `DoFirst`s (first mark as `PreSelected` and then after doing current `DoFirst` selecting new one from `PreSelected`). Using conditional formatting to make easy understanding done and awaiting routines.

Cisco IOS (SLA / Track / default route change). Using alternative default route on fly.

Many branches were configured only with default route  of the `master` ASA and no other routes (so-called stub network). When migrating from old ASA to the new one I needed to make this migration `painless` (minimal downtime).

These are steps to change default routes on the routers:

#configure SLA:
#ping default-gateway

ip sla 1
 icmp-echo 172.16.0.8

exit

ip sla schedule 1 life forever start-time now

#configure tracker of this SLA:

track 1 ip sla 1 reachability

#configure main route with track number 1 :

ip route 0.0.0.0 0.0.0.0 172.16.0.8 track 1

#delete old default route:

no ip route 0.0.0.0 0.0.0.0 172.16.0.8

#add backup route with higher metric:

ip route 0.0.0.0 0.0.0.0 172.16.0.8 10

When  main route is unreachable, backup route with metric 10 will be used.

After finishing migration we can completely remove old route and add new route with default metric:

ip route 0.0.0.0 0.0.0.0 172.16.0.8

after issuing this command route with metric 10 is automatically removed.

Then on an old ASA deny all needed protocols for migrating router. In my environment:

access-list regions.acl line 1 extended deny esp host <needed-IP> any

access-list regions.acl line 1 extended deny ip host <needed-IP> any

access-list regions.acl line 1 extended deny icmp host <needed-IP> any

It's obvious that this is only one step of the migration process but other steps were more obvious for me than  this one.

Cisco IOS (SLA / Track / event manager). Restarting Interface on fail.

Recently it appeared that one of the branches I'm responsible for loses link on one of interfaces. I don't know why but when interface is `shutdown-noshutdown`-ed everything becomes OK. So I looked for a way to `restart` interface automatically:

#First we'll create SLA (Service Level Agreement)
#sla number is `1`
#it will ping `172.16.0.70` from `172.16.0.77` IP source address
#ping will be issued every 3sec
#threshold must be < timeout
#threshold is RTT (Round Trip Time - time needed to reach the other side and to return) in msec - needed for SLA Statistics and Monitoring
#timeout - if no answer in 1.5sec - consider it as failed attempt
ip sla 1
 icmp-echo 172.16.0.70 source-ip 172.16.0.77
 threshold 1000
 timeout 1500
 frequency 3
exit
#Then we'll make it endless and start it right now:
ip sla schedule 1 life forever start-time now
#show commands fro SLA:
sh ip sla configuration  1
sh ip sla statistics 1

#Now we'll configure tracker of the SLA
# track number 10
#uses SLA number 1
#consider down if down for 10sec
#consider up if up for 60sec
track 10 ip sla 1 reachability
 delay down 10 up 60
#to view:
sh track brief
sh track 10

#Configure event manager (response on event)
#event manager named restart_interface
#will use track number 10
#restart_interface will be executed if track 10 state is down
event manager applet restart_interface
 event track 10 state down
 action 0   cli command "enable"
 action 1   cli command "conf t"
 action 2   cli command "int fa0/1"
 action 2.5 cli command "shut"
 action 3   cli command "no shut"
 action 4   cli command "end"
#to view:
sh even manager policy registered
sh event manager history events

MySQL files extensions

grep data  /etc/my.cnf
datadir=/var/lib/mysql

We can finf below files in the `/var/lib/mysql/<db_name>`:

db.opt
this file contains characteristics of database like so:
default-character-set=latin1
default-collation=latin1_swedish_ci
every MyISAM table will have frm / MYD /  MYI and TRG (if this table has trigger)
.frm :
MySQL represents each table by an .frm table format file, stores table definition in the database directory. It is stored as  .frm under data directory.

.MYD :
This is the extension of the Data files for MyISAM tables.

.MYI :
This is the extension of the Index files for MyISAM tables.

<table_name>.TRG
contains SQL script of trigger for this table

<trigger_name>.TRN
contains a table name this trigger belongs to

.ibd :
If you specify innodb_file_per_table option to my.cnf, InnoDB stores each table in its own .ibd file in the same MySQL database directory where the .frm file is created. For InnoDB tables, the data and the indexes are stored together in the .ibd file.
 

MySQL root password recovery

You can recover MySQL database server root password following below six easy steps:

1. Stop the MySQL server process:
1. /etc/init.d/mysqld stop
2.  Start the MySQL (mysqld) server/daemon process with the --skip-grant-tables option so that it will not prompt for password:
1.  mysqld_safe --skip-grant-tables &
2.  /etc/init.d/mysqld start
3.  Connect to mysql server as the root user:
1.  mysql -u root
4.  Setup new mysql root account password i.e. reset mysql password:
1. mysql> use mysql;
2. Change root password (read all 3 below before issuing commands):
1. MySQL 5.7.6 and later: ALTER USER 'root'@'localhost' IDENTIFIED BY 'MyNewPass';
2. MySQL 5.7.5 and earlier: SET PASSWORD FOR 'root'@'localhost' = PASSWORD('MyNewPass'); 
3. Or for any MySQL version: update user set Password=PASSWORD('MyNewPass') where User='root';
3. mysql> flush privileges;
4. mysql> quit 
5. Exit and restart the MySQL server:
1.  /etc/init.d/mysqld restart
6.  Connect to MySQL and verify if new root password is accepted:
1. mysql -u root -p

MySQL Slow connection

When connection to MySQL server is too slow it may be because of name resolving (at least attempt to resolve names). To change this behaviour:

edit my.cnf:

Add this line under the `[mysqld]` section:

skip-name-resolve

And then restart mysqld daemon