Tuesday, March 7, 2017

Sorting and Counting output in Linux shell

Sometimes we need to make some operations on commands output in Linux. Very intensive operation on output I do is sorting and counting. For the sake of an example - every time I setup a new equipment I make backup and save it in `back` directory, with the file-names like:
ls -1 back  |  head -n3
101-back-2017-02-21
102-back-2017-03-01
106-back-2017-02-21
Where numbers at the starting of the names of the backups are branch codes.
When I want to learn how many equipment was installed I can simply do:
ls back | wc -l
98
But this is total number of all configured equipment. what if I want to find how much equipment I setup every month?
Let's go:
  1. we need to list all files in `back` directory along with their creation times:
    1. ll back/ 
  2. now we need to select only month (6 column) and print it:
    1. ll back/ | awk '{print $6}'
  3. now we need to sort output alphabetically:
    1. ll back/ | awk '{print $6}' | sort
  4. we needed sorting because uniq command filtering matching adjacent lines and then making operations according to specified options:
    1. ll back/ | awk '{print $6}' | sort | uniq

      1. Feb
      2. Mar
    2. we'll use `-c` which makes `uniq` count all occurrences of unique values  (Mar and Feb in our case) in the output:
      1. ll back/ | awk '{print $6}' | sort | uniq -c
        1. 1
        2. 60 Feb
        3. 38 Mar
      2. You see line which count is `1` this is because of the `ll` commands `total` first line, to avoid this you can use:
        1.  ll back/ | grep -v total | awk '{print $6}' | sort | uniq -c
  5. If you need to view results per day of the month additional use 7th column (use "-" to see `-` as month and day of the month delimiter):
    1. Sorting using default `sort` settings
    2. ll back/ | grep -v total | awk '{print $6 "-" $7}' | sort | uniq -c
      1. 1 Feb-20
      2. 59 Feb-21
      3. 23 Mar-1
      4. 9 Mar-2
      5. 6 Mar-7
    3. Soring using day of the month and numeric sorting with `-` delimiter (to sort in reverse order, use `-r` option > '-k2nr' instead of `-k2n`):
    4. ll back/ | grep -v total | awk '{print $6 "-" $7}' | sort -t"-" -k2n | uniq -c
      1. 23 Mar-1
      2. 9 Mar-2
      3. 6 Mar-7
      4. 1 Feb-20
      5. 59 Feb-21
     


No comments:

Post a Comment