Menu

Dipesh Majumdar

Blog and Paintings

Using awk to manage string output

August 13, 2014

====Part 1=====

For example you have a file like this -> awk_example1.txt

xyz_user@pqr_host:~/scripts> cat awk_example1.txt
output: one two 3 13:45 and there_are_so_many_things etc

As you see above in the above file 'awk_example1.txt' there is just one line and if we consider column separator to be space, then there are seven columns

1. one
2. two
3. 3
4. 13:45
5. and
6. there_are_so_many_things
7. etc

And if you consider the column separators to be colon (:) then there are two columns

1. one two 3 13
2. 45 and there_are_so_many_things etc

Now let's use awk, considering column separator to be space. We will use awk to print the 4th column.

xyz_user@pqr_host:~/scripts> awk -F' ' '{print $4}' awk_example1.txt
output: 13:45

NOw considering column separator to be column, let's print the 2nd column

xyz_user@pqr_host:~/scripts> awk -F':' '{print $2}' awk_example1.txt
output: 45 and there_are_so_many_things etc

Now where will this come handy? In other words where can you use this?

Well, if there is an output of a shell script for a process and which shows time like 13:45 and you want only the time to extract from the line, you can do that using the above awk command. There can be many other similar cases where it can be used.

====Part 2====

Now we are all set to perform an interesting little exercise that will help our confidence in using awk.
First do an ls -ltra over the awk file. Let me do that for you; here it goes

xyz_user@pqr_host:~/scripts> ls -ltra awk_example1.txt
-rw-r--r-- 1 xyz_user some_group 49 Aug 13 19:37 awk_example1.txt

As you see the output in one line above.

Now if we want to have the output as separate strings as Month(Mon), Day (dd), hour (hh) and minute (mm), can you do the same using awk.
I would urge you to do on your own before trying the solution below.

So here is the solution -


xyz_user@pqr_host:~/scripts> ls -ltra awk_example1.txt> awk_example1_part1.txt
xyz_user@pqr_host:~/scripts> awk -F':' '{print $1}' awk_example1_part1.txt > awk_example1_part1a.txt
xyz_user@pqr_host:~/scripts> awk -F':' '{print $2}' awk_example1_part1.txt > awk_example1_part1b.txt
xyz_user@pqr_host:~/scripts>
xyz_user@pqr_host:~/scripts> export awk_example1_part1_month=$(awk -F' ' '{print$6}' awk_example1_part1a.txt)
xyz_user@pqr_host:~/scripts> export awk_example1_part1_day=$(awk -F' ' '{print$7}' awk_example1_part1a.txt)
xyz_user@pqr_host:~/scripts> export awk_example1_part1_hour=$(awk -F' ' '{print$8}' awk_example1_part1a.txt)
xyz_user@pqr_host:~/scripts> export awk_example1_part1_min=$(awk -F' ' '{print$1}' awk_example1_part1b.txt)
xyz_user@pqr_host:~/scripts>
xyz_user@pqr_host:~/scripts>
xyz_user@pqr_host:~/scripts> echo $awk_example1_part1_month
Aug
xyz_user@pqr_host:~/scripts> echo $awk_example1_part1_day
13
xyz_user@pqr_host:~/scripts> echo $awk_example1_part1_hour
19
xyz_user@pqr_host:~/scripts> echo $awk_example1_part1_min
37

 

Now that was a lengthy process - a much better process is shown below - 

xyz_user@pqr_host:~/scripts> ls -ltra awk_example1.txt
-rw-r--r-- 1 xyz_user some_group 49 Mar  3 19:08 awk_example1.txt

xyz_user@pqr_host:~/scripts> ls -ltra awk_example1.txt |awk -F' ' '{print $7}'
3
xyz_user@pqr_host:~/scripts> ls -ltra awk_example1.txt |awk -F' ' '{print $8}'|awk -F':' '{print $1}'
19
xyz_user@pqr_host:~/scripts> ls -ltra awk_example1.txt |awk -F' ' '{print $8}'|awk -F':' '{print $2}'
08

How to attach a string with a grep output?

I tried to search this in Google but failed to find... This might help therefore who are searching for this exact requirement. Illustrating with an example - 

First create a test file - test.txt

echo -e "dip\ndipu\ndipesh\nsomething_else" > test.txt
cat test.txt
dipesh@hostname:~$ cat test.txt
dip
dipu
dipesh
something_else
dipesh@hostname:~$ grep -c "dip*" test.txt
3

what if i want a string in the output line - string should be like this: count of all words that start with dip - checked for hostname of my current server

Solution: 
i understand i have to take the help of awk
so i should try something like this ->
grep -c "dip*" test.txt |awk -F: '{print $1}'
ok... now i get the same result but i can add something more as a test...
grep -c "dip*" test.txt |awk -F: '{print $1" can add a string as a test"}'
so far so good... now i replace this string with this - 
grep -c "dip*" test.txt |awk -F: '{print $1" - For server: "}'
Works fine - but now last thing to do - i have to attach a variable to provide me hostname of the server where i am sitting...
grep -c "dip*" test.txt |awk -F: '{print $1" - For server: '$(hostname)'"}'
So here is the final output - 
dipesh@hostname:~$ grep -c "dip*" test.txt |awk -F: '{print $1" - For server: '$(hostname)'"}'
3 - For server: hostname

Works like Magic!!!

===========================================================================================================

 

Go Back

Comment