Code

That’s Awksome!

0

Have you ever been doing something on the terminal and realise that most of the information thats being reported is useless to you? Ever thought, “Hey it would be really cool if it would only show me ‘just that bit’”?

Awk could be the answer. Awk is a DSL or Domain Specific Language aimed at text processing, which is used by many command line guru’s. Although awk has an extremely large amount of uses, I will only be detailing its print statement in use with a pipe.

The awk print statement is used, obviously, to print things, not that spectacular but the real magic comes with awk’s variables. When you pass a stream to awk it counts the number of columns, or parts of the stream separated by the input field separator, which by default is whitespace, it then assigns each section a variable $num. So in the following line

This is a line.

$1, $2, $3, $4 = This, is, a, line.

So you may be able to see how it is useful, but you may not be able to see how practical this is.

Lets do an example run using ls. If I run

ls -lh

in a terminal I get the following output:

drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 Angus,Thongs.And.Perfect.Snogging[2008]DvDrip-aXXo
-rw-r--r-- 1 bod bod 149M 2009-06-22 07:50 archlinux-2009.02-ftp-i686.iso
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 Bend It Like Beckham (2002) DVDRip XviD [desidhamal.com]
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 Bride Wars[2009]DvDrip[Eng]-FXG
drwxr-xr-x 2 bod bod 4.0K 2009-06-23 13:46 Desktop
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 downloads
drwxr-xr-x 6 bod bod 4.0K 2009-06-22 07:46 Hes.Just.Not.That.Into.You.DVDRip.XviD-DiAMOND
drwxr-xr-x 3 bod bod 4.0K 2009-06-22 07:46 Hitch[2005]DVDrip[Eng]AC3[5.1]-Atlas47
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 Ice Road Truckers.S 03e02.rookie.run
-rw-r--r-- 1 bod bod 348M 2009-06-22 08:17 Ice.Road.Truckers.S03E03.Canadian.Invasion.DSR.XviD-KRS.avi
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 livecd-i686-installer-2008.0-r1
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 lose a guy torrent
-rw-r--r-- 1 bod bod  303 2009-05-29 19:03 not_found.html
-rw-r--r-- 1 bod bod  303 2009-05-29 19:03 not_found.html.1
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 Ocean's.12[2004]DvDrip[Eng]-aXXo
drwxr-xr-x 2 bod bod 4.0K 2009-06-23 08:46 perl
drwxr-xr-x 2 bod bod 4.0K 2009-06-25 17:51 python
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:52 Relapse
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 Spore-RELOADED
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 The.Accidental.Husband[2008]DvDrip-aXXo
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 The.Illusionist[2006]DvDrip[Eng]-aXXo
-rw-r--r-- 1 bod bod 2.4K 2008-12-11 21:54 t_skariah.asc.gpg
-rw-r--r-- 1 bod bod 3.0K 2009-06-23 13:59 tuto.txt
-rw-r--r-- 1 bod bod  575 2009-06-23 08:45 tuto.txt~
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 Underworld Rise of the Lycans[2009]DvDrip[Eng]-FXG
drwxr-xr-x 2 bod bod 4.0K 2009-06-22 07:46 Yes.Man.2008.DvDRip-FxM

Now lets say, I only wanted to know about the name and the permissions. I could use awk to only give me those sections. Note: The use of the ls flag –quoting-style=locale makes ls quote filenames meaning awk will print filenames with spaces as one column instead of splitting them.

ls -lh --quoting-style=locale | grep -v "total " | awk '{print $1 "\t" $8}'

That would output this:

drwxr-xr-x	`Angus,Thongs.And.Perfect.Snogging[2008]DvDrip-aXXo'
-rw-r--r--	`archlinux-2009.02-ftp-i686.iso'
drwxr-xr-x	`Bend
drwxr-xr-x	`Bride
drwxr-xr-x	`Desktop'
drwxr-xr-x	`downloads'
drwxr-xr-x	`Hes.Just.Not.That.Into.You.DVDRip.XviD-DiAMOND'
drwxr-xr-x	`Hitch[2005]DVDrip[Eng]AC3[5.1]-Atlas47'
drwxr-xr-x	`Ice
-rw-r--r--	`Ice.Road.Truckers.S03E03.Canadian.Invasion.DSR.XviD-KRS.avi'
drwxr-xr-x	`livecd-i686-installer-2008.0-r1'
drwxr-xr-x	`lose
-rw-r--r--	`not_found.html'
-rw-r--r--	`not_found.html.1'
drwxr-xr-x	`Ocean\'s.12[2004]DvDrip[Eng]-aXXo'
drwxr-xr-x	`perl'
drwxr-xr-x	`python'
drwxr-xr-x	`Relapse'
drwxr-xr-x	`Spore-RELOADED'
drwxr-xr-x	`The.Accidental.Husband[2008]DvDrip-aXXo'
drwxr-xr-x	`The.Illusionist[2006]DvDrip[Eng]-aXXo'
-rw-r--r--	`t_skariah.asc.gpg'
-rw-r--r--	`tuto.txt'
-rw-r--r--	`tuto.txt~'
drwxr-xr-x	`Underworld
drwxr-xr-x	`Yes.Man.2008.DvDRip-FxM'

Pretty cool yeah? So, thats all well and good, but I want to add my own spin to the output, the following is a bit different but shows awk’s brilliance even more.

ls -lh --quoting-style=locale | grep -v "total " | awk '{print "Size is ", $5 "\tfor ", $8}'

This will print out the following:

Size is  4.0K	for  `Angus,Thongs.And.Perfect.Snogging[2008]DvDrip-aXXo'
Size is  149M	for  `archlinux-2009.02-ftp-i686.iso'
Size is  4.0K	for  `Bend
Size is  4.0K	for  `Bride
Size is  4.0K	for  `Desktop'
Size is  4.0K	for  `downloads'
Size is  4.0K	for  `Hes.Just.Not.That.Into.You.DVDRip.XviD-DiAMOND'
Size is  4.0K	for  `Hitch[2005]DVDrip[Eng]AC3[5.1]-Atlas47'
Size is  4.0K	for  `Ice
Size is  348M	for  `Ice.Road.Truckers.S03E03.Canadian.Invasion.DSR.XviD-KRS.avi'
Size is  4.0K	for  `livecd-i686-installer-2008.0-r1'
Size is  4.0K	for  `lose
Size is  303	for  `not_found.html'
Size is  303	for  `not_found.html.1'
Size is  4.0K	for  `Ocean\'s.12[2004]DvDrip[Eng]-aXXo'
Size is  4.0K	for  `perl'
Size is  4.0K	for  `python'
Size is  4.0K	for  `Relapse'
Size is  4.0K	for  `Spore-RELOADED'
Size is  4.0K	for  `The.Accidental.Husband[2008]DvDrip-aXXo'
Size is  4.0K	for  `The.Illusionist[2006]DvDrip[Eng]-aXXo'
Size is  2.4K	for  `t_skariah.asc.gpg'
Size is  3.0K	for  `tuto.txt'
Size is  575	for  `tuto.txt~'
Size is  4.0K	for  `Underworld
Size is  4.0K	for  `Yes.Man.2008.DvDRip-FxM'

Now, lets pick that command apart. The first section:

ls -lh --quoting-style=locale

Gives us a listing of the current working directory in a human readable long listing style. Meaning it prints the file sizes with K/M/G/T on the end, long listing means it gives you loads of extra info, like permissions, group user ownership, timestamps etc.

The next section:

grep -v "total "

Means, before we pass the text to awk, get rid of any line which contains “total “, ls -lh would usually print “total ” as the first line, which would muck up our awk command.

The final section:

awk '{print "Size is ", $5 "\tfor ", $8}'

Means, print the words “Size is” followed by column 5 then the word “for” then print column 8. The ‘\t’ is an escape character which means tab, it tells awk to insert a tab there. The various ‘,’ and whitespace in the quotes are just for presentation, you’ll have to experiment with them yourselves.

In case you didn’t know, the pipes

|

send the output of the command preceding the pipe, to the input of the command succeeding the pipe.

Hopefully now you can see the power of awk. This is by no means whatsoever a complete guide, It barely covers the basics. If you wnt to find out more, check out chapter 6 of the bash beginners guide.

Thanks for reading.

PS: Angus,Thongs.And.Perfect.Snogging[2008]DvDrip-aXXo is not porn, its a film my girlfriend bought :)

Baki Goxhaj
I’m a web consultant specializing in WordPress. I love elegant solutions, quality code and a good laughter.
You may also like
UKlocum, my Latest WordPress Freelance Project
WordPress Bug: Tags do not appear under WP_Query

Leave Your Comment

Your Comment*

Your Name*
Your Webpage