...making Linux just a little more fun!
By Ben Okopnik
Originally published in Issue 54 of Linux Gazette, May
2000
Last month, we took a look at some basics of creating a shell script, as well as a few of the underlying mechanisms that make it all work. This time around, we'll see how loops and conditional execution let us direct program flow in scripts, as well as looking at a few good shell-writing practices.
Often, scripts are written to automate some repetitive task; as a random example, if you have to repeatedly edit a series of files in a specific directory, you might have a script that looks like this:
#!/bin/bash for n in ~/weekly/*.txt do ae $n done echo "Done."or like this:
#!/bin/bash for n in ~/weekly/*.txt; do ae $n; done; echo "Done."The code in both does exactly the same thing - but the first version is much more readable, especially if you're building large scripts with several levels. As good general practice in writing code, you should indent each level (the commands inside the loops); it makes troubleshooting and following your code much easier.
The above control structure is called a 'for' loop - it looks for items remaining in a list (e.g., 'are there any more files, beyond the ones we have already read, that fit the "~/weekly/*.txt" template?'). If the test returns true, it assigns the name of the current item in the list to the loop variable ("n" in this case) and executes the loop body (the part between "do" and "done"), then checks again. Whenever the list runs out, 'for' stops looping and passes control to the line following the 'done' keyword - in our example, the "echo" statement.
A little trick I'd like to mention here. If you want to make the "for" loop 'spin' a certain number of times, the shell syntax can be somewhat tiresome:
#!/bin/bash for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 do echo $i doneWhat a pain! If you wanted it to iterate, say, 250 times, you'd have to type all of that out! Fortunately, there's a 'shortcut' - the '
seq
' command, which prints a sequence of numbers from 1 to
the given maximum, e.g.,
#!/bin/bash for i in $(seq 15) do echo $i doneThis is functionally the same as the previous script. '
seq
'
is part of the GNU "shellutils" package and is probably already installed
on your system. There's also the option of doing this sort of iteration by
using a "while" loop, but it's a bit more tricky.
while; do; done
Often, we need a control mechanism that acts based on a specified condition rather than iterating through a list. The 'while' loop fills this requirement:
#!/bin/bash pppd call provider & while [ -n "$(ping -c 1 192.168.0.1|grep 100%)" ] do echo "Connecting..." done echo "Connection established."The general flow of this script is: we invoke '
pppd
', the PPP
paenguin... I mean, daemon :), then keep looping until an actual
connection is established (if you want to use this script, replace
192.168.0.1 with your ISPs IP address). Here are the details:
1) The "ping -c 1 xxx.xxx.xxx.xxx
" command sends a single
ping to the supplied IP address; note that it has to be an IP address and
not a URL - "ping
" will fail immediately due to lack of DNS
otherwise. If there's no response within 10 seconds, it will print
something like
PING xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx): 56 data bytes ping: sendto: Network is unreachable ping: wrote xxx.xxx.xxx.xxx 64 chars, ret=-1 --- xxx.xxx.xxx.xxx ping statistics --- 1 packets transmitted, 0 packets received, 100% packet loss
2) The only line we're interested in is the one that gives us the packet loss percentage; with a single packet, it can only be 0% (i.e., a successful ping) or 100%. By piping the output of "ping" through the "grep 100%" command, we narrow it down to that line, if the loss is indeed 100%; a 0% loss will not produce any output. Note that the "100%" string isn't anything special: we could have used "ret=-1", "unreachable", or anything else that's unique to a failure response.
3) The square brackets that contain the statement are a synonym for the
'test' command, which returns '0' or '1' (true or false) based on the
evaluation of whatever's inside the brackets. The '-n' operator returns
'true' if the length of a given string is greater than 0. Since the string
is assumed to be contiguous (no spaces), and the line we're checking for is
not, we need to surround the output in double quotes - this is a technique
that you will use again and again in script writing. Do note that the
square brackets require spaces around them - i.e., [-n $STRING]
won't work; [ -n $STRING ]
is correct. For more info on the
operators used with 'test', type "help test"; a number of very useful ones
are available.
4) As long as the above test returns "true" (i.e., as long as the "ping" fails), the 'while' loop will continue to execute - by printing the "Connecting..." string every ten seconds. As soon as a single ping is successful (i.e., the test returns "false"), the 'while' loop will break and pass control to the statement after "done".
"until; do; done"
The 'until' loop is the reverse of the 'while' - it continues to loop as long as the test is false, and fails when it becomes true. I've rarely had the occasion to use it; the 'while' loop and the flexibility of the available tests usually suffice, and this construct is just "syntactic sugar" for those who prefer to avoid logical inversions.
"if; then; [else]; fi"
There are many times when we need to check for the existence of a condition and branch the execution based on the result. For those times, we have the 'if' statement:
... if [ $BOSS="jerk" ] then echo 'Take this job and shove it!' else echo 'Stick around; the money is good.' fi ...<grin> I guess it's not quite that easy... but the logic makes sense. Anyway, if a variable called BOSS has been defined as "jerk" (C programmers take note: '=' and '==' are equivalent in a test statement - no assignment occurs), then the first 'echo' statement will be executed. In all other cases, the second 'echo' statement will run (if $BOSS="idiot", you'll still be working there. Sorry about that. :). Note that the 'else' statement is optional, as in this script fragment:
... if [ -n $ERROR ] then echo 'Detected an error; exiting.' exit fi ...
This routine will obviously exit if the ERROR variable is anything other than empty - but it will not affect the program flow otherwise.
"case; in; esac"
The remaining tool that we can use for conditional branching is basically a multiple 'if' statement, based on the evaluation of a test. If, for example, we know that the only possible outputs from an imaginary program called 'intel_cpu_test' are 4, 8, 16, 32, or 64, then we can write the following:
#!/bin/bash case $(intel_cpu_test) in 4) echo "You're running Linux on a calculator??";; 8) echo "That 8088 is past retirement age...";; 16) echo "A 286 kinda guy, are you?";; 32) echo "One of them new-fangled gadgets!";; 64) echo "Oooh... serious CPU envy!";; *) echo "What the heck are you running, anyway?";; esac
(Before all you folks flood me with mail about running Linux on an 8088... you can't run it on a calculator either. :)
Obviously, the "*" at the end is a catch-all: if someone at the Intel Secret Lab runs this on their new CPU (code name "UltraSuperHyperWhizBang"), we want the script to come back with a controlled response rather than a failure. Note the double semicolons - they 'close' each of the "pattern/command" sets and are (for some reason) a common error in "case/esac" constructs. Pay extra attention to yours!
break and continue
These statements interrupt the program flow in specific ways. The "break", once executed, immediately exits the enclosing loop; the "continue" statement skips the current loop iteration. This is useful in a number of situations, particularly in long loops where the existence of a given condition makes all further tests unnecessary. Here's a long (but hopefully understandable) pseudo-example:
... while [ "$hosting_party" = "true" ] do case $FOOD_STATUS in potato_chips_gone) replace_potato_chips;; peanuts_finished) refill_peanut_bowl;; pretzels_gone) open_new_pretzel_bag;; ... ... esac if [ police_on_scene ] then talk_to_nice_officers continue fi case $LIQUOR_STATUS in vodka_gone) open_new_vodka_bottle;; rum_gone) open_new_rum_bottle;; ... ... esac case $ANALYZE_GUEST_BEHAVIOR in lampshade_on_head) echo "He's been drinking";; talking_to_plants) echo "She's been smoking";; talking_to_martians) echo "They're doing LSD";; levitating_objects) echo "Who spiked my lemonade??";; ... ... ... esac done echo "Dude... what day is it?"
A couple of key points: note that in checking the status of various party supplies, you might be better off writing multiple "if" statements - both potato chips and pretzels may run out at the same time (i.e., they are not mutually exclusive). The way it is now, the chips have top priority; if two items do run out simultaneously, it will take two loops to replace them.
We can keep checking the food status while trying to convince the cops that we're actually holding a stamp-collectors' meeting (in fact, maintaining the doughnut supply is a crucial factor at this point), but we'll skip right past the liquor status - as it was, we got Joe down off the chandelier just in time...
The "continue" statement skips the last part of the "while" loop as long as the "police_on_scene" function returns 'true'; essentially, the loop body is truncated at that point. Note that even though it is actually inside the "if" construct, it affects the loop that surrounds it: both "continue" and "break" apply only to loops, i.e., "for", "while", and "until" constructs.
#!/bin/bash # "bkup" - copies specified files to the user's ~/Backup # directory after checking for name conflicts. a=$(date +%T-%d_%m_%Y) cp -i $1 ~/Backup/$1.$aInterestingly enough, shortly after finishing last month's article, I was cranking out a bit of C code on a machine that didn't have 'rcs' (the GNU Revision Control System) installed - and this script came in very handy as a 'micro-RCS'; I used it to take "snapshots" of the project status. Simple, generalized scripts of this sort become very useful at odd times...
cp: missing destination file Try `cp --help' for more information.For everyone else, and for ourselves down the road when we forget exactly how to use this tremendously complex script with innumerable options :), we need to put in error checking - specifically, syntax/usage information. Let's see how what we've just learned might apply:
#!/bin/bash if [ -z $1 ] then echo "'bkup' - copies the specified file to the user's" echo "~/Backup directory after checking for name conflicts." echo echo "Usage: bkup <filename>" echo exit fiThe '-z' operator of 'test' returns '0' (true) for a zero-length string; what we're testing for is 'bkup' being run without a filename. The very beginning is, in my opinion, the best place to put help/usage information in a script - if you forget what the options are, just run the script without any, and you'll get an instant 'refresher course' in using it. You don't even have to put in the original comments, now - note that we've basically incorporated our earlier comments into the usage info. It's still a good idea to put in comments at any non-obvious or tricky places in the script - that brilliant trick you've managed to pull off may cause you to cuss and scratch your head next year if you don't.a=$(date +%T-%d_%m_%Y) cp -i $1 ~/Backup/$1.$a
Before we wrap up playing with this script, let's give it a few more capabilities. What if you wanted to be able to send different types of files into different directories? Let's give that a shot, using what we've learned:
#!/bin/bash if [ -z $1 ] then echo "'bkup' - copies the specified file to the user's ~/Backup" echo "directory tree after checking for name conflicts." echo echo "Usage: bkup filename [bkup_dir]" echo echo "bkup_dir Optional subdirectory in '~/Backup' where the file" echo " will be stored." echo exit fi if [ -n $2 ] then if [ -d ~/Backup/$2 ] then subdir=$2/ else mkdir -p ~/Backup/$2 subdir=$2/ fi fi a=$(date +%T-%d_%m_%Y) cp -i $1 ~/Backup/$subdir$1.$aHere is the summary of changes:
1) The comment section of the help now reads "...directory tree" rather than just "directory", indicating the change we've made.
2) The "Usage:" line has been expanded to show the optional (as shown by the square brackets) argument; we've also added an explanation of how to use that argument, since it might not be obvious to someone else.
3) An added "if" construct that checks to see if $2 (a second argument to 'bkup') exists; if so, it checks for a directory with the given name under "~/Backup", and creates one if it does not exist (the "-d" tests if the file exists and is a directory).
4) The 'cp' command now has a 'subdir' variable tucked in between "Backup/" and "$1".
Now, you can type things like
bkup my_new_program.c c bkup filter.awk awk bkup filter.awk filters bkup Letter_to_Mom.txt docsetc., and sort everything into whatever categories you like. Plus, the old behavior of "bkup" is still available -
bkup file.xyz
will send a backup of "file.xyz" to the "~/Backup" directory itself; useful for files that fall outside of your sorting criteria.
By the way: why are we appending a "/" to $2 in the "if" statement instead of right in the "cp" line? Well, if $2 doesn't exist, then we want 'bkup' to act as it did originally, i.e., send the file to the "Backup" directory. If we write something like
cp -i $1 ~/Backup/$subdir/$1.$a
(note the extra "/" between $subdir and $1), and $2 isn't specified, then $subdir becomes blank, and the line above becomes
cp -i $1 ~/Backup//$1.$a
- not that it hurts anything, but we want to stick with standard shell syntactic practice wherever possible (since shell quirks, such as a double '/' being ignored, are not guaranteed to stick around.)
In fact, it's a really good idea to consider all the possibilities whenever you're building variables into a string; a classic mistake of that sort can be seen in the following script -
DO NOT USE THIS SCRIPT! #!/bin/bash # Written by Larry, Moe, and Shemp - the Deleshun PoWeR TeaM!!! # Checked by Curly: "Why, soitainly it woiks! Nyuk-nyuk-nyuk!" # All you've gotta do is enter the name of this file followed by # whatever you want to delete - directories, dot files, multiple # files, anything is OK! rm -rf $1* DO NOT USE THIS SCRIPT!
Well, at least they commented it. :)
What happens if somebody does run "three_stooges", and doesn't enter a parameter? The active line in the script becomes
rm -rf *
Assuming that you're Joe User in your home directory, the result is pretty horrible - it'll wipe out all of your personal files. It becomes a catastrophe if you're the root user in the root directory - the entire system goes away!!
Viruses seem like such friendly, harmless things about now...
Be careful with your script writing. As you have just seen, you have the power to destroy your entire system in a blink.
Unix was never designed to keep people from doing stupid things, because that policy would also keep them from doing clever things. -- Doug Gwyn Unix gives you just enough rope to hang yourself - and then a couple more feet, just to be sure. -- Eric Allman
The philosophy makes sense: unlimited power in the tools, restriction by permissions - but it imposes a responsibility: you must take appropriate care. As a corollary, whenever you're logged in as root, do not run any shell scripts that are not provably harmless (note the Very Large assumptions hanging off that phrase - "provably harmless"...)
Next month, we'll take a look at some tools that are commonly used in shell scripts - tools that may be very familiar to you as command-line utilities - and explore how they may be connected together to produce desired results. We'll also dissect a couple of scripts - mine, if no one else is brave enough to send in the results of their keyboard concoctions. (Be Afraid. Be Very Afraid. :)
I welcome all comments and corrections in regard to this series of articles,
as well as any interesting scripts that you may send in. All flames will
be sent to /dev/null
(oh no, it's full...)
Until next month -
Happy Linuxing!
What's this script do? 'unzip; touch; finger; mount; gasp; yes; umount; sleep' Hint for the answer: not everything is computer-oriented. Sometimes you're in a sleeping bag, camping out with your girlfriend.'' -- Frans van der Zande
References
The "man" pages for 'bash', 'seq', 'ping', 'grep'
The "help" command for 'for', 'while', 'until', 'if', 'case', 'test', 'break', 'continue'
"Introduction to Shell Scripting - The Basics" in the previous issue
Ben is the Editor-in-Chief for Linux Gazette and a member of The Answer Gang.
Ben was born in Moscow, Russia in 1962. He became interested in electricity
at the tender age of six, promptly demonstrated it by sticking a fork into
a socket and starting a fire, and has been falling down technological
mineshafts ever since. He has been working with computers since the Elder
Days, when they had to be built by soldering parts onto printed circuit
boards and programs had to fit into 4k of memory. He would gladly pay good
money to any psychologist who can cure him of the recurrent nightmares.
His subsequent experiences include creating software in nearly a dozen
languages, network and database maintenance during the approach of a
hurricane, and writing articles for publications ranging from sailing
magazines to technological journals. After a seven-year Atlantic/Caribbean
cruise under sail and passages up and down the East coast of the US, he is
currently anchored in St. Augustine, Florida. He works as a technical
instructor for Sun Microsystems and a private Open Source consultant/Web
developer. His current set of hobbies includes flying, yoga, martial arts,
motorcycles, writing, and Roman history; his Palm Pilot is crammed full of
alarms, many of which contain exclamation points.
He has been working with Linux since 1997, and credits it with his complete
loss of interest in waging nuclear warfare on parts of the Pacific Northwest.