Statements: For
Writing for
statements with no unintended side effects is probably the
hardest part of writing shell scripts. That was my opinion, until now…
Using nullglob
to loop over files
Let’s say you want to print files in a directory:
for file in directory/*
do
printf "%s\n" "${file}"
done
This works, if you have some files in the directory. If not, you get
literally the output “directory/*
. Getting this unexpanded string back is extremely useless. Consulting Stack Overflow or - if all else fails - the documentation,
we fix this to read:
shopt -s nullglob
for file in directory/*
do
printf "%s\n" "${file}"
done
shopt -u nullglob
That code is fine… unless you later change the printf
line to call another
function, that did not expect nullglob
to be set. This is an easy mistake
to make, so I’d expect, that I will make it. The actual proper solution is:
shopt -s nullglob
for file in directory/*
do
shopt -u nullglob
printf "%s\n" "${file}"
done
shopt -u nullglob
This is correct, because the bash expands all items of the for loop during
the interpretation of the for line and not again afterwards. It is necessary
to keep the final shopt -u nullglob
, in case the directory is empty and no
looping is done.
I find this quite ugly and it is a lot of type-work. A source of bugs, if you ask me.
Beauty in macros
Macros in bash ? Yes there is a macro facility in bash. It’s called alias
and
it’s available for scripting with shopt -s expand_aliases
(bash) or
setopt aliases
(zsh). alias as a macro facility is very limited. It can not
be used in the middle of a statement and it knows no arguments.
Note
That you can use aliases in shell scripts is something I just recently learned through the yoctu project. To me learning this, felt like a game changer.
Lets define these aliases (bash shown, for zsh see mulle-compatibility.sh):
alias .foreachfile="set +f; shopt -s nullglob; IFS=' '$'\t'$'\n'; for"
alias .do="do
set +f; shopt -u nullglob; IFS=' '$'\t'$'\n'"
alias .done="done;set +f; shopt -u nullglob; IFS=' '$'\t'$'\n'"
Now the loop can be rewritten to:
.foreachfile file in directory/*
.do
printf "%s\n" "${file}"
.done
which is very nice looking and foolproof.
Looping over words
You expect a list of names and you want to run a loop over them. But some joker is actually passing you wildcards:
foo()
{
local names="$1"
for name in ${names}
do
printf "%s\n" "${name}"
done
}
foo 'a * b'
The output includes the unintended list of files of the current directory
a
404.html
atom.xml
_config.yml
css
fonts
Gemfile
Gemfile.lock
images
_includes
index.markdown
_layouts
modern-bash-scripting.sublime-project
modern-bash-scripting.sublime-workspace
_posts
_site
b
This is the so called “globbing” in effect, which is on by default. It can be
turned on with set +f
and off with set -f
. Above code can be fixed with
another macro alias .for="set -f; for
:
foo()
{
local names="$1"
.for name in ${names}
.do
printf "%s\n" "${name}"
.done
}
foo 'a * b'
Output:
a
*
b
Note
I find turning off globbing globally for the whole script, as a way to circumvent the problem, to be inconvenient. Any interactive shell has globbing enabled, and this would require me to remember to also selectively turn globbing on and off, to test my code in a copy/paste fashion.
Looping over a PATH
By setting the global variable IFS
you can control the way the shell splits
a string up as items for the for
loop. Forgetting to reset IFS properly can
be even more devastating then a missing nullglob.
IFS=':'
for x in a:b:c:d
do
IFS=' '$'\t'$'\n'
printf "%s\n" "$x"
done
IFS=' '$'\t'$'\n'
With the macro alias .foreachpath="set -f; IFS=':'; for"
this can be
beautified to:
.foreachpath x in a:b:c:d
.do
printf "%s\n" "$x"
.done
Some more macros
Here are some more useful macros. Each specify a different IFS.
alias .foreachline="set -f; IFS=$'\n'; for"
alias .foreachitem="set -f; IFS=','; for"
alias .foreachcolumn="set -f; IFS=';'; for"