User Tools

Site Tools


wiki:2018conditionals

Loops and More Conditionals 🙉

Loops are a way to execute the same action across multiple inputs.

Suppose we have 3 commands that must be executed for each sample: cmd1, cmd2, and cmd3.

cmd1 sample_1
cmd2 sample_1
cmd3 sample_1
cmd1 sample_2
cmd2 sample_2
cmd3 sample_2
     .
     .
     .
cmd1 sample_n
cmd2 sample_n
cmd3 sample_n

… for n samples.

It is useful to generalize this workflow into:

for sample_i in { sample_1 ... sample_n }
do
    cmd1 sample_i
    cmd2 sample_i
    cmd3 sample_i
done

Benefits:

  • there is much less written code
  • can work on any number of samples
  • generalized

The most conventional form of a loop is the while loop.

while [ CONDITION ]
do
   statements
       .
       .
       .
done

Examples

A common task is to execute a loop with a counter, which is a variable that gets increments (i.e., it counts) every loop iteration.

While loops

Start a new file: while_loop.bash

$ cd ~/dev
$ pwd
~/dev
$ nano while_loop.bash

Enter this code into while_loop.bash

#!/usr/bin/env bash
 
# while_loop.bash
 
counter=0
while [ $counter -lt 10 ];
do
    let counter=counter+1
    echo "This is loop iteration $counter"
 
done

Change permissions and run:

$ chmod 755 while_loop.bash
$ ./while_loop.bash
This is loop iteration 1
This is loop iteration 2
This is loop iteration 3
This is loop iteration 4
This is loop iteration 5
This is loop iteration 6
This is loop iteration 7
This is loop iteration 8
This is loop iteration 9
This is loop iteration 10

continue

The keyword continue means “skip to the next iteration”. Suppose we only want odd-numbered lines.

counter=0
while [ $counter -lt 10 ];
do
    let counter=counter+1
    if [[ $((counter % 2)) == 0 ]] # use modulo (%) to check if divisible by 2
    then
        continue
    fi  
    echo "This is loop iteration $counter"
 
done
This is loop iteration 1
This is loop iteration 3
This is loop iteration 5
This is loop iteration 7
This is loop iteration 9

Here we “continue”, or skip to the beginning of the loop when counter is evenly divisible by 2 (modulus is 0).

break

Suppose we detect a situation that causes us to end the loop for a reason other than that specified by the conditional.

#!/usr/bin/env bash
 
# while_loop.bash
 
counter=0
while [ $counter -lt 10 ];
do
    let counter=counter+1
    if [[ $((counter % 2)) == 0 ]] # use modulo (%) to check if divisible by 2
    then
        continue
    elif [[ $counter > 5 ]]
    then
        break
    fi  
    echo "This is loop iteration $counter"
 
done
This is loop iteration 1
This is loop iteration 3
This is loop iteration 5

Arithmetic

let

The let keyword can be used to do simple arithmetic on a variable.

$ var=1
$ echo $var
1
$ let var=var+1
$ echo $var
2

$((expr))

An alternate form is:

$ var=1
$ echo $var
1
$ var=$((var+1))
$ echo $var
2
$ echo $((var % 2))
0

This is the syntax we used to compute the remainder (modulo) above.

For loops

Looping over a list

Sometimes you have a set of items that you want to apply a task to. In this case, there is a more convenient form.

$ pwd
~/dev
$ nano for_loop.bash

Insert the following code into for_loop.bash

#!/usr/bin/env bash
 
# for_loop.bash
 
list="lions tigers bears"
for item in $list
do
   echo "$item"
done

Change permissions and run:

$ chmod 755 for_loop.bash
$ ./for_loop.bash
lions
tigers
bears

A script on multiple args

Let's write a script that loops over the arguments on the command line:

$ pwd
~/dev
$ nano loop_over_args.bash

Insert the following code into loop_over_args.bash

#!/usr/bin/env bash
 
# loop_over_args.bash
i=0
for arg in $@;
do
  echo "$i:$arg"
  let i=i+1
done

Change permissions and run:

$ chmod 755 loop_over_args.bash
$ ./loop_over_args.bash a b c
0:a
1:b
2:c
$ ./loop_over_args.bash tinker tailor soldier spy 
0:tinker
1:tailor
2:soldier
3:spy

Looping over files

A common task in building pipelines is looping over files. To loop over all the files in your current directory, do:

$ for f in `ls`; do echo $f; done

We used the backquote `cmd` syntax here. The following is also valid:

$ for f in $(ls); do echo $f; done

The list that is being iterated over comes from the output of the ls command.

Let's make something useful:

$ pwd
~/dev
$ nano file_tests.bash

write the following into file_tests.bash

#!/usr/bin/env bash
 
# file_tests.bash
 
for f in $(ls)
do
    if [ -d $f ]
    then
        echo "$f is a directory"
    elif [ -x $f ]
    then
        echo "$f is executable"
    else
        echo "$f is just a regular file."
    fi
done

Make executable and run:

$ chmod 755 file_tests.bash
$ ./file_tests.bash

You will see each file in your directory broken down into directory, executable, or regular file. There are many more operators that are useful, whereas some check for filetypes that we would not encounter (block device files, sockets,…). For a complete list, see http://tldp.org/LDP/abs/html/fto.html

More comparison operators

There are many ways to express comparisons in BASH.

Integer Equality

Integers can be tested for equality using the -eq operator or the == operator (most common in other languages).

-eq

This operator and its counterparts (-ne -lt -gt -ge -le) is not common in other languages, but can be used interchangeably with the different test and bracket constructs.

$ test 2 -eq 2 &&  echo "true" || echo "false"
true
$ test 1 -eq 2 &&  echo "true" || echo "false"
false
$ [ 2 -eq 2 ] &&  echo "true" || echo "false"
true
$ [ 1 -eq 2 ] &&  echo "true" || echo "false"
false
$ [[ 2 -eq 2 ]] &&  echo "true" || echo "false"
true
$ [[ 1 -eq 2 ]] &&  echo "true" || echo "false"
false

==

Symbolic operators (== < > <= >=) must be used within double parentheses to ensure numeric comparison (rather than alphanumeric).

## '=='
$ (( 2 == 2 )) &&  echo "true" || echo "false"
true
$ (( 1 == 2 )) &&  echo "true" || echo "false"
false

Integer comparison

Integers can be tested for specific inequalities using flag operators and brackets or symbol operators and double parenthesis

$ [ 1 -lt 0 ] &&  echo "true" || echo "false"
false
$ [[ 1 -lt 0 ]] &&  echo "true" || echo "false"
false
$ (( 1 < 0 )) &&  echo "true" || echo "false"
false

Keeping track of comparison syntax

It is difficult to anticipate what syntax will behave as expected. There are things you can do to make sure you do it right:

Negation

You can reverse the orientation of a test outcome by include the BOOLEAN NOT (!) operator.

$ [[ 1 -lt 0 ]] &&  echo "true" || echo "false"
false
$ [[ ! 1 -lt 0 ]] &&  echo "true" || echo "false"
true

In English, you can compare the differences in these two statements as

  • if 1 is less than 0… to
  • if NOT 1 is less than 0…

This is clunky in spoken English, but is perfectly normal in Boolean constructs.

Boolean Operators and Compound Conditionals

The use of the NOT (!) operator reverses the outcome of a boolean expression. The use of operators AND (&&) and OR (||) can also be used to make more complex statements.

We've already encountered the use of these operators as shorthand for if-statements on the command line.

$ CONDITIONAL && echo "true" || echo "false"

This is actually a special case of mixing commands and conditionals, because BASH has the convenient ability to treat command exit statuses as boolean outcomes.

The conventional use of boolean operators is to build compound conditionals out of comparison statements.

The following echoes true:

flag="-a"
if [[ 1 -lt 2 && $flag == '-a' ]]
then
   echo "true"
else
   echo "false"
fi

Whereas the following echoes false

flag="-x"
if [[ 1 -lt 2 && $flag == '-a' ]]
then
   echo "true"
else
   echo "false"
fi

However, what if we only needed the numeric comparison OR the string comparison to be true, rather than both?

flag="-x"
if [[ 1 -lt 2 || $flag == '-a' ]]
then
   echo "true"
else
   echo "false"
fi

This statement echoes true, because I change the operator from AND (&&) to OR (||). 8-O 8-O 8-O 8-O 8-O 8-O 8-O 8-O 8-O

Command exit status as boolean

BASH can mix command exit status and other boolean concepts. This is useful in pipeline building.

Command Boolean value
Normal command such as ls, echo TRUE
Command that fails, such trying to ls a file that isn't there. FALSE
Command not found, such as typing gibberish on the command line. FALSE

Let's try some:

Doing ls on a non-existent file:

$ ls notafile 
ls: cannot access notafile: No such file or directory

Now let's append an OR (||) to the attempted command.

$ ls notafile || echo "something went wrong"
ls: cannot access notafile: No such file or directory
something went wrong

Now you have detected, in a single statement, that the command failed. We already knew that because we're purposefully causing it. But in a pipeline, you may need to check for programs failing.

# example pipeline
 
failures=0
 
ls notafile || failures=$((failures+1))
 
echo "YAY" || failures=$((failures+1))
 
asdfasdf || failures=$((failures+1))
 
echo "There were $failures failures"

Gives the error messages from our failed commands, plus the value of failures.

ls: cannot access notafile: No such file or directory
-bash: asdfasdf: command not found
There were 2 failures

This works because, in an OR-construct, if the left-hand-side resolve to FALSE, then it always evaluates the right-hand-side.

Likewise, if the left-hand-side resolves to TRUE, it doesn't need to evaluate the right-hand-side, so the command

failures=$((failures+1))

doesn't get executed.

Alternatively, in an AND-construct, if the left-hand-side resolves to TRUE, then it has to evaluate the right-hand-side.

# example pipeline
 
successes=0
 
ls notafile && successes=$((successes+1))
 
echo "YAY" && successes=$((successes+1))
 
asdfasdf && successes=$((successes+1))
 
echo "There was/were $successes successes"

I'm being lazy with my verb usage, but the outcome is:

ls: cannot access notafile: No such file or directory
-bash: asdfasdf: command not found
There was/were 1 successes.

Thus, the right-hand-side of (&&) doesn't get evaluated when the left-hand-side is FALSE.

Combining the two constructs:

# example pipeline
 
successes=0
failures=0
 
ls notafile && successes=$((successes+1)) || failures=$((failures+1))
 
echo "YAY" && successes=$((successes+1)) || failures=$((failures+1))
 
asdfasdf && successes=$((successes+1)) || failures=$((failures+1))
 
echo "There was/were $successes successes"
echo "There were $failures failures"

Gives

ls: cannot access notafile: No such file or directory
YAY
exit_conditionals.bash: line 11: asdfasdf: command not found
There was/were 1 successes
There were 2 failures

This is jumbled and kind of ridiculous, but shows how you can use the exit status to detect errors in your script.

Chaining

Here are some other examples you might encounter.

CMD1 && CMD2 && CMD3 || echo "ANY of the commands failed"
 
CMD1 || CMD2 || CMD3 || echo "ALL of the commands failed"

Suppressing error messages

If you want to suppress the error output of a command, you can redirect standard error to a special file /dev/null

$ loudcommand 2> /dev/null

This strategy can be used to clean-up output by discarding standard error.

successes=0
failures=0
 
ls notafile 2> /dev/null && successes=$((successes+1)) || failures=$((failures+1))
 
echo "YAY" 2> /dev/null && successes=$((successes+1)) || failures=$((failures+1))
 
asdfasdf 2> /dev/null && successes=$((successes+1)) || failures=$((failures+1))
 
echo "There was/were $successes successes"
echo "There were $failures failures"

Gives

YAY
There was/were 1 successes
There were 2 failures

The output is easier to read, but the code is harder to read.

On to Sed, grep and awk 🙊

wiki/2018conditionals.txt · Last modified: 2018/09/07 14:19 by david