bash

Fighting with bash
bash
snippets

December 6, 2020

set -euo pipefail

By default, bash will continue after errors.
set -e stops the script on errors.

By default, unset variables don’t error.
set -u stops the script on unset variables.

By default, a command failing doesn’t fail the whole pipeline.
set -o pipefail prevents this.

2>&1

File descriptor 1 is the standard output (stdout).
File descriptor 2 is the standard error (stderr).

Here is one way to remember this construct (although it is not entirely accurate): at first, 2>1 may look like a good way to redirect stderr to stdout. However, it will actually be interpreted as “redirect stderr to a file named 1”. & indicates that what follows and precedes is a file descriptor and not a filename. So the construct becomes: 2>&1.

Consider >& as redirect merger operator.

!!

History substitution: When you use history substitution, the shell first displays the command that it’s about to execute with all the substitutions shown, and then executes it.

[[

Note that [[ is actually a command/program that returns either 0 (true) or 1 (false).

[[ -s FILE ]]

Size is > 0 bytes.
The -s test returns true if file.txt exists and has a size greater than zero

|| true

In cases where a command in the script is allowed to fail, adding || true ensures that the resulting compound command always exits with status zero, so the script doesn’t abort. For example, removing a directory shouldn’t be a fatal error (preventing a package from being removed); so we’d use

rmdir ... || true

since rmdir doesn’t have an option to tell it to ignore errors.

[[ -z STRING ]]

Empty string

[[ -n STRING ]]

Not empty string

awk

For manipulating columns of data.
Basic awk structure: BEGIN {...} CONDITION {action} CONDITION {action} END {...}
Examples:

  • print thrid and fourth column of every row
awk '{print $3 "\t" $4}' test.txt
  • print all lines that match a pattern (print is default so {print $0} could be omitted)
awk '/a/ {print $0}' test.txt

grep

-i case insensitive
-E use if you want regular expressions like “.+” to work. Otherwise you need to use “.+”.
-v invert match: find all lines that don’t match.
-o only print the matching part of the line (not the whole line)

lsof

Stands for list open files For each open file:

  • pid
  • file type (regular? directory? FIFO? socket?)
  • file descriptor (FD column)
  • user
  • file name/socket address

sed

sed is most often used for replacing text in a file.

sed s/cat/dog/g file.txt

sed 5d - delete 5th line
sed /cat/d - delete lines matching /cat/
sed -n 5,30p - print lines 5-30
sed s+cat/+dog/+ - use + as regex delimiter (way easier than escaping /s like s/cat\//dog\//)
set '/cat/a dog' - append ‘dog’ after lines containing ‘cat’

find

find searches a directory for files.
https://wizardzines.com/comics/find/

find /tmp -type d -print

Substitution

name="John"
echo ${name/J/j}

=> “john” (substitution)