BASH - Ninja
-
NOTE - This is NOT a general guide. This page is meant to be a quick reference / reminder for me.
- Single quotes - perserve literal value of each char, including $
- Double quotes - perserve literal value of each char, except $, `, \
Notable Special Variables
| $IFS | list of characters that separate fields for word splitting |
| $PPID | shell’s PPID |
| $@ | Positional parameters, starting from one. |
| $# | Number of positional parameters |
| $? | Exit status of most recent executed foreground pipeline |
| $$ | PID of Shell |
| $! | PID of most recent executed background (asynchronous) command |
| $0 | Name of shell or script |
Brace expansion
echo f{oo,ee,a}d # food feed fad
echo {000..2} # new in Bash 4, 000 001 002
echo {00..8..2} # 00 02 04 06 08
echo {D..T..4} # D to T increment by 4
echo start{a,b{1,2,3},c}end # Nested brace expansion
for num in {000..2}; do echo "$num"; done # in a for loop
Arithmetic Expansion
$(( 5 + 5 ))
$[ 5 + 5 ] # older
Command Substitution, Subshell
- can be nested
(ls -l) # subshell
x=$(ls -l) # newer, easier to nest, POSIX
x=`ls -l` # older, can have some escape chars?
Word splitting
-
happens when not using double quotes
-
IFS -value is “‘<space><tab><newline>’” when unset, will split on any of these, when null no splitting occurs
IFS=$'\n' # set IFS to just new lines
for i in $(ps -ef ); do echo $i; done # now it splits by line
File name expansion ( globbing )
ls -l *
ls -l test?.txt
ls -l test[1-5].txt
ls -l test[a-z].txt
Character classes:
ls -ld [[:digit:]]*
ls -ld [[:upper:]]*
# "alnum", "alpha", "ascii", "blank", "cntrl", "digit", "graph", "lower", "print", "punct", "space", "upper", "word" or "xdigit"
Aliases
alias ll='ls -l' # create alias
xargs
The xargs command is used to send batches of arguments to a command. The command is run for each batch. Number of args is limted and running process per arg is slow.
find /data1 | xargs rm # remove all files listed
find /data1 | xargs grep "Error" # grep for a string inside all files found
find /data1 -print0 | xargs -0 grep "Error" # same but handle files with whitespace in name
ls *.txt | xargs gzip # gzip all files listed
find /data1 | xargs gzip # gzip all files found
Conditionals - if/else
If/else syntax:
if [ $x -eq "abc" ]; then
echo "match 1"
elif [ $x -eq "xyz ]; then
echo "match 2"
else
echo "no match"
fi
if [ $x -eq "abc" ]; then echo "match 1"; fi # one liner
Common tests:
if [ -z $x ]; # check if string is: unset, empty, or all spaces
if [ -e "$filename" ]; # check if file exists ( -f regular file, -d dir, -e exists ), quotes to avoid word splitting
if [[ -e $filename ]]; # [[ to avoid word splitting
if [ $a == "test" ]; # strings equal
if [ $a != "test" ]; # strings not equal
if [ $x -eq 5 ]; # equal, can also use -eq, -ne, -lt, le, -gt, ge
if [ $? -eq 0 ]; # check exit status OK
if ! grep $extra_user /etc/passwd; # not found
if [ "$(whoami)" != 'root' ]; then # not root
if [[ "$x" == t* ]]; # use a regex
if [ ! $# == 2 ]; # check number of args
if ls test1.txt; then echo asdf; else echo xxxxx; fi # one liner, directly use command
Expressions, Logic, and checks:
((a++)) # increment variable
((x = 42)) # assign variable, arithmetic
((x=5+4)) # assign variable, arithmetic
x=$((5+4)) # assign variable, arithmetic
for ((i=0; i<10; i++)) # multiple operations used for for loop
echo $((a + b + (14 * c)))
if (( x = "5" )); ... # test assignment
if ((5 == 5)); ... # arrithmetic
if [ 5 -eq 5 ]; ... # arrithmetic
if [ "5" -eq "5" ]; ... # arrithmetic, string treated as integer
if [ "abc" -eq "xyz" ]; ... # FAIL - expects integer
if [ "abc" == "xyz" ]; ... # string compare
if [ 5 == 5 ]; ... # works but is it comaring as string or int????
if [ 5 = 5 ]; ... # works, aparently not assignment outside of arithmetic expansion
if [[ 5 -eq 5 || 1 -eq 3 ]] ... # logical or
if [[ 5 -eq 5 && 1 -eq 1 ]]; ... # logical and
if [[ (5 -eq 5 ) || (1 -eq 1 && 2 -eq 2) ]]; # group for logic
if [[ $x =~ ab* ]]; # regex ( * still matches multiple chars ????)
if [[ $x =~ a.* ]]; # regex
Redirection, Background, and Pipes
# Redirection
ls > test1.txt # redirect command output ( STDOUT ) to a file and **OVERWRITE** file
ls >> test1 # redirect command output ( STDOUT ) to a file and **APPEND** to file
ls asdf 2> test1.txt # redirect command output ( STDERR ) to a file and **OVERWRITE** file
ls asdf 2>> test1.txt # redirect command output ( STDERR ) to a file and **APPEND** to file
ls asdf &> test1.txt # redirect command output ( STDOUT and STDERR ) to a file ( since Bash 4 )
ls asdf >> output.log 2>&1 # append STDOUT to file, also redirect STDERR to STDOUT ( both go to file )
ls asdf >> output.log 2>&1 & # also run in background with extra &
# Background
./server1 &
./process_data.sh &
# Pipes
cat data1.csv | sort
ps -ef | grep -i nginx
Loops
- NOTE - Loops can be nested.
For Loops
x="a b c"
for i in `cat list`; do
echo $i ;
done
for i in `cat list`; do echo $i ; done
for i in `ls`; do echo $i ; done
for i in a b c; do echo $i; done
for i in {1..10}; do echo $i; done
for i in `seq 1 10`; do echo $i; done
for i in $x; do echo $i; done # loop over string ( word splitting )
for i in /opt/*; do echo $i; done # loop over globbed file list
for (( i=0; i<10; i++)); do echo $i; done
IFS=$'\n' # split on new line, not space or tab
for i in `ps -ef`; do echo $i; done
While Loops
i=0; while [ $i -lt 4 ]; do echo $i; i=$[$i+1];done
i=0
while [ $i -lt 4 ]; do
echo $i
i=$[$i+1]
done
while true; do # infinite loop
echo "."
sleep 5
done
Arrays
declare -a ARRAYNAME
a[3]=5 # assign element, will declare even if not first element
a=(1 2 3) # declare, initialize - no commas, just spaces
a=(a b c) # declare, initialize - no commas, just spaces
echo ${a[3]} # specific element
echo ${a[$x]} # variable as an index
echo ${a[*]} # entire array
echo ${a[@]} # entire array
echo ${a[@]:2:4} # slice array (index, len )
${#x[@]} # array length ( differnt from string len )
unset a[3] # remove element
unset a # remove array
for i in ${a[@]}; do echo $i; done # loop over array
a+=("test") # append element to end of array
a+="test" # append value to first element
Associative Arrays / Hashs ( BASH 4+ )
- Associative arrays must be declared explicitly
# Declare and initialize:
declare -A a=(
[color]="red"
[size]="5"
[shape]="square"
)
# Declare then initialize
declare -A a
a[color]="red"
a[size]="5"
a[shape]="square"
echo ${a[color]} # access element
unset a[color] # remove an element
echo ${a[@]} # all values
echo ${!a[@]} # all keys
echo ${#a[@]} # length
# Loop over hash:
for i in "${!a[@]}"; do
echo $i ": " ${a[$i]}
done
Functions
- “The curly braces must be separated from the body by spaces.”
- “The body of a function should end in a semicolon or a newline.”
- Function needs to be defined before being called.
test1 () { # totally valid, just need that space before {
echo "Values: " $1 $2 $3 $#
return 0
}
test1 ()
{
local x='abc' # local variable
y='xyz' # not local, survives outside the funciton even if first used here
echo "Values: " $1 $2 $3 $#
return 123 # between 0-255
}
test1 a b c
echo $?
Signals
| Signal name | Signal value | Effect |
| SIGHUP | 1 | Hangup |
| SIGINT | 2 | Interrupt from keyboard ctrl-c |
| SIGKILL | 9 | Kill signal, can’t block or ignore |
| SIGTERM | 15 | Termination signal |
| SIGSTOP | 17,19,23 | Stop the process, 19 is ctrl-z, can’t block or ignore |
- SIGKILL and SIGSTOP can not be caught, blocked or ignored.
Send signals to processes:
kill 5607 # SIGTERM 15 - kill process
kill -9 5607 # SIGKILL 9 - force kill process
kill -HUP 5607 # SIGHUP 1 - usually reload configs if supported
Dir Stack - Pushd / Popd
Navigate quickly with these:
dirs # show dir stack
dirs -p # show dir stack one entry per line
dirs -v # same but with numbers
pushd /etc # add this dir to the stack and switch to it
pushd # swap top two dirs,
pushd +2 # dir 2 and below shifted to top
popd # remove top dir from stack and change to that dir
popd +2 # remove dir 2 from stack