Page Comparison

...

What this section covers

• Standard Unix streams, redirection and piping
• How and why to direct script diagnostic messages to standard error, but function results to standard output
• Argument defaulting
• Automatically creating a log file for a script
• Capturing standard output in a variable using backtick quoting

Standard streams

As described Intro Unix: Standard streams and redirection each with a well-defined stream number:

...

• redirect standard output to a file, overwriting any exiting contents:
  echo "Output text" > out.txt
echo "Some output" 1> out.txt

• redirect standard output to a file, appending to any exiting contents:
  echo "Some text" >> out.txt
echo "More text" 1>> out.txt
• redirect standard error output to a file, overwriting any exiting contents:
ls xxxx 2> err.txt
• redirect standard error to standard output, and redirect standard output to a file:
  ls xxxx > ls.log 2>&1

• redirect standard output to standard error:
  echo "Output that will be redirected to standard error" 1>&2
echo "Output that will be redirected standard error" 2> err.txt 1>&2


...

For example, here's how to call the step_01.sh script so that its standard output goes to both the terminal Terminal and to a file.

~/workshop/step_01.sh helloWorld My name is Anna | tee step_01.log

See Standard streams and piping in the Intro Unix wiki.

When standard output and standard error streams are used

...

Here's a step_02.sh script that builds on our step_01.sh work. It is located at ~/workshop/step_02.sh. Make sure it is executable (chmod +x ~/workshop/step_02.txt).

#!/bin/bash

# Script version global variable. Edit this whenever changes are made.
__ADVANCED_BASH_VERSION__="step_02"

# =======================================================================
#  Helper functions
# =======================================================================

# Echo's its arguments to std error
echo_se() { echo "$@" 1>&2; }

# Sets up auto-logging to a log file in the current directory
# using the specified logFileTag (arg 1) in the log file name.
auto_log() {
  local logFileTag="$1"
  if [[ "$logFileTag" != "" ]]; then
    local logFilePath="./autoLog_${logFileTag}.log"
    echo_se ".. logging to $logFilePath"
    exec 1> >(tee "$logFilePath") 2>&1
  else
    echo_se "** ERROR in auto_log: no logFile argument provided"
    exit 255
  fi
}

# =======================================================================
#  Command processing functions
# =======================================================================

# function that says "Hello World!" and displays user-specified text.
function helloWorld() {
  local txt1=$1
  local txt2=$2
  shift; shift
  local rest=$@
  echo "Hello World!"
  echo "  text 1: '$txt1'"
  echo "  text 2: '$txt2'"
  echo "  rest:   '$rest'"
}

# function that displays its 1st argument on standard output and
# its 2nd argument on standard error
function stdStreams() {
  local outTxt=${1:-"text for standard output"}
  local errTxt=${2:-"text for standard error"}
  echo    "to standard output: '$outTxt'"
  echo_se "to standard error:  '$errTxt'"
}

# function that illustrates auto-logging and capturing function output
#  arg 1 - (required) tag to identify the logfile
#  arg 2 - (optional) text for standard output
#  arg 3 - (optional) text for standard error
function testAutolog() {
  local logFileTag="$1"
  local outTxt=${2:-"text for standard output"}
  local errTxt=${3:-"text for standard error"}

  auto_log "$logFileTag"

  echo -e "\n1) Call stdStreams with output and error text:"
  stdStreams "$outTxt" "$errTxt"

  echo -e "\n2) Capture echo output in a variable and display it:"
  local output=`echo $outTxt`
  echo -e "   echo output was:\n$output"

  echo -e "\n3) Call echo_se with error text:"
  echo_se "$errTxt"

  echo -e "\n4)Capture echo_se function output in a variable and display it:"
  output=`echo_se "$errTxt"`
  echo -e "echo_se output was: '$output'"
}

# =======================================================================
#  Main script command-line processing
# =======================================================================

function usage() {
  echo "
advanced_bash.sh, version $__ADVANCED_BASH_VERSION__

Usage: advanced_bash.sh <command> [arg1 arg2...]

Commands:
  helloWorld [text to display]
  stdStreams [text for stdout] [text for stderr]
  testAutolog <logFileTag> [text for stdout] [text for stderr]
"
  exit 255
}

CMD=$1    # initially $1 will be the command
shift     # after "shift", $1 will be the 2nd command-line argument; $2 the 3rd, etc.
          # and $@ will be arguments 2, 3, etc.
case "$CMD" in
  helloWorld) helloWorld "$@"
    ;;
  stdStreams) stdStreams "$1" "$2"
    ;;
  testAutolog) testAutolog "$1" "$2" "$3"
    ;;
  *) usage
    ;;
esac

...

In our new Helper functions section we've defined a function that redirects its arguments to standard error. This allows the script to separate diagnostic messages about its execution, from function "results" written to standard output.

...

In our new Command processing functions section we've defined a function that writes its first argument to standard output and its second argument to standard error:

# function that displays its 1st argument on standard output and
# its 2nd argument on standard error
function stdStreams() {
  local outTxt=${1:-"text for standard output"}
  local errTxt=${2:-"text for standard error"}
  echo    "to standard output: '$outTxt'"
  echo_se "to standard error:  '$errTxt'"
}

...

Here's how to call the stdStreams command specifying the text "hello world!" for standard output and "goodbye world!" for standard error.

~/workshop/step_02.sh stdStreams '"hello world!' "'goodbye world!"'

But how can you tell which stream is which? They're both written to the terminal Terminal by default!

exercise 1

Call the stdStreams command in some manner that can distinguish between standard output and standard error.

...

to standard output: 'hello world'

...

• exec 1> causes redirection of all standard output for the duration of the current shell environment (here the script).
• the 2>&1 at the end is our normal "redirect standard error to standard output" idiom
• >(tee "my_logfile.log") is the magic that says to write standard output (which now includes standard error text) to a file via tee.
  • the >( ) syntax also sends all the current execution environment's standard output to a sub-shell (more on this later)
  • appears to be necessary because otherwise tee would just act on its piped-in standard input

auto_log function

The new auto_log helper function sets up automatic logging to a log file in the current directory, named using a tag string specified as its 1st argument.

...

Importantly, auto_log reports an error and terminates script execution if no tag string is specified, via exit 255 (we'll see much more on error handling shortly).

The conditional test is made using a bash if/else/fi block, where double brackets ('"[[ ]]") enclose the test, always followed by a semicolon (";") or a new line. Note that there must always be a space after the open brackets, and one before the close brackets. Here the test is for string equality (see https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-Expressions.html for a complete list of bash comparison operators). Note that there must always be a space after the open brackets, and one before the close brackets.

# Sets up auto-logging to a log file in the current directory
# using the specified logFileTag (arg 1) in the log file name.
auto_log() {
  local logFileTag="$1"
  if [[ "$logFileTag" != "" ]]; then
    local logFilePath="./autoLog_${logFileTag}.log"
    echo_se ".. logging to $logFilePath"
    exec 1> >(tee "$logFilePath") 2>&1
  else
    echo_se "** ERROR in auto_log: no logFile argument provided"
    exit 255
  fi
}

...

• Starts automatic logging to a log file named using its 1st logFileTag argument, by calling the auto_log function.
• Uses echo -e where the -e argument to echo enables interpretation of backslash escapes.
  • e.g. "\t" as a tab Tab character and "\n" will be interpreted as a newline.
• Calls stdStreams with its 2nd and 3rd arguments.
• Calls echo capturing its output in a local variable using backtick execution syntax, then displays the captured text.
  • see Quoting in the shell Intro Unix:Backtick evaluation quoting
• Calls the echo_se function with some text.
• Calls the echo_se function again, capturing its output in a local variable, then displays the captured text.
  
  

...

...

What log file is produced? What are its contents, and why?

1) Call stdStreams with output and error text:
to standard output: 'text for standard output'
to standard error:  'text for standard error'

2) Capture echo output in a variable and display it:
   echo output was:
text for standard output

3) Call echo_se with error text:
text for standard error

4)Capture echo_se function output in a variable and display it:
text for standard error
echo_se output was: ''

...