...
As described in a section of Linux Fundamentals page, there are three built-in, Standard Streams, each with a well-defined stream number:
- 0 - standard input
- 1 - standard output
- 2 - standard error
Redirection characters allow you to control the stream input or output
...
- redirect standard output to a file, overwriting any exiting contents:
echo "Output text" > out.txt
echo "Output
textSome output" 1> out.txt - redirect standard output to a file, appending to any exiting contents:
echo "More Some text" >> out.txt
echo "More text" 1>> out.txt - redirect standard error output to a file, overwriting any exiting contents:
echo "Error
text" 2> ls xxxx 2> err.txt - redirect standard error to standard output:
ls xxxx > ls.log 2>&1
- redirect standard output to standard error:
echo "
Output that will go to standard error
" 1>&2
echo "Output that will go to standard error" 2> err.txt 1>&2
There's also the tee program, which takes its standard input and writes it to the specified file as well as to its standard output.
...
Code Block | ||
---|---|---|
| ||
# list 2 files, one that exists and one that does not
ls ~/.profile ~/xxx |
Produces this output in your terminal:
Code Block |
---|
ls: cannot access 'xxx': No such file or directory .profile |
...
Code Block | ||
---|---|---|
| ||
ls ~/.profile ~/xxx 1>stdout.txt 2>stderr.txt cat stdout.txt cat stderr.txt |
The step_02.sh Script
Here's a step_02.sh script that builds on our step_01.sh work. It is located at ~/workshop/step_02.sh. Make sure it is executable (chmod +x ~/step_02.txt
).
...
language | bash |
---|
...
Finally redirect both standard output and standard error to a single file:
Code Block | ||
---|---|---|
| ||
ls ~/.profile ~/xxx > ls.log 2>&1
# or
ls ~/.profile ~/xxx 2>1 1> ls.log
# But this does not redirect stdout
ls ~/.profile ~/xxx 2>&1 1> ls.log |
The step_02.sh Script
Here's a step_02.sh script that builds on our step_01.sh work. It is located at ~/workshop/step_02.sh. Make sure it is executable (chmod +x ~/step_02.txt
).
Code Block | ||
---|---|---|
| ||
#!/bin/bash # Script version global variable. Edit this whenever changes are made. __ADVANCED_BASH_VERSION__="step_02" # ======================================================================= # Helper functions # ======================================================================= # Echo's its arguments to std error echo_se() { echo "$@" 1>&2; } # Sets up auto-logging to a log file in the current directory # using the specified logFileTag (arg 1) in the log file name. auto_log() { local logFileTag="$1" if [[ "$logFileTag" != "" ]]; then local logFilePath="./autoLog_${logFileTag}.log" echo_se ".. logging to $logFilePath" exec 1> >(tee "$logFilePath") 2>&1 else echo_se "** ERROR in autoLogauto_log: no logFile argument provided" exit 255 fi } # ======================================================================= # Command processing functions # ======================================================================= # function that says "Hello World!" and displays user-specified text. function helloWorld() { local txt1=$1 local txt2=$2 shift; shift local rest=$@ echo "Hello World!" echo " text 1: '$txt1'" echo " text 2: '$txt2'" echo " rest: '$rest'" } # function that displays its 1st argument on standard output and # its 2nd argument on standard error function stdStreams() { local outTxt=${1:-"text for standard output"} local errTxt=${2:-"text for standard error"} echo "to standard output: '$outTxt'" echo_se "to standard error: '$errTxt'" } # function that illustrates auto-logging and capturing function output # arg 1 - (required) tag to identify the logfile # arg 2 - (optional) text for standard output # arg 3 - (optional) text for standard error function testAutolog() { local logFileTag="$1" local outTxt=${2:-"text for standard output"} local errTxt=${3:-"text for standard error"} auto_log "$logFileTag" echo -e "\n1) Call stdStreams with output and error text:" stdStreams "$outTxt" "$outErr$errTxt" echo -e "\n2) Capture stdStreamsecho output in a variable and display it:" local output=`stdStreams`echo "$outTxt" "$outErr"`$outTxt` echo -e "\tstdStreams echo output was:\n$output" echo -e "\n3) Call echo_se with someerror text:" echo_se "Some random text$errTxt" echo -e "\n4)Capture echo_se function output in a variable and display it:" output=`echo_se "Some random text$errTxt"` echo -e "echo_se output was: '$output'" } # ======================================================================= # Main script command-line processing # ======================================================================= function usage() { echo " advanced_bash.sh, version $__ADVANCED_BASH_VERSION__ Usage: advanced_bash.sh <command> [arg1 arg2...] Commands: helloWorld [text to display] stdStreams [text for stdout] [text for stderr] testAutolog <logFileTag> [text for stdout] [text for stderr] " exit 255 } CMD=$1 # initially $1 will be the command shift # after "shift", $1 will be the 2nd command-line argument; $2 the 3rd, etc. # and $@ will be arguments 2, 3, etc. case "$CMD" in helloWorld) helloWorld "$@" ;; stdStreams) stdStreams "$1" "$2" ;; testAutolog) testAutolog "$1" "$2" "$3" ;; *) usage ;; esac |
...
Code Block | ||
---|---|---|
| ||
# function that displays its 1st argument on standard output and # its 2nd argument on standard error function stdStreams() { local outTxt=${1:-"text for standard output"} local errTxt=${2:-"text for standard error"} echo "to standard output: '$outTxt'" echo_se "to standard error: '$errTxt'" } |
...
argument defaulting
Often you will want to provide a default for arguments not explicitly provided by the user. This can be done using this rather odd syntax, with ":-" characters separating the positional argument number from the desired default. The entire defaulting construct is enclosed in "${ }".
Code Block | ||
---|---|---|
| ||
textArg=${1:-"text default for 1st argument"} integerArg=${2:-54321} |
simpler usage function
We've replaced the multiple echo lines in usage also added command-line processing and usage support for the new functions.
In the usage function, we've replaced the multiple echo lines with a single echo'd string, illustrating that the invisible line feeds in the echo'd text are faithfully preserved. The function also exits with a non-0 ("failure") return code, since no valid command processing was performed.
Code Block | ||
---|---|---|
| ||
function usage() { echo " advanced_bash.sh, version $__ADVANCED_BASH_VERSION__ Usage: advanced_bash.sh <command> [arg1 arg2...] Commands: helloWorld [text to display] stdStreams [text for stdout] [text for stderr] testAutolog <logFileTag> [text for stdout] [text for stderr] " exit 1255 } |
Here's how to call the stdStreams command specifying the text "hello world!" for standard output and "goodbye world!" for standard error.
...
Expand | |||||
---|---|---|---|---|---|
| |||||
|
More Parts - automatic logging
So you have written a user-callable script like step_02.sh, and you want to make sure all its output is logged to a log file. This would save the user from having to do the needed redirection on their command line – it would be done automatically by the script itself. Is this possible?
Yes! But the syntax is really weird, and I'm not sure even I can completely explain it:
Code Block | ||
---|---|---|
| ||
exec 1> >(tee my_logfile.log) 2>&1 |
As far as I can tell:
...
exercise 3
What is written to the ssout.txt file when the following is executed, and why?
Code Block | ||
---|---|---|
| ||
~/workshop/step_02.sh stdStreams 'hello world!' "goodbye world!" | tee ssout.txt
|
Expand | ||
---|---|---|
| ||
Only the standard output text is written:
Remember the pipe ("|") connects one program's standard output (here from the step_02.sh script) to the next program's standard input (here to the tee program). Standard error is ignored by the pipe (unless redirected). |
More Parts - automatic logging
So you have written a user-callable script like step_02.sh, and you want to make sure all its output is logged to a log file. This would save the user from having to do the needed redirection on their command line – it would be done automatically by the script itself. Is this possible?
Yes! But the syntax is really weird, and I'm not sure even I can completely explain it:
Code Block | ||
---|---|---|
| ||
exec 1> >(tee my_logfile.log) 2>&1 |
As far as I can tell:
- exec 1> causes redirection of all standard output for the duration of the current shell environment (here the script).
- the 2>&1 at the end is our normal "redirect standard error to standard output" idiom
- >(tee "my_logfile.log") is the magic that says to write standard output (which now includes standard error text) to a file via tee.
- the >( ) syntax also sends all the current execution environment's standard output to a sub-shell (more on this later), which
- appears to be necessary because otherwise tee would just act on its piped-in standard input.
auto_log function
The new auto_log helper function sets up automatic logging to a log file in the current directory, named using a tag string specified as its 1st argument.
...
Importantly, auto_log reports an error and terminates script execution if no tag string is specified, via exit 255 (we'll see much more on error handling shortly).
The test is made using a bash if/else/fi block, where double brackets ('"[[ ]]") enclose the test, always followed by a semicolon (";"). Here the test is for string equality (see https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-Expressions.html for a complete list of bash comparison operators).
...
Note that there must always be a space after the open brackets, and one before the close brackets.
Code Block | ||
---|---|---|
| ||
# Sets up auto-logging to a log file in the current directory # using the specified logFileTag (arg 1) in the log file name. auto_log() { local logFileTag="$1" if [[ "$logFileTag" != "" ]]; then local logFilePath="./autoLog_${logFileTag}.log" echo_se ".. logging to $logFilePath" exec 1> >(tee "$logFilePath") 2>&1 else echo_se "** ERROR in autoLogauto_log: no logFile argument provided" exit 255 fi } |
testAutolog function and command processing
The new testAutolog command processing function has a lot going on. It:
...
...
Code Block | ||
---|---|---|
| ||
# function that illustrates auto-logging and capturing function |
The general form of an if/then/else/fi statement is:
if [[ some_test ]]
then
what_to_do_when_some_test_is_true (0)
else
what_to_do_when_some_test_is_false (not 0)
fi
As always in bash, clauses (technically commands themselves) can be put on one line if separated by a semicolon ( ; ).
if [[ some_test ]]
; then echo "Test was true"; else echo "Test was false"; fi
testAutolog function and command processing
The new testAutolog command processing function has a lot going on. It:
- Starts automatic logging to a log file named using its 1st logFileTag argument, by calling the auto_log function.
- Uses echo -e where the -e argument to echo enables interpretation of backslash escapes.
- e.g. "\t" as a tab character and "\n" will be interpreted as a newline
- Calls stdStreams with its 2nd and 3rd arguments.
- Calls echo capturing its output in a local variable using backtick execution syntax, then displays the captured text.
- Calls the echo_se function with some text.
- Calls the echo_se function again, capturing its output in a local variable, then displays the captured text.
Code Block | ||
---|---|---|
| ||
# function that illustrates auto-logging and capturing function output # arg 1 - (required) tag to identify the logfile # arg 2 - (optional) text for standard output # arg 3 - (optional) text for standard error function testAutolog() { local logFileTag="$1" local outTxt=${2:-"text for standard output"} local errTxt=${3:-"text for standard error"} auto_log "$logFileTag" echo -e "\n1) Call stdStreams with output and error text:" stdStreams "$outTxt" "$outErr$errTxt" echo -e "\n2) Capture stdStreamsecho output in a variable and display it:" local output=`stdStreams "$outTxt" "$outErr"``echo $outTxt` echo -e "\tstdStreams echo output was:\n$output" echo -e "\n3) Call echo_se with someerror text:" echo_se "Some random text$errTxt" echo -e "\n4)Capture echo_se function output in a variable and display it:" output=`echo_se "Some random text$errTxt"` echo -e "echo_se output was: '$output'" } |
So why is echo -e used in the testAutoLog function? Because the -e argument to echo enables interpretation of backslash escapes. For example, "\n" will be interpreted as a newline, and "\t" as a tab character.
exercise 3
...
}
|
exercise 4
Call the testAutolog command with no further command line arguments. What happens, and why?
Expand | |||||||
---|---|---|---|---|---|---|---|
| |||||||
Executing:
produces this output:
No further code is executed after the auto_log helper function detects that no tag string has been provicedprovided, because it calls exit. |
exercise
...
5
What output is produced when you call the testAutoLog testAutolog command with a tag string of "test1".
Expand | |||||||
---|---|---|---|---|---|---|---|
| |||||||
Executing:
produces this output:
|
Why are both standard error and standard output text strings displayed in segment 1) when stdStreams is called?
Expand | ||
---|---|---|
| ||
Because stdStreams echo's to both standard error and standard output in the current execution environment, where both are directed to the terminal by auto-logging. |
Why is only standard output displayed in segment 2) when stdStreams is called with output being captureddoes the $output variable contain the output text in 2) ?
Expand | ||
---|---|---|
| ||
Because backtick evaluation creates a new execution environment, and only standard output is returned replaces the command in backticks ( `...` ) with the command's standard output. |
Why is the $output variable empty in segment 4) when echo_se "Some random text"se is called with output being captured?
Expand | ||
---|---|---|
| ||
Because backtick evaluation creates a new execution environment, and only Because only a command's standard output is returned by backtick evaluation, not standard error. |
What log file is produced, what ? What are its contents, and why?
Expand | |||
---|---|---|---|
| |||
The log file produced is autoLog_test1.log, written to the current directory in force when step_02.sh was called. This name is based on the logFileTag we specified as "test1". Its contents (below) are nearly the same as when Code Block | because that was written before automatic logging was started.
|
exercise 5
What is displayed in the terminal when you call the testAutoLog command but redirect standard output to /dev/null? What is in the log file, and why?
Expand | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
Executing:
produces only this output:
because after automatic logging is turned on, both standard output and standard error for the script are redirected to standard output by the "
|