We've looked at some ways of viewing text, so now we'll address how to write it.
Table of Contents |
---|
echo - the bash print function
Other programming languages usually supply a print statement or function that can direct text to files a file or to standard output.
In bash, the print utility directs text to actual printers, so there is a different method of text output: the echo command. Let's give it a try:
...
Code Block | ||
---|---|---|
| ||
echo hello # displays "hello" on the next terminal line, then the prompt on a new line
echo -n hello # displays "hello" on the next terminal line followed by the prompt (no newline) |
Exercise 3-1
What is the difference in character count when you echo hello with and without the -n option?
...
The reason for this difference is that by default, echo actually reads then outputs all the characters on the line, including the trailing linefeed. To understand this better, and to understand what is meant by "interpretation of backslash escapes", we need to first look at how the shell evaluates other metacharacters, and how that is affected by quoting in the shell.
Environment variables
Environment variables are just like variables in a programming language (in fact bash is a complete programming language): they are "names" that hold a value assigned to them. As with all programming language variables, they have two operations:
- variable definition - assign a value to a variable name
- variable reference - use the variable name to represent the value it holds
...
The shell will evaluate the varname $varname variable and substitute its value before writing output text.
...
There are a number of pre-defined environment variables in the shell, such as USER (your account name) and PATH (more on PATH later). The env command will list them all along with their values.
Exercise 3-2
Output a string that includes your account name and your Unix group name using environment variables.
Expand | ||
---|---|---|
| ||
Use env to see your built-in environment variables. You may want to pipe output to grep -i, or less -iI then search for "group" ignoring case (/group in less). |
Expand | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
Examining the env output we find that the variable MY_GROUP contains our Unix group.
|
Quoting in the shell
When the shell processes a command line, it first parses the text into tokens ("words"), which are groups of characters separated by whitespace (one or more space characters). Quoting affects how this parsing happens, including how metacharacters are treated and how text is grouped.
There are three types of quoting in the shell:
- single quoting (e.g. 'some text') – this serves two purposes
- it groups together all text inside the quotes into a single token
- it tells the shell not to "look inside" the quotes to perform anyevaluations, so evaluation
- all metacharacters inside the single quotes are ignored
- in particular, any environment variables in the single-quoted text are not evaluated
- no pathname globbing (e.g. *) is performed (more on globbing later...)
- double quoting (e.g. "some text") – also serves two purposes
- it groups together all text inside the quotes into a single token
- it allows environment variable evaluation, but inhibits pathname globbing
- some metacharcters
- e.g.
- evaluates the expression inside the backtick marks ( ` )
- the standard
- asterisk ( * ) pathname globbing (more on globbing later...)
- and some other metacharacters
- double quoting also preserves any special characters in the text
- e.g. newlines (\n) or Tabs (\t)
- backtick quoting (e.g. `date`)
- evaluates the expression inside the backtick marks ( ` )
- the standard output of the expression replaces the text inside the backtick marks ( ` )
Let's look at examples of these.
Single and double quotes
First, to see how quoting affects text grouping, The first rule of quoting is: always enclose a command argument in quotes if it contains spaces so that the command sees the quoted text as one item.
To see more on how quoting affects text grouping, we'll use quotes to define some multi-word environment variables.
Tip |
---|
Always use single ( ' ) or double ( " ) quotes when you define an environment variable whose value contains spaces. |
See the difference between:
Code Block | ||
---|---|---|
| ||
foo="My name is Anna"; echo $foo
foo='My name is Anna'; echo $foo
|
The above two expressions produce the same output because the assigned text does not contain any special metacharacters.
But these two expressions are different:
Code Block | ||
---|---|---|
| ||
foo="My USER name is $USER"; echo $foo # The text "$USER" is evaluated and its value substituted
foo='My USER name is $USER'; echo $foo # The text "$USER" is left as-is
|
Here the single quotes tell the shell to treat the quoted text as a literal, and not to look inside it for metacharacter processing.
So far so good. But what if you want text to include quotes? For example, if you want to output this text:
The value of variable 'FOO' is "$FOO"
Two common approaches:
- Use the backslash ( \ ) character that escapes the following character
- escaping means treat the next character as a literal, even if it is a special metacharacter
- Use combinations of single and double quotes.
Examples
Code Block | ||
---|---|---|
| ||
FOO="Hello world!"
echo "The value of variable 'FOO' is \"$FOO\"" # Escape the double quotes inside double quotes
echo "The value of variable 'FOO' is"' "$FOO"' # Single-quoted text after double-quoted text
echo 'The value of variable 'FOO' is "$FOO"' # Two single-quoted strings along with unquoted text FOO |
Tip |
---|
If you see the greater than ( > ) character after pressing Enter, it can mean that your quotes are not paired, and the shell is waiting for more input to contain the missing quote of the pair (either single or double). Just use Ctrl-c to get back to the prompt. |
Exercise 3-3
How would you output this text: The backslash character \ is used for escaping
...
title | Answer... |
---|
A couple of possibilities:
Code Block | ||
---|---|---|
| ||
echo 'The backslash character \ is used for escaping' # Single quotes inhibit metacharacter processing
echo "The backslash character \\ is used for escaping" # Escape the escape character |
Multi-line text
If you want to output multi-line text, you can:
- Start the text with a single or double quote
- press Enter when you want to start a new line
- keep entering text and Enter
- until you're satisfied
- enter the matching single or double quote then Enter
- Use echo -e to "enable interpretation of backslash escapes"
- Now we know what that means!
- Note that backslash escapes include some that represent non-printable characters
- e.g. newline/linefeed ( \n ), and tab ( \t )
- Use a heredoc to define a block of text (more on this later)
Exercise 3-4
How would you output this text:
My
name is
Anna
Expand | ||
---|---|---|
| ||
A couple of 'Hello world' # correct - defines variable "foo" to have value "Hello world"
foo=Hello world # error - no command called "world"
|
The 2nd expression above, without the quotes, produces an error. What's going on? The shell parses the input into two tokens: "foo=Hello" and "world". It assigns the value "Hello" to the variable foo, then tries to execute world, which it thinks is a command.
As for the difference between single quotes and double quotes, these two expressions produce the same output because the assigned text does not contain any special metacharacters:
Code Block | ||
---|---|---|
| ||
foo="My name is Anna"; echo $foo
foo='My name is Anna'; echo $foo
|
But these two expressions are different:
Code Block | ||
---|---|---|
| ||
foo="My USER name is $USER"; echo $foo # The text "$USER" is evaluated and its value substituted
foo='My USER name is $USER'; echo $foo # The text "$USER" is left as-is
|
Here the single quotes tell the shell to treat the quoted text as a literal, and not to look inside it for metacharacter processing.
So far so good. But what if you want text to include quotes? For example, if you want to output this text:
The value of variable 'FOO' is "$FOO"
Two common approaches:
- Use the backslash ( \ ) character that escapes the following character
- escaping means treat the next character as a literal, even if it is a special metacharacter
- Use combinations of single and double quotes.
Examples
Code Block | ||
---|---|---|
| ||
FOO="Hello world!"
echo "The value of variable 'FOO' is \"$FOO\"" # Escape the double quotes inside double quotes
echo "The value of variable 'FOO' is "'"'$FOO'"' # Single-quoted text after double-quoted text
|
Tip |
---|
If you see the greater than ( > ) character after pressing Enter, it can mean that your quotes are not paired, and the shell is waiting for more input to contain the missing quote of the pair (either single or double). Just use Ctrl-c to get back to the prompt. |
Note that the quote characters themselves ( ' " ` ) are metacharacters that tell the shell to "start a quoting process" then "end a quoting process" when the matching quote is found. Since they are part of the processing, the enclosing quotes are not included in the output.
Exercise 3-3
How would you output this text: The backslash character \ is used for escaping
Expand | |||||
---|---|---|---|---|---|
| |||||
A couple of possibilities:
|
Backtick evaluation quoting
backtick ( ` ) evaluation quoting is one of the underappreciated wonders of Unix.
The shell:
- evaluates the expression/command inside the backtick marks ( ` )
- the standard output of the expression replaces the text inside the backtick marks ( ` )
Examples, using the date function that just writes the current date and time to standard output, which appears on your Terminal.
Code Block | ||
---|---|---|
| ||
date # Calling the date command just displays date/time information
echo date # Here "date" is treated as a literal word, and written to output
echo `date` # The date command is evaluated and its output replaces the command |
Exercise 3-5
How would you output this text using a command to calculate the number of lines: The haiku.txt file has 11 lines
Expand | ||
---|---|---|
| ||
Use backtick evaluation and wc -l |
...
title | Answer... |
---|
Code Block | ||
---|---|---|
| ||
echo "The haiku.txt file has `cat haiku.txt | wc -l` lines" |
Notice that the backticked expression can be complex!
Redirection
So far text we've been working with has been output to standard output, which I keep reminding you is mapped to your Terminal. But you can redirect text elsewhere.
Recall the three standard Unix streams: they each have a number, a name and redirection syntax:
- standard output is stream 1
- redirect standard output to a file with a the > or 1> operator
- a single > or 1> overwrites any existing data in the target file
- a double >> or 1>> appends to any existing data in the target file
- redirect standard output to a file with a the > or 1> operator
- standard error is stream 2
- redirect standard error to a file with a the 2> operator
- a single 2> overwrites any existing data in the target file
- a double 2>> appends to any existing data in the target file
- redirect standard error to a file with a the 2> operator
...
|
Multi-line text
If you want to output multi-line text, you can:
- Start the text with a single or double quote
- press Enter when you want to start a new line
- keep entering text and Enter
- until you're satisfied
- enter the matching single or double quote then Enter
- Use echo -e to "enable interpretation of backslash escapes"
- Now we know what that means!
- Note that backslash escapes include some that represent non-printable characters
- e.g. newline/linefeed ( \n ), and tab ( \t )
Exercise 3-4
How would you output this text:
My
name is
Anna
Expand | |||||
---|---|---|---|---|---|
| |||||
A couple of possibilities:
|
Backtick evaluation quoting
backtick ( ` ) evaluation quoting is one of the underappreciated wonders of Unix.
The shell:
- evaluates the expression/command inside the backtick marks ( ` )
- the standard output of the expression replaces the text inside the backtick marks ( ` )
Examples, using the date function that just writes the current date and time to standard output, which appears on your Terminal.
Code Block | ||
---|---|---|
| ||
date # Calling the date command just displays date/time information
echo date # Here "date" is treated as a literal word, and written to output
echo `date` # The date command is evaluated and its output replaces the command
# Assign a string including today's date to variable "today"
today="Today is: `date`"; echo $today |
The slightly odd syntax $(<some command>), is equivalent to `<some command>`, and can be easier to read when the command to be evaluated is complex.
Code Block | ||
---|---|---|
| ||
echo "Today is: `date`"
echo "Today is $(date)"
|
Exercise 3-5
How would you output this text using a command to calculate the number of lines: The haiku.txt file has 11 lines
Expand | ||
---|---|---|
| ||
Use backtick evaluation and wc -l |
Expand | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
Notice that the backticked expression can be complex! |
Redirection
So far text we've been working with output to standard output, which I keep reminding you is mapped to your Terminal. But you can redirect text elsewhere.
Recall the three standard Unix streams: they each have a number, a name and redirection syntax:
- standard output is stream 1
- redirect standard output to a file with a the > or 1> operator
- a single > or 1> overwrites any existing data in the target file
- a double >> or 1>> appends to any existing data in the target file
- redirect standard output to a file with a the > or 1> operator
- standard error is stream 2
- redirect standard error to a file with a the 2> operator
- a single 2> overwrites any existing data in the target file
- a double 2>> appends to any existing data in the target file
- redirect standard error to a file with a the 2> operator
Some standard output examples:
Code Block | ||
---|---|---|
| ||
echo hello > out.txt cat out.txt # Displays "hello" echo world 1>> out.txt # Appends "world" to out.txt cat out.txt # Displays "hello" "world" on 2 lines echo goodbye world 1>out.txt # Overwrites out.txt file cat out.txt # Displays "goodbye world" |
Notice when using redirection, the output does not appear on the Terminal, it only goes to the specified file.
If you want output to go to both the Terminal and a file, you can use the tee command:
Code Block | ||
---|---|---|
| ||
student01@gsafcomp01:~$ echo Hello world! | tee out.txt Hello world! student01@gsafcomp01:~$ cat out.txt# Hello world! |
You can also specify the tee -a option to append the input text to the file you specify.
The standard error stream
So what's this standard error stream? Recall our discussion of Command input errors? Well, error information is written to standard error, not to standard output!
It is easy to not notice the difference between standard output and standard error when you're in an interactive Terminal session – because both outputs are sent to the Terminal. But they are separate streams, with different meanings. When executing commands you will want to manipulate standard output and standard error appropriately – especially for 3rd party programs.
...
Displays "goodbye world" |
Notice when using redirection, the output does not appear on the Terminal, it only goes to the specified file.
If you want output to go to both the Terminal and a file, you can use the tee command:
Code Block | ||
---|---|---|
| ||
ls haiku.txt xxx.txt |
Produces this output in your Terminal:
Code Block |
---|
ls: cannot access 'xxx.txt': No such file or directory
haiku.txt |
What is not obvious, since both streams are displayed on the Terminal, is that:
- the diagnostic text "ls: cannot access 'xxx.txt': No such file or directory" is being written to standard error
- the listing of the existing file ("haiku.txt") is being written to standard output
To see this, redirect standard output and standard error to different files and look at their contents:
Code Block | ||
---|---|---|
| ||
ls haiku.txt xxx.txt 1> stdout.txt 2>stderr.txt
cat stdout.txt # Displays "haiku.txt"
cat stderr.txt # Displays "ls: cannot access 'xxx.txt': No such file or directory" |
What if you want both standard output and standard error to go to the same file? You use this somewhat odd 2>&1 redirection syntax:
Code Block | ||
---|---|---|
| ||
# Redirect both standard output and standard error to the out.txt file student01@gsafcomp01:~$ student01@gsafcomp01:~$ echo Hello world! | tee out.txt Hello world! student01@gsafcomp01:~$ cat out.txt Hello world! |
You can also specify the tee -a option to append the input text to the file you specify.
Note that the > redirection metacharacter sends its output to a file, not to another program's standard input stream as with the | pipe metacharacter. (There are some cases where redirection involves something other than a file, but that's a topic for the Advanced Bash scripting class.)
The standard error stream
So what's this standard error stream? Recall our discussion of Command input errors? Well, error information is written to standard error, not to standard output!
It is easy to not notice the difference between standard output and standard error when you're in an interactive Terminal session – because both outputs are sent to the Terminal. But they are separate streams, with different meanings.
When executing commands you will want to manipulate standard output and standard error appropriately – especially for 3rd party programs.
Let's look at a command that shows the difference between standard error and standard output:
Code Block | ||
---|---|---|
| ||
ls haiku.txt xxx.txt > out.txt 2>&1
# Display the contents of the out.txt file
student01@gsafcomp01:~$ cat out.txt
ls: cannot access |
Produces this output in your Terminal:
Code Block |
---|
ls: cannot access 'xxx.txt': No such file or directory
haiku.txt |
What is not obvious, since both streams are displayed on the Terminal, is that:
- the diagnostic text "ls: cannot access 'xxx.txt':
...
- No
...
- such
...
- file
...
- or
...
One final redirection trick. There is a special Linux file called /dev/null that serves as a "global trash can". That is – it just throws away anything you write to it. So you can direct standard output and/or standard error to /dev/null to ignore it completely.
Exercise 3-6
...
- directory" is being written to standard error
- the listing of the existing file ("haiku.txt") is being written to standard output
To see this, redirect standard output and standard error by redirecting standard error to /dev/null.
...
title | Answer... |
---|
...
to different files and look at their contents:
Code Block | ||
---|---|---|
| ||
ls haiku.txt xxx.txt 2>/dev/null |
Show the difference between standard output and standard error by redirecting standard output to /dev/null.
Expand | |||||
---|---|---|---|---|---|
| |||||
This will only display "ls: cannot access 1> out.txt 2>err.txt cat out.txt # Displays "haiku.txt" cat err.txt # Displays "ls: cannot access 'xxx.txt':No such file or directory"
|
Writing multiple text lines using a heredoc
In addition to the methods of writing multi-line text previously discussed, there's another one that can be useful for composing a large block of text for output to a file. This is done using the heredoc syntax to define a block of text between two user-supplied block delimiters, sending the text to a specified command.
...
No such file or directory" |
What if you want both standard output and standard error to go to the same file? You use this somewhat odd 2>&1 redirection syntax:
Code Block | ||
---|---|---|
| ||
COMMAND << DELIMITER
..text...
..text...
DELIMITER |
For example, using the delimiter EOF and the cat command:
Code Block | ||
---|---|---|
| ||
cat << EOF
This text will be output
And this USER environment variable will be evaluated: $USER
EOF |
...
# Redirect both standard output and standard error to the out.txt file
student01@gsafcomp01:~$ ls haiku.txt xxx.txt > out.txt 2>&1
# Display the contents of the out.txt file
student01@gsafcomp01:~$ cat out.txt
ls: cannot access 'xxx.txt': No such file or directory
haiku.txt |
One final redirection trick. There is a special Linux file called /dev/null that serves as a "global trash can". That is – it just throws away anything you write to it. So you can direct standard output and/or standard error to /dev/null to ignore it completely.
Exercise 3-6
Show the difference between standard output and standard error by redirecting standard error to /dev/null.
Expand | |||||
---|---|---|---|---|---|
| |||||
This will only display "haiku.txt"
|
The out.txt file will then contain this text:
Code Block | ||
---|---|---|
| ||
This text will be output
And this USER environment variable will be evaluated: student01
|
Tip |
---|
The 2nd (ending) block delimiter you specify for a heredoc must appear at the start of a new line. |
Editing text
We've now covered viewing existing file text and writing new text to a file. But what if you want to edit/change text in an existing file?
There are two main approaches to editing Unix files:
...
- nano is extremely simple and is a good choice as a first local text editor
- warning: nano has a tendency to break long single lines into multiple lines
- vi and emacs are extremely powerful but also quite complex
- emacs reference sheet: https://www.gnu.org/software/emacs/refcards/pdf/refcard.pdf
- vi reference sheet: http://www.atmos.albany.edu/daes/atmclasses/atm350/vi_cheat_sheet.pdf
...
- Once mounted, the remote storage appears as a local volume/drive.
- Then, you can use any text editor or IDE on your local computer to open/edit/save remote files, although it may be slower than local file editing
- Software programs that can mount remote data include ExpanDrive for Windows or Mac (costs $$, but has a free trial), TextWrangler for Mac.
- Remote file system protocols include Samba (Windows, Mac) and NFS (Linux)
Knowing the basics of at least one Linux command-line text editor is useful for creating/editing small files, and we'll explore nano in this class. For editing larger files, you may find options #2 or #3 more useful.
nano
nano is a very simple editor available on most Linux systems. If you are able to ssh into a remote system, you can use nano there.
To invoke nano to edit a new or existing file just type nano <filename>. For example:
Code Block | ||
---|---|---|
| ||
nano newfile.txt
|
You'll see the name of the file (if you supplied one) on the top line of the Terminal window.
You can just type in text, and navigate around using arrow keys (up/down/left/right). A couple of other navigation shortcuts:
- Ctrl-a - go to start of line
- Ctrl-e - go to end of line
Once you've positioned the cursor where you want it, just type in your text.
To remove text:
- the Backspace key deletes the character before the cursor
- the Delete key, or Ctrl-d deletes the character after the cursor
- Ctrl-k deletes all text on the current line after the cursor
Once you're satisfied with your edits:
- use Ctrl-o - write out the file
- use Ctrl-x - exit nano
...
|
Show the difference between standard output and standard error by redirecting standard output to /dev/null.
Expand | |||||
---|---|---|---|---|---|
| |||||
This will only display "ls: cannot access 'xxx.txt': No such file or directory"
|
Editing text
We've now covered viewing existing file text and writing new text to a file. But what if you want to edit/change text in an existing file?
There are three main approaches to editing Unix files:
- Use a command-line program that lets you enter/edit text in a Terminal window (e.g. nano, vi/vim, emacs)
- nano is extremely simple and is a good choice as a first local text editor
- warning: nano has a tendency to break long single lines into multiple lines
- vi and emacs are extremely powerful but also quite complex
- emacs reference sheet: https://www.gnu.org/software/emacs/refcards/pdf/refcard.pdf
- vi reference sheet: http://www.atmos.albany.edu/daes/atmclasses/atm350/vi_cheat_sheet.pdf
- nano is extremely simple and is a good choice as a first local text editor
- Use a text editor or IDE (Integrated Development Environment) program that runs on your local computer but has an SFTP (secure FTP) interface that lets you connect to a remote computer
- E.g.,Komodo IDE (Windows & Mac) or Notepad++ (Windows). Both are no-cost.
- Once you connect to the remote host, you can navigate its directory structure and edit files.
- When you open a file, its contents are brought over the network into the text editor's edit window, then saved back when you save the file.
- E.g.,Komodo IDE (Windows & Mac) or Notepad++ (Windows). Both are no-cost.
- Use software or protocols that allow you to "mount" remote server directories
- Once mounted, the remote storage appears as a local volume/drive.
- Then, you can use any text editor or IDE on your local computer to open/edit/save remote files.
- Software programs that can mount remote data include ExpanDrive for Windows or Mac (costs $$, but has a free trial), TextWrangler for Mac.
- Remote file system protocols include Samba (Windows, Mac) and NFS (Linux)
- Once mounted, the remote storage appears as a local volume/drive.
Knowing the basics of at least one Linux command-line text editor is useful for creating/editing small files, and we'll explore nano in this class. For editing larger files, you may find options #2 or #3 more useful.
nano
nano is a very simple editor available on most Linux systems. If you are able to ssh into a remote system, you can use nano there.
To invoke nano to edit a new or existing file just type nano <filename>. For example:
Code Block | ||
---|---|---|
| ||
nano newfile.txt
|
You'll see the name of the file (if you supplied one) on the top line of the Terminal window.
Navigation and operations in nano are similar to those we discussed in Command line editing.
You can just type in text, and navigate around using arrow keys (up/down/left/right). A couple of other navigation shortcuts:
- Ctrl-a - go to start of line
- Ctrl-e - go to end of line
- Ctrl-v - go forward one page
- Arrow keys are also modified by Ctrl- (Windows) or Option- (Mac)
- Ctrl-right-arrow (Windows) or Option-right-arrow (Mac) will skip by "word" forward
- Ctrl-left-arrow (Windows) or Option-left-arrow (Mac) will skip by "word" backward
Once you've positioned the cursor where you want it, just type in your text.
To remove text:
- To delete text after the cursor, use:
- Delete key on Windows
- Function-Delete keys on Macintosh
- To delete text before the cursor, use:
- Backspace key on Windows
- Delete key on Macintosh
- Use Ctrl-k (kill) to delete everything on the line
- This is different from Ctrl-k on the command line where it deletes everything after the cursor
Once you're satisfied with your edits:
- use Ctrl-o - write out the file
- use Ctrl-x - exit nano
These and other important nano operations are displayed in a menu at the bottom of the Terminal window. Note that the ^ character means Ctrl- in this menu.
Edit your login script
Whenever you login to a remote server, a login script located in your Home directory is executed. This file, usually your ~/.profile and/or ~/.bashrc file, has expressions that customize your shell environment. These customizations are temporary – they are in effect only during your login sessions, which is why they have to be re-established every time you login.
On our system, ~/.profile file is your login script. Use more ~/.profile to look at it.
Exercise 3-7
Use nano to remove the pound sign ( # ) comment character from one of the two lines starting with #export LS_COLORS
After you save the file and exit nano, use the exit command to end your current Terminal session. Then log back on to the GSAF pod server to see that the colors used by ls are changed without you having to change the LS_COLORS environment variable on the command line.