Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Open a new Terminal (Mac) or Command Prompt (Window) window on your local computer (not logged in to your student account), and try the following, using your studentNN account and GSAF pod host. 

Note that you will always be prompted for your credentials on the remote host when you execute an scp command.

To copy a remote file:

Code Block
languagebash
titlescp a single file
# On your local computer - not gsafcomp01 or gsafcomp02
# Be sure to use your assigned student account and hostname

# copy "haiku.txt" from your remote student Home directory to your current local directory
scp student01@gsafcomp01.ccbb.utexas.edu:~/haiku.txt . 

# copy "haiku.txt", now in your local current directory, to your remote student 
# Home directory with the name "haiku2.txt"
scp ./haiku.txt student01@gsafcomp01.ccbb.utexas.edu:~/haiku2.txt

To copy a remote directory:

Code Block
languagebash
titlescp a directory
# On your local computer - not gsafcomp01 or gsafcomp02  # Be sure to use your assigned student account and hostname
 
# copy the "docs" directory and its contents from your remote student Home directory 
# to a local sub-directory called "local_docs"
scp -r student01@gsafcomp01.ccbb.utexas.edu:~/docs/ ./local_docs/

# copy the "local_docs" sub-directory in your local current directory, to your 
#  remote student Home directory with the name "remote_docs"
scp -r ./local_docs/ student01@gsafcomp01.ccbb.utexas.edu:~/remote_docs/ -r ./local_docs/ student01@gsafcomp01.ccbb.utexas.edu:~/remote_docs/


Tip

When transferring files between your computer and a remote server, you always need to execute the command on your computer. This is because your personal computer does not have an entry in the global hostname database (a.k.a. the Domain Name Service, or DNS), whereas the remote computer does.

wget (web get)

The wget <url> command lets you retrieve the contents of a valid Internet URL (e.g. http, https, ftp).

...

Code Block
# Make a new "wget" directory in your student Home directory and change into it
mkdir -p ~/wget; cd ~/wget

# download a Gencode statistics file using default output file naming
wget "https://ftp.ebi.ac.uk/pub/databases/gencode/_README_stats.txt"
wc -l _README_stats.txt

# if you execute the same wget again, and the output file already exists
# wget will create a new one with a numeric extension
wget "https://ftp.ebi.ac.uk/pub/databases/gencode/_README_stats.txt"
wc -l _README_stats.txt.1*

# download the same Gencode statistics file to a different local filename
wget -O gencode_stats.txt "https://ftp.ebi.ac.uk/pub/databases/gencode/_README_stats.txt"
wc -l gencode_stats.txt

...

  • looks for files matching <expression> in <in_directory> and its sub-directories
  • <expression> can be a double-quoted string including pathname wildcards (e.g. "[a-g]*.txt")
  • there are tons of operators and tests:
    • -type f (file) and -type d (directory) are useful tests
    • -maxdepth NNis a useful operator to limit the depth of recursion.
  • returns a list of matching relative pathnames, relative to in the <in_directory>, one per output line.

...

Expand
titleAnswer...

find /stor/work/CBRS_unix/fastq/ -name "*2020-07-10*" -type d  | wc -l
reports 2 directory paths

...

  • ln -s <path> says to create a symbolic link (symlink) to the specified file (or directory) in the current directory
    • always use the -s option to avoid creating a hard link, which behaves quite differently
  • the default link name corresponds to the last name component in <path>
    • you can name the link file differently by supplying an optional link_file_name.
  • it is best to change into (cd) the directory where you want the link before executing ln -s
  • a symbolic link can be deleted without affecting the linked-to file
  • the -f (force) option says to overwrite any existing file symbolic link with the same name

Examples:

...

  • The 10-character permissions field (lrwxrwxrwx) has an l in the left-most file type position, indicating this is a symbolic link.
  • The symlink itself is colored differently – in cyan
  • There are two extra fields after the symlink name
    • field 10 has an arrow -> pointing to ...field 11
    • field 11 the path of the linked-to file ("../haiku.txt")

...