Versions Compared

Old Version 33

changes.mady.by.user Anna Battenhouse

Saved on

compared with

New Version 34

changes.mady.by.user Anna Battenhouse

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Both the source and target directories are local (in some file system accessible directly from stampede2 lonestar6). Either full or relative path syntax can be used for both. The -avW options above stand for:

...

Tip
titleAlways add a trailing slash ( / ) after directory names

The trailing slash ( / ) on the source and destination directories are very important for rsync (and for other Linux copy commands also)!

rsync will create the last directory level for you, but earlier levels must already exist.

...

Code Block
languagebash
titlersync (local directory)
mkdir -p $SCRATCH/data
cds
rsync -avWavrW $CORENGS/custom_tracks/ data/custom_tracks/

...

Expand
titleHint
ls $SCRATCH/data/custom_tracks
# or
ls ~/scratch/data/custom_tracks
# or
cds; cd data/custom_tracks; ls
# or
tree $SCRATCH/data

Now repeat the rsync and see the difference.

...

Code Block
languagebash
rsync -avWavrW /work/projects/BioITeam/projects/courses/Core_NGS_Tools/custom_tracks/ data/custom_tracks/

...

Tip

The bash shell has several convenient line editing features:

  • use the Up arrow to scroll back through the command line history; Down arrow goes forward
  • use Ctrl-a to move the cursor to the beginning of a line; Ctrl-e to the end
  • use Backspace to remove text before the cursor; Delete to remove text after the cursor
  • use Ctrl-a and then Ctrl-k to k ("kill") to delete all text on your command line after the cursor
  • Ctrl-y ("yank") to copy the last killed text to where the cursor is

Copy from a remote computer - scp or rsync

...

Code Block
titlesingle remote file copy with scp
cat $CORENGS/tacc/dragonfly_access.txt
cds
mkdir -p data/test2
scp corengstools@dragonfly.icmb.utexas.edu:~/custom_tracks/progeria_ctcf.vcf.gz ./data/test2/
lstree .$SCRATCH/data/test2

Notes:

  • The 1st time you access a new host the SSH security prompt will appear
  • You will be prompted for your remote host password
  • The  -r recursive argument works for scp also, just like for cp

...

  • The tilde ( ~ ) at the start of the path means "relative to my home directory"
  • We use the tilde ( ~ ) in the destination to traverse the ~/scratch symbolic link in your home directory.

Code Block
languagebash
titlersync (remote directory)
cat $CORENGS/tacc/dragonfly_access.txt
rsync -avWavrW corengstools@dragonfly.icmb.utexas.edu:~/custom_tracks/ ~/scratch/data/custom_tracks/

...

Expand
titleAnswer

No, because all the source files were already present in the destination directory (you copied the same files earlier) with the same names, file sizes and timestamps. So rsync had nothing to do!

...

Expand
titleStep 3 answer

From inside your ~/what/starts/here directory:

Code Block
titlePlay a scavenger hunt for more practice
scp -r /work/projects/BioITeam/projects/courses/Core_NGS_Tools/linuxpractice/changes/ changes/
# or
rsync -ptrvP /work/projects/BioITeam/projects/courses/Core_NGS_Tools/linuxpractice/changes/ changes/
# Note: rsync -avPavrP ... will also work, but will report an error because the destination file and
# directory ownership cannot be changed to match the source. But the files will be copied, and
# ownership assigned to you.
 
# Then
cd changes 
more largeFile.txt


...

Expand
titleStep 4 answer

From inside your ~/what/starts/here/changes directory:

Code Block
titlePlay a scavenger hunt for more practice
rsync -avPavrP corengstools@dragonfly.icmb.utexas.edu:~/the/ the/
# or
scp -r corengstools@dragonfly.icmb.utexas.edu:~/the/ the/

cd the
cat instr5.txt
cd world
cat instr6.txt

The path to the directory you're in now should be:

~/what/starts/here/changes/the/world