...
Stampede2 is a collection of 4,200 KNL nodes (computers), 1736 SKX nodes (computers) connected to three file servers, each with unique characteristics.
You need to understand the file servers to know how to use them.
...
| Code Block |
|---|
cp $WORK2/my_fastq_data/*fastq $SCRATCH/my_project/ |
Stampede2's /home and /scratch file systems are mounted and visible only on Stampede2, but the work file system mounted on Stampede2 is part of the global file system hosted on Stockyard. This means /work and /work2 are visible on common among TACC clusters : lonestar5, stampede2, and frontera.
General Guidelines to reduce File I/O load on TACC:
TACC staff now recommends that you run your jobs out of the $SCRATCH file system instead of the global $WORK file system. Copy input files to $SCRATCH, run analyses, output should be written to $SCRATCH. Copy results to $WORK when done.
General Guidelines to when transferring files to/from TACC:
- Don't do too many (>3) simultaneous file transfers.
- If you need to transfer recursive files (directories within directories, lots of small files), create a TAR archive before transferring.
Now let's go on to look at how jobs are run on stampede2.