/
2012.06.28 Range Maps

2012.06.28 Range Maps

June 28, 2012 Parametric launcher

Participants

John Donoghue, Edwin Skidmore, Martha Narro

Agenda

Edwin teaching John how to submit a job to TACC using the parametric launcher

General Info

  • Module launcher
  • Login to longhorn
  • A module is a command that loads programs to be used

File systems

  • In HPC, there are three work locations (file systems):
    • Home dir <cd or cd $HOME>
    • Work dir
    • Scratch dir <cds or cd $SCRATCH>
  • Longhorn directly exposes home and scratch, but not work
  • There’s no quota on scratch, but those files will get deleted fairly often, as often as weekly.
  • Depends on last time files were accessed.
  • If you need to keep some files for a while, log in daily and touch all the files.
  • For example in scratch directory:
    • cds
    • touch
    • find . --exe touch {} \:
  • John needs to write to scratch since his scripts generate lots of output files.
  • Put all files in the iPlant Data Store.
  • Use iget to move them to Longhorn.
  • Since the total size of input files is small, they could go in the home directory.
  • If it was large, would need to put them in scratch.

Edwin's Example Scripts

launcher1

  • Launcher1 is the submit script
  • Read the batch queue info at http://www.tacc.utexas.edu/user-services/user-guides/longhorn-user-guide . (see “Running your Applications)
  • For the first large scale range model run, Edwin used the normal queue
  • There is a priority for long queue jobs.
  • There’s a limit to the number of jobs a user can run at a time.
    • The others are held in queue
  • The # alone indicates a comment.
  • If # is followed by $ or @ it indicates parameters.
    • # comment
    • #$, #@ are parameters
  • job – one or more tasks
  • -pe 8way queues 8 jobs at once
  • Edwin had to submit another job with tasks that didn’t finish.
  • module load launcher - sets up the environment
  • qsub - sumits job to the queue
  • qstat – tells you if job is running, where, how long, etc.
  • Parameters in launcher1 are defined in longhorn documentation (user guide)
  • Edwin suggested John run a hello world version of submitting a job. Use the development node (queue).
  • Use 8way 8

paramlist

  • Lists all the tasks for a given job.
  • >& pipes the output 

genparam

  • generates the paramlist
  • change paramlist1 to whatever number 

paramlist-tar

  • tar
  • tar as if in output dir compress dir boundingbox 
  • Test runs
  • run 8-16 on a paramlist on development queue to see it runs
  • run RSCRIPT command (no interactive shell) 
  • xsede.org
  • user portal
  • my xsede 
  • tg-login.longhorn.tacc.teragrid.org 
  • Note the help and helpdesk
  • can login using ssh client