Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
$ nohup work_queue_worker -N wq_test_${USER} --cores all --debug-rotate-max=0 -d all -o worker.dbg > log_file_2.txt 2>&1 &
  • -N wq_test_${USER} sets the project name to maker_run_test. This is mandatory if we need to run WQ-MAKER. This is the same id that we have specified with MASTER

  • --debug-rotate-max=0 Set the maximum size of the debug log (default 10M, 0 disables)

  • -d all Sets the debug flag for Work Queue. For all debugging output, try 'all'
  • -o worker.dbg Sets the debug file for Work Queue

  • --cores all Uses all the cores on the machine
Note
titleAdvanced users

You can use Anisble method to launch jobs without ssh'ing into WORKERS from the MASTER itself. Once the maker run is started on the master, and once your WORKERS are in active state 

Step 1: Copy ansible.cfg file into your home directory which will help you to avoid host verification

Code Block
$ cp /opt/WQ-MAKER_example_data/.ansible.cfg ~

Step 2. Add ssh keys of MASTER to the JetStream atmosphere. This will allow Ansible to launch the jobs without ssh into the WORKERS.

Step 3: Copy maker-hosts file into your working directory and populate it with ip addresses of the WORKERS

Code Block
$ cp /opt/WQ-MAKER_example_data/maker-hosts .
$ echo "149.165.169.21" >> maker-hosts # This ip address of the WORKER is specific to my account. This will not work for you
$ echo "149.165.169.78" >> maker-hosts # This ip address of the WORKER is specific to my account. This will not work for you

Step 34: Copy the Ansible playbook to your working directory

Code Block
$ cp /opt/WQ-MAKER_example_data/worker-launch.yml .
$ cat worker-launch.yml

---
- hosts : workers
  environment:
    PATH: "{{ ansible_env.PATH }}:/home/${USER}/bin:/home/${USER}/.local/bin:/opt/icommands:/opt/icommands:/opt/exonerate-2.2.0-x86_64/bin/:/opt/cctools/bin:/opt/ncbi-blast-2.6.0+/bin/:/opt/snoscan-0.9.1/:/opt/tRNAscan-SE-1.3.1/:/opt/snap/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/augustus-3.2.2/bin:/opt/maker/bin:/opt/RepeatMasker:/opt/snap"
    PERL5LIB: "/opt/tRNAscan-SE-1.3.1::/opt/cctools/lib/perl5/site_perl"
  tasks :
  - name : Execute the script
    shell : /opt/cctools/bin/work_queue_worker -N wq_test_${USER} -s /home/${USER} --cores all --debug-rotate-max=0 -d all -o /home/${USER}/worker.dbg
  • - hosts is the name of the hosts (workers in this case. It can be anything)
  • tasks is the task that need to be performed by the Ansible (In this case run work_queue_worker)
  • name is just name of the task (It can be anything) 
  • -N maker_run_test sets the project name to maker_run_test. This is mandatory if we need to run WQ-MAKER
  • -s /home/upendra/ Set the location for creating the working directory of the worker

  • --debug-rotate-max=0 Set the maximum size of the debug log (default 10M, 0 disables)

  • -d all Sets the debug flag for Work Queue. For all debugging output, try 'all'
  • -o worker.dbg Sets the debug file for Work Queue

Step 45: Run WQ-MAKER on the WORKERS

Code Block
 $ nohup ansible-playbook -u ${USER} -i maker-hosts worker-launch.yml > log_file_2.txt 2>&1 &

...