SI2007 track1 Tutorial Using CSF4 and OSG

From NBCRwiki

Jump to: navigation, search

Back to Track 1

This tutorial explains how to use CSF4 and OSG to Access PRAGMA and NBCR Resources

Contents

CSF4

CSF4 download

  1. From Sourceforge.net (recommended)
  2. From CVS
  3. From Globus Toolkit 4 package; CSF4 is in the "contrib" directory, but the package may not be updated.

Required software

  1. Globus Toolkit 4
  2. jdk (1.5 is recommended)
  3. Ant (1.6.x or later)

Installation

Download the package csf-4.0.x-src.tar.gz. Run the two commands as globus admin user (the user installed gt4):

gpt-build csf-4.0-src.tar.gz (less than 1 minute)
gpt-postinstall

Note: make sure current directory is writable

Configure PRAGMA Resource

Resource Manager Configure: edit the file $GLOBUS_LOCATION/etc/metascheduler/resourcemanager-config.xml

   <cluster>
    <name> PRAGMA_SDSC_52 </name>
    <type> GRAM </type>
    <host> rocks-52.sdsc.edu/jobmanager-sge </host>
    <port> 2119 </port>
    <version>2.4</version>
   </cluster>
   
   <cluster>
    <name> NBCR_Mirume </name>
    <type> GRAM </type>
    <host> mirume.nbcr.net/jobmanager-fork </host>
    <port> 2119 </port>
    <version>2.4</version>
   </cluster>

Testing

Source gt4 and csf4 environment variables

source $GLOBUS_LOCATION/etc/globus-user-env.sh
source $GLOBUS_LOCATION/etc/globus-devel-env.sh

Query available resource manager

csf-resource-list

Edit a job description file named autodock.rsl

&(executable=/home/wilfred/autodocksuite-4.0.1/bin/i86Linux2/autogrid4)
(arguments="-p" "/home/wilfred/hsg1.gpf")
(stdin=/home/wilfred/hsg1.gpf)
(stdout=/home/wilfred/hsg1.glg)
(stderr=/home/wilfred/error)

Submit job with stagein and stageout

csf-job-create -rsl autodock.rls -name myjob1 -stagein -stageout -submit

List existing jobs

csf-job-list

View Job Output

less hsg1.glg

OSG

Installation OSG Client

wget http://physics.bu.edu/pacman/sample_cache/tarballs/pacman-latest.tar.gz
tar xvzf pacman-latest.tar.gz
source pacman-3.xx/setup.sh (xx is a version number)
mkdir ~/osg_client
cd ~/osg_client
pacman -get OSG:client

Start Condor

Source OSG Environment Variable

source ~/osg_client/setup.sh

Start Condor

condor_master

Query Condor Daemon

ps -ef | grep condor

Write a Job Submission Script

Edit a file name myjdl

 universe = globus
 #executable on your machine (this will be transferred)
 Executable = /home/zding/autodocksuite-4.0.1/bin/i86Linux2/autogrid4
 Arguments = -p hsg1.gpf
 #queue you want to use
 GlobusScheduler= pebble.nbcr.net/jobmanager-sge
 #other files you want to transfer
 transfer_input_files = /home/zding/hsg1.gpf,/home/zding/hsg1.pdbqs
 should_transfer_files = YES
 when_to_transfer_output = ON_EXIT
 #output file condor will write to on your local machine.
 Output = /home/zding/condor.out
 Error = /home/zding/condor.err
 Log = /home/zding/condor.log
 stream_output = FALSE
 stream_error = FALSE
 ENABLE_GRID_MONITOR = TRUE
 Queue

Source OSG setup.sh

source /export/osg_client/setup.sh

Submit the job

condor_submit myjdl

Query the job

condor_q

View the job output

less /home/zding/condor.out

Back to Track 1

Personal tools