SI2006 track 1 Community Scheduler Framework

From NBCRwiki

Jump to: navigation, search

Back to Track 1


Related Link

  1. CSF Project Page on
  2. CSF4 documents on
  3. A brief CSF4 introduction


How to get CSF4 package

  1. From (recommended)
  2. From CVS
  3. From Globus Toolkit 4 package
    • CSF4 is in the "contrib" directory, but the package may not be updated.

Required software

  1. Globus Toolkit 4
    • export GLOBUS_LOCATION=/opt/gt4
    • mkdir -p /opt/gt4
    • chown globus.globus /opt/gt4
    • ./configure --prefix=$GLOBUS_LOCATION
    • make
    • make install


remove $GLOBUS_LOCATION if a reinstall is required.

  1. jdk (1.5 is recommended)
  2. Ant (1.6.x or later)

Required configuration for GT4

WS GRAM configuration

Required for use of WS-GRAM

Consult GT4 WS GRAM

  1. visudo
  2. scheduler-adaptor for WS-GRAM
    1. SGE
  3. RFT configuration
  4. edit pg_hda.conf
  5. createuser globus


Grid Security Configuration

Consult (GSI)Grid Security Infrastructure

Generate a certificate for globus user using the GAMA server would allow you to run globus-stop-container, though not a requirement.


  1. If download from CVS:
    • Enter into packaging/
    • run this command:
    • . ./make-src-package
    • to generate the file: csf-4.0.x-src.tar.gz
    • run this command as globus admin user (the user installed gt4)
    • $ gpt-build csf-4.0.x-src.tar.gz
    • $ gpt-postinstall
  2. If download the package: csf-4.0.x-src.tar.gz
    • run this command as globus admin user (the user installed gt4)
    • $ gpt-build csf-4.0-src.tar.gz
    • $ gpt-postinstall

Note: make sure current directory is writable


  • Run these commands as globus admin user
  • $ant -f share/globus_wsrf_common/build-packages.xml undeployGar
  • $rm -rf etc/gpt/packages/setup/csf
  • $rm -rf etc/gpt/packages/csf


  • Meta-Scheduler Configure
  • edit the file $GLOBUS_LOCATION/etc/metascheduler/metascheduler-config.xml
  • CommunityGISHandle : handle of Default Index Service

For example:

  • job:pullingInterval : the Interval seconds to get Job data from Local Resource Manager
  • job:queueName : default queue name
  • queuing:plugin : scheduling policies class name
  • queuing:parameter : the parameter for scheduling policies
  • ResourceManager Configure
  • edit the file $GLOBUS_LOCATION/etc/metascheduler/resourcemanager-config.xml
    • The config file is for the resource managers other than WS GRAM, such as Pre-WS GRAM and LSF web broker.

Pre-WS GRAM Example , it is the gatekeeper on and the jobmanager is sge:

    <name> sge-32 </name>
    <type> GRAM </type>
    <host> </host>
    <port> 2119 </port>

LSF web broker Example , the web broker running on and with port 1975:

    <name> jlu_lsf01 </name>
    <type> LSF </type>
    <host> </host>
    <port> 1975</port>


Before testing

  1. Source gt4 and csf4 environment variables
    • source either $GLOBUS_LOCATION/etc/globus-user-env.csh or $GLOBUS_LOCATION/etc/
    • source either $GLOBUS_LOCATION/etc/globus-devel-env.csh or $GLOBUS_LOCATION/etc/
  2. Generate user proxy
    • $grid-proxy-init
  3. Start GT4 container as globus
    • Verify GT4 set up and test WS-GRAM accordingly before hand.
    • $globus-start-container

Test Cases

  1. Query available resource manager
    • $ csf-resource-list
  2. Create a WS GRAM job
    • $ csf-job-create -rsl $GLOBUS_LOCATION/docs/metascheduler/examples/gramjob.xml -name job1
  3. Create a Pre-WS GRAM job
    • $ csf-job-create -rsl $GLOBUS_LOCATION/docs/metascheduler/examples/job.rsl -name job2
  4. Submit job to queue
    • $ csf-job-submit job1 -q normal
  5. Submit job with full delegation
    • $ csf-job-submit job2 -q normal -d Full
  6. Query job status
    • $ csf-job-status job1
    • $ csf-job-status job2
  7. List all existing jobs
    • $ csf-job-list
  8. Get job detail information
    • $ csf-job-data job1
  9. Specify a resource for a job
    • csf-job-create -rsl job.rsl -submit -c sge32

Back to Track 1

Personal tools