edu.isi.pegasus.planner.refiner.createdir
Class HourGlass

java.lang.Object
  edu.isi.pegasus.planner.refiner.createdir.AbstractStrategy
      edu.isi.pegasus.planner.refiner.createdir.HourGlass

All Implemented Interfaces:

Strategy

public class HourGlass
extends AbstractStrategy

This class inserts the nodes for creating the random directories on the remote execution pools. This is done when the resources have already been selected to execute the jobs in the Dag. It adds a make directory node at the top level of the graph, and all these concat to a single dummy job before branching out to the root nodes of the original/ concrete dag so far. So we end up introducing a classic X shape at the top of the graph. Hence the name HourGlass.

Version:

$Revision: 4553 $

Author:

Karan Vahi, Gaurang Mehta

Field Summary

static String	COMPLETE_TRANSFORMATION_NAME The complete TC name for dirmanager.
static String	DERIVATION_NAME The logical name of the transformation that creates directories on the remote execution pools.
static String	DERIVATION_NAMESPACE The derivation namespace for the create dir jobs.
static String	DERIVATION_VERSION The version number for the derivations for create dir jobs.
static String	DUMMY_CONCAT_JOB The name concatenating dummy job that ensures that Condor does not start staging in before the directories are created.
static String	DUMMY_CONCAT_JOB_PREFIX The prefix assigned to the concatenating dummy job that ensures that Condor does not start staging in before the directories are created.
static String	TRANSFORMATION_NAME The logical name of the transformation that creates directories on the remote execution pools.
static String	TRANSFORMATION_NAMESPACE The transformation namespace for the create dir jobs.
static String	TRANSFORMATION_VERSION The version number for the derivations for create dir jobs.

Fields inherited from class edu.isi.pegasus.planner.refiner.createdir.AbstractStrategy

CREATE_DIR_PREFIX, CREATE_DIR_SUFFIX, mImpl, mJobPrefix, mLogger, mSiteStore, mUseMkdir

Fields inherited from interface edu.isi.pegasus.planner.refiner.createdir.Strategy

VERSION

Constructor Summary

HourGlass()

Method Summary

ADag	addCreateDirectoryNodes(ADag dag) It modifies the concrete dag passed in the constructor and adds the create random directory nodes to it at the root level.
private void	construct(Job job, String key, String value) Constructs a condor variable in the condor profile namespace associated with the job.
protected String	getConcatJobname(ADag dag) Returns the name of the concat job
void	initialize(PegasusBag bag, Implementation impl) Intializes the class.
private void	introduceRootDependencies(ADag dag, String newRoot) It traverses through the root jobs of the dag and introduces a new super root node to it.
Job	makeDummyConcatJob(ADag dag) It creates a dummy concat job that is run at the local submit host.

Methods inherited from class edu.isi.pegasus.planner.refiner.createdir.AbstractStrategy

getCreateDirJobName, getCreateDirSites

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DUMMY_CONCAT_JOB

public static final String DUMMY_CONCAT_JOB

The name concatenating dummy job that ensures that Condor does not start staging in before the directories are created.

See Also:

Constant Field Values

DUMMY_CONCAT_JOB_PREFIX

public static final String DUMMY_CONCAT_JOB_PREFIX

The prefix assigned to the concatenating dummy job that ensures that Condor does not start staging in before the directories are created.

See Also:

Constant Field Values

TRANSFORMATION_NAMESPACE

public static final String TRANSFORMATION_NAMESPACE

The transformation namespace for the create dir jobs.

See Also:

Constant Field Values

TRANSFORMATION_NAME

public static final String TRANSFORMATION_NAME

The logical name of the transformation that creates directories on the remote execution pools.

See Also:

Constant Field Values

TRANSFORMATION_VERSION

public static final String TRANSFORMATION_VERSION

The version number for the derivations for create dir jobs.

COMPLETE_TRANSFORMATION_NAME

public static final String COMPLETE_TRANSFORMATION_NAME

The complete TC name for dirmanager.

DERIVATION_NAMESPACE

public static final String DERIVATION_NAMESPACE

The derivation namespace for the create dir jobs.

See Also:

Constant Field Values

DERIVATION_NAME

public static final String DERIVATION_NAME

The logical name of the transformation that creates directories on the remote execution pools.

See Also:

Constant Field Values

DERIVATION_VERSION

public static final String DERIVATION_VERSION

The version number for the derivations for create dir jobs.

See Also:

Constant Field Values

Constructor Detail

HourGlass

public HourGlass()

Method Detail

initialize

public void initialize(PegasusBag bag,
                       Implementation impl)

Intializes the class.

Specified by:

initialize in interface Strategy

Overrides:

initialize in class AbstractStrategy

Parameters:

bag - bag of initialization objects

impl - the implementation instance that creates create dir job

addCreateDirectoryNodes

public ADag addCreateDirectoryNodes(ADag dag)

It modifies the concrete dag passed in the constructor and adds the create random directory nodes to it at the root level. These directory nodes have a common child that acts as a concatenating job and ensures that Condor does not start staging in the data before the directories have been added. The root nodes in the unmodified dag are now chidren of this concatenating dummy job.

Parameters:

dag - the workflow to which the nodes have to be added.

Returns:

the added workflow

introduceRootDependencies

private void introduceRootDependencies(ADag dag,
                                       String newRoot)

It traverses through the root jobs of the dag and introduces a new super root node to it.

Parameters:

dag - the DAG

newRoot - the name of the job that is the new root of the graph.

makeDummyConcatJob

public Job makeDummyConcatJob(ADag dag)

It creates a dummy concat job that is run at the local submit host. This job should run always provided the directories were created successfully.

Parameters:

dag - the workflow

Returns:

the dummy concat job.

getConcatJobname

protected String getConcatJobname(ADag dag)

Returns the name of the concat job

Returns:

name

construct

private void construct(Job job,
                       String key,
                       String value)

Constructs a condor variable in the condor profile namespace associated with the job. Overrides any preexisting key values.

Parameters:

job - contains the job description.

key - the key of the profile.

value - the associated value.