edu.isi.pegasus.planner.cluster.aggregator
Class Abstract

java.lang.Object
  edu.isi.pegasus.planner.cluster.aggregator.Abstract

All Implemented Interfaces:

JobAggregator

Direct Known Subclasses:

MPIExec, SeqExec

public abstract class Abstract
extends Object
implements JobAggregator

An abstract implementation of the JobAggregator interface, which the other implementations can choose to extend.

Version:

$Revision: 4737 $

Author:

Karan Vahi vahi@isi.edu

Field Summary

static String	CLUSTERED_JOB_PREFIX The prefix that is assigned to the jobname of the collapsed jobs to get the jobname for the fat job.
static String	DERIVATION_NAMESPACE The derivation namespace for the cluster jobs.
static String	DERIVATION_VERSION The version number for the derivations for cluster jobs.
protected PegasusBag	mBag Bag of initialization objects.
protected ADag	mClusteredADag The handle to the ADag object that contains the workflow being clustered.
protected String	mDirectory The directory, where the stdin file of the fat jobs are created.
protected GridStartFactory	mGridStartFactory The handle to the GridStart Factory.
protected LogManager	mLogger The handle to the LogManager that logs all the messages.
static String	MONITORD_COMMENT_MARKER The marker to designate a line in the input file reserved for monitord purposes.
protected PegasusProperties	mProps The object holding all the properties pertaining to Pegasus.
protected SiteStore	mSiteStore Handle to the site catalog store
protected TransformationCatalog	mTCHandle The handle to the transformation catalog.
static String	TRANSFORMATION_NAMESPACE The transformation namespace for the cluster jobs.
static String	TRANSFORMATION_VERSION The version number for the derivations for cluster jobs

Fields inherited from interface edu.isi.pegasus.planner.cluster.JobAggregator

VERSION

Constructor Summary

Abstract()
The default constructor.

Method Summary

abstract String	aggregatedJobArguments(AggregatedJob job) Returns the arguments with which the AggregatedJob needs to be invoked with.
AggregatedJob	constructAbstractAggregatedJob(List jobs, String name, String id) Constructs a new aggregated job that contains all the jobs passed to it.
AggregatedJob	constructAbstractAggregatedJob(List jobs, String name, String id, String mergeLFN) Constructs an abstract aggregated job that has a handle to the appropriate JobAggregator that will be used to aggregate the jobs.
private TransformationCatalogEntry	defaultTCEntry(String name, String executableBasename, String site) Returns a default TC entry to be used in case entry is not found in the transformation catalog.
protected boolean	entryNotInTC(String namespace, String name, String version, String executableBasename, String site) Determines whether there is NOT an entry in the transformation catalog for a particular transformation on a particular site.
protected String	getCommentString(Job job, int taskid) Generates the comment string for the job .
static String	getCompleteTranformationName(String name) A convenience method to return the complete transformation name being used to construct jobs in this class.
protected TransformationCatalogEntry	getTCEntry(Job job) Helper method to get an entry from the transformation catalog for an installed executable.
void	initialize(ADag dag, PegasusBag bag) Initializes the JobAggregator impelementation
void	makeAbstractAggregatedJobConcrete(AggregatedJob job) Enables the abstract clustered job for execution and converts it to it's executable form
protected void	setDirectory(String directory) Sets the directory where the stdin files are to be generated.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface edu.isi.pegasus.planner.cluster.JobAggregator

abortOnFristJobFailure, entryNotInTC, getClusterExecutableBasename, getClusterExecutableLFN, setAbortOnFirstJobFailure

Field Detail

CLUSTERED_JOB_PREFIX

public static final String CLUSTERED_JOB_PREFIX

The prefix that is assigned to the jobname of the collapsed jobs to get the jobname for the fat job.

See Also:

Constant Field Values

TRANSFORMATION_NAMESPACE

public static final String TRANSFORMATION_NAMESPACE

The transformation namespace for the cluster jobs.

See Also:

Constant Field Values

TRANSFORMATION_VERSION

public static final String TRANSFORMATION_VERSION

The version number for the derivations for cluster jobs

DERIVATION_NAMESPACE

public static final String DERIVATION_NAMESPACE

The derivation namespace for the cluster jobs.

See Also:

Constant Field Values

DERIVATION_VERSION

public static final String DERIVATION_VERSION

The version number for the derivations for cluster jobs.

See Also:

Constant Field Values

MONITORD_COMMENT_MARKER

public static final String MONITORD_COMMENT_MARKER

The marker to designate a line in the input file reserved for monitord purposes.

See Also:

Constant Field Values

mDirectory

protected String mDirectory

The directory, where the stdin file of the fat jobs are created. It should be the submit file directory that the user mentions at runtime.

mProps

protected PegasusProperties mProps

The object holding all the properties pertaining to Pegasus.

mLogger

protected LogManager mLogger

The handle to the LogManager that logs all the messages.

mTCHandle

protected TransformationCatalog mTCHandle

The handle to the transformation catalog.

mSiteStore

protected SiteStore mSiteStore

Handle to the site catalog store

mClusteredADag

protected ADag mClusteredADag

The handle to the ADag object that contains the workflow being clustered.

mGridStartFactory

protected GridStartFactory mGridStartFactory

The handle to the GridStart Factory.

mBag

protected PegasusBag mBag

Bag of initialization objects.

Constructor Detail

Abstract

public Abstract()

The default constructor.

Method Detail

getCompleteTranformationName

public static String getCompleteTranformationName(String name)

A convenience method to return the complete transformation name being used to construct jobs in this class.

Parameters:

name - the name of the transformation

Returns:

the complete transformation name

initialize

public void initialize(ADag dag,
                       PegasusBag bag)

Initializes the JobAggregator impelementation

Specified by:

initialize in interface JobAggregator

Parameters:

dag - the workflow that is being clustered.

bag - the bag of objects that is useful for initialization.

aggregatedJobArguments

public abstract String aggregatedJobArguments(AggregatedJob job)

Returns the arguments with which the AggregatedJob needs to be invoked with.

Parameters:

job - the AggregatedJob for which the arguments have to be constructed.

Returns:

argument string

constructAbstractAggregatedJob

public AggregatedJob constructAbstractAggregatedJob(List jobs,
                                                    String name,
                                                    String id)

Constructs a new aggregated job that contains all the jobs passed to it. The new aggregated job, appears as a single job in the workflow and replaces the jobs it contains in the workflow.

Specified by:

constructAbstractAggregatedJob in interface JobAggregator

Parameters:

jobs - the list of Job objects that need to be collapsed. All the jobs being collapsed should be scheduled at the same pool, to maintain correct semantics.

name - the logical name of the jobs in the list passed to this function.

id - the id that is given to the new job.

Returns:

the Job object corresponding to the aggregated job containing the jobs passed as List in the input, null if the list of jobs is empty

constructAbstractAggregatedJob

public AggregatedJob constructAbstractAggregatedJob(List jobs,
                                                    String name,
                                                    String id,
                                                    String mergeLFN)

Constructs an abstract aggregated job that has a handle to the appropriate JobAggregator that will be used to aggregate the jobs.

Parameters:

jobs - the list of SubInfo objects that need to be collapsed. All the jobs being collapsed should be scheduled at the same pool, to maintain correct semantics.

name - the logical name of the jobs in the list passed to this function.

id - the id that is given to the new job.

mergeLFN - the logical name for the aggregated job that has to be constructed.

Returns:

the SubInfo object corresponding to the aggregated job containing the jobs passed as List in the input, null if the list of jobs is empty

makeAbstractAggregatedJobConcrete

public void makeAbstractAggregatedJobConcrete(AggregatedJob job)

Enables the abstract clustered job for execution and converts it to it's executable form

Specified by:

makeAbstractAggregatedJobConcrete in interface JobAggregator

Parameters:

job - the abstract clustered job

getCommentString

protected String getCommentString(Job job,
                                  int taskid)

Generates the comment string for the job . It generates a comment of the format # task_id transformation derivation.

Parameters:

job - the job for which

taskid - the task id to put in.

Returns:

the comment invocation

getTCEntry

protected TransformationCatalogEntry getTCEntry(Job job)

Helper method to get an entry from the transformation catalog for an installed executable. It does the traversal from the list of entries to return a single TransformationCatalogEntry object, and dies with an appropriate error message if the object is not found. The pool and the name are retrieved from job object.

Parameters:

job - the job whose corresponding TransformationCatalogEntry you want.

Returns:

the TransformationCatalogEntry corresponding to the entry in the TC.

defaultTCEntry

private TransformationCatalogEntry defaultTCEntry(String name,
                                                  String executableBasename,
                                                  String site)

Returns a default TC entry to be used in case entry is not found in the transformation catalog.

Parameters:

name - the logical name for the clustering transformation.

executableBasename - the basename for the executable in the bin directory of a Pegasus installation

site - the site for which the default entry is required.

Returns:

the default entry.

entryNotInTC

protected boolean entryNotInTC(String namespace,
                               String name,
                               String version,
                               String executableBasename,
                               String site)

Determines whether there is NOT an entry in the transformation catalog for a particular transformation on a particular site.

Parameters:

namespace - the logical namespace of the transformation.

name - the logical name of the transformation.

version - the version of the transformation.

executableBasename - basename of the executable that does the clustering.

site - the site at which existence check is required.

Returns:

boolean true if an entry does not exists, false otherwise.

setDirectory

protected void setDirectory(String directory)

Sets the directory where the stdin files are to be generated.

Parameters:

directory - the path to the directory to which it needs to be set.