edu.isi.pegasus.planner.refiner.cleanup
Class InPlace

java.lang.Object
  edu.isi.pegasus.planner.refiner.cleanup.InPlace

All Implemented Interfaces:

CleanupStrategy

public class InPlace
extends Object
implements CleanupStrategy

This generates cleanup jobs in the workflow itself.

Version:

$Revision: 4551 $

Author:

Arun ramakrishnan, Karan Vahi

Field Summary

static String	CLEANUP_JOB_PREFIX The prefix for CLEANUP_JOB ID i.e prefix+the parent compute_job ID becomes ID of the cleanup job.
private HashSet	mDoNotClean HashSet of Files that should not be cleaned up
private CleanupImplementation	mImpl The handle to the CleanupImplementation instance that creates the jobs for us.
private LogManager	mLogger The handle to the logging object used for logging.
private int	mMaxDepth The max depth of any job in the workflow useful for a priorityQueue implementation in an array
private PegasusProperties	mProps The handle to the properties passed to Pegasus.
private HashMap	mResMap The mapping to siteHandle to all the jobs that are mapped to it mapping to siteHandle(String) to Set
private HashMap	mResMapLeaves The mapping of siteHandle to all subset of the jobs mapped to it that are leaves in the workflow mapping to siteHandle(String) to Set.
private HashMap	mResMapRoots The mapping of siteHandle to all subset of the jobs mapped to it that are leaves in the workflow mapping to siteHandle(String) to Set.

Fields inherited from interface edu.isi.pegasus.planner.refiner.cleanup.CleanupStrategy

VERSION

Constructor Summary

InPlace()
The default constructor.

Method Summary

Graph	addCleanupJobs(Graph workflow) Adds cleanup jobs to the workflow.
private void	addCleanUpJobs(String site, Set leaves, Graph workflow) Adds cleanup jobs for the workflow scheduled to a particular site a best first search strategy is implemented based on the depth of the job in the workflow
protected void	applyJobPriorities(Graph workflow) Adds job priorities to the jobs in the workflow on the basis of the levels in the traversal order given by the iterator.
protected String	generateCleanupID(Job job) Returns the identifier that is to be assigned to cleanup job.
protected String	getSiteForCleanup(Job job) Returns site to be used for the cleanup algorithm.
void	initialize(PegasusBag bag, CleanupImplementation impl) Intializes the class.
protected void	reset() Resets the internal data structures.
private void	setDepth_ResMap(List roots) A BFS implementation to set depth value (roots have depth 1) and also to populate mResMap ,mResMapLeaves,mResMapRoots which contains all the jobs that are assigned to a particular resource
protected boolean	typeNeedsCleanUp(int type) Checks to see which job types are required to be looked at for cleanup.
protected boolean	typeStageOut(int type) Checks to see if job type is a stageout job type.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLEANUP_JOB_PREFIX

public static final String CLEANUP_JOB_PREFIX

The prefix for CLEANUP_JOB ID i.e prefix+the parent compute_job ID becomes ID of the cleanup job.

See Also:

Constant Field Values

mResMap

private HashMap mResMap

The mapping to siteHandle to all the jobs that are mapped to it mapping to siteHandle(String) to Set

mResMapLeaves

private HashMap mResMapLeaves

The mapping of siteHandle to all subset of the jobs mapped to it that are leaves in the workflow mapping to siteHandle(String) to Set.

mResMapRoots

private HashMap mResMapRoots

The mapping of siteHandle to all subset of the jobs mapped to it that are leaves in the workflow mapping to siteHandle(String) to Set.

mMaxDepth

private int mMaxDepth

The max depth of any job in the workflow useful for a priorityQueue implementation in an array

mDoNotClean

private HashSet mDoNotClean

HashSet of Files that should not be cleaned up

mImpl

private CleanupImplementation mImpl

The handle to the CleanupImplementation instance that creates the jobs for us.

mProps

private PegasusProperties mProps

The handle to the properties passed to Pegasus.

mLogger

private LogManager mLogger

The handle to the logging object used for logging.

Constructor Detail

InPlace

public InPlace()

The default constructor.

Method Detail

initialize

public void initialize(PegasusBag bag,
                       CleanupImplementation impl)

Intializes the class.

Specified by:

initialize in interface CleanupStrategy

Parameters:

bag - bag of initialization objects

impl - the implementation instance that creates cleanup job

addCleanupJobs

public Graph addCleanupJobs(Graph workflow)

Adds cleanup jobs to the workflow.

Specified by:

addCleanupJobs in interface CleanupStrategy

Parameters:

workflow - the workflow to add cleanup jobs to.

Returns:

the workflow with cleanup jobs added to it.

reset

protected void reset()

Resets the internal data structures.

setDepth_ResMap

private void setDepth_ResMap(List roots)

A BFS implementation to set depth value (roots have depth 1) and also to populate mResMap ,mResMapLeaves,mResMapRoots which contains all the jobs that are assigned to a particular resource

Parameters:

roots - List of GraphNode objects that are roots

addCleanUpJobs

private void addCleanUpJobs(String site,
                            Set leaves,
                            Graph workflow)

Adds cleanup jobs for the workflow scheduled to a particular site a best first search strategy is implemented based on the depth of the job in the workflow

Parameters:

site - the site ID

leaves - the leaf jobs that are scheduled to site

workflow - the Graph into which new cleanup jobs can be added

applyJobPriorities

protected void applyJobPriorities(Graph workflow)

Adds job priorities to the jobs in the workflow on the basis of the levels in the traversal order given by the iterator. Later on this would be a separate refiner.

Parameters:

workflow - the workflow on which to apply job priorities.

generateCleanupID

protected String generateCleanupID(Job job)

Returns the identifier that is to be assigned to cleanup job.

Parameters:

job - the job with which the cleanup job is primarily associated.

Returns:

the identifier for a cleanup job.

typeNeedsCleanUp

protected boolean typeNeedsCleanUp(int type)

Checks to see which job types are required to be looked at for cleanup. COMPUTE_JOB , STAGE_OUT_JOB , INTER_POOL_JOB are the ones that need cleanup

Parameters:

type - the type of the job.

Returns:

boolean

typeStageOut

protected boolean typeStageOut(int type)

Checks to see if job type is a stageout job type.

Parameters:

type - the type of the job.

Returns:

boolean

getSiteForCleanup

protected String getSiteForCleanup(Job job)

Returns site to be used for the cleanup algorithm. For compute jobs the staging site is used, while for stageout jobs is used. For all other jobs the execution site is used.

Parameters:

job - the job

Returns:

the site to be used