edu.isi.pegasus.planner.code.generator
Class Stampede

java.lang.Object
  edu.isi.pegasus.planner.code.generator.Stampede

All Implemented Interfaces:

CodeGenerator

public class Stampede
extends Object
implements CodeGenerator

A Stampede Events Code Generator that generates events in netlogger format for the exectuable workflow. This generators generates the events about

   the tasks int he abstract workflow
   the edges in the abstract workflow
   jobs in the executable workflow
   the edges in the executable workflow
   relationship about how the tasks in the abstract workflow map to jobs in the
   executable workflow.
 

Version:

$Revision: 4369 $

Author:

Karan Vahi

Field Summary

static String	ARGUMENTS_KEY The attribute key for task arguments.
static String	CHILD_JOB_ID_KEY The atrribute key for child job id.
static String	CHILD_TASK_ID_KEY The atrribute key for child task id.
static String	JOB_CLUSTERED_KEY The attribute key for whether a job is clustered or not
static String	JOB_EDGE_EVENT_NAME The event name for job edge
static String	JOB_EVENT_NAME The event name for a job
static String	JOB_EXECUTABLE_KEY The attribute key for the executable
static String	JOB_ID_KEY The attribute key for job id
static String	JOB_MAX_RETRIES_KEY The attribute key for how many times a job is retried
static String	JOB_SUBMIT_FILE_KEY Teh attribute key for the submit file
static String	JOB_TASK_COUNT_KEY The attribute key for the number of tasks in the job
protected PegasusBag	mBag The bag of initialization objects.
private LogFormatter	mLogFormatter The handle to the netlogger log formatter.
protected LogManager	mLogger The handle to the logging object.
protected PlannerOptions	mPOptions The object containing the command line options specified to the planner at runtime.
protected PegasusProperties	mProps The object holding all the properties pertaining to Pegasus.
protected String	mSubmitFileDir The directory where all the submit files are to be generated.
static String	NETLOGGER_BP_FILE_SUFFIX The suffix to use while constructing the name of the metrics file
static String	NETLOGGER_LOG_FORMATTER_IMPLEMENTOR
static String	PARENT_JOB_ID_KEY The atrribute key for parent job id.
static String	PARENT_TASK_ID_KEY The atrribute key for parent task id.
static String	TASK_EDGE_EVENT_NAME The event name for task edge
static String	TASK_EVENT_NAME The event name for task info
static String	TASK_ID_KEY The attribute key for task id
static String	TASK_MAP_EVENT_NAME The event name for task map event
static String	TASK_TRANSFORMATION_KEY The attribute key for transformation
static String	TYPE_DESCRIPTION_KEY The attribute key for type description
static String	TYPE_KEY The attribute key for task type
static String	WORKFLOW_ID_KEY The attribute key for workflow id.

Fields inherited from interface edu.isi.pegasus.planner.code.CodeGenerator

VERSION

Constructor Summary

Stampede()

Method Summary

String	booleanToInt(boolean value) Returns boolean as an integer
Collection<File>	generateCode(ADag dag) Generates the code for the executable workflow in terms of a braindump file that contains workflow metadata useful for monitoring daemons etc.
void	generateCode(ADag dag, Job job) Method not implemented.
protected void	generateEventsForDAXTask(PrintWriter writer, ADag workflow, Job job) Generates stampede events corresponding to jobs/tasks in the DAX
protected void	generateEventsForExecutableJob(PrintWriter writer, ADag dag, Job job) Generates stampede events corresponding to an executable job
protected void	generateTaskMapEvents(PrintWriter writer, ADag dag, Job job) Generates the task.map events that link the jobs in the DAX with the jobs in the executable workflow
private int	getTaskCount(Job job) Returns the task count for a job.
void	initialize(PegasusBag bag) Initializes the Code Generator implementation.
void	reset() Resets the Code Generator implementation.
boolean	startMonitoring() Starts monitoring of the workflow by invoking a workflow monitor daemon.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NETLOGGER_BP_FILE_SUFFIX

public static final String NETLOGGER_BP_FILE_SUFFIX

The suffix to use while constructing the name of the metrics file

See Also:

Constant Field Values

NETLOGGER_LOG_FORMATTER_IMPLEMENTOR

public static final String NETLOGGER_LOG_FORMATTER_IMPLEMENTOR

See Also:

Constant Field Values

WORKFLOW_ID_KEY

public static final String WORKFLOW_ID_KEY

The attribute key for workflow id.

See Also:

Constant Field Values

TASK_EVENT_NAME

public static final String TASK_EVENT_NAME

The event name for task info

See Also:

Constant Field Values

TASK_ID_KEY

public static final String TASK_ID_KEY

The attribute key for task id

See Also:

Constant Field Values

TYPE_KEY

public static final String TYPE_KEY

The attribute key for task type

See Also:

Constant Field Values

TYPE_DESCRIPTION_KEY

public static final String TYPE_DESCRIPTION_KEY

The attribute key for type description

See Also:

Constant Field Values

TASK_TRANSFORMATION_KEY

public static final String TASK_TRANSFORMATION_KEY

The attribute key for transformation

See Also:

Constant Field Values

ARGUMENTS_KEY

public static final String ARGUMENTS_KEY

The attribute key for task arguments.

See Also:

Constant Field Values

TASK_EDGE_EVENT_NAME

public static final String TASK_EDGE_EVENT_NAME

The event name for task edge

See Also:

Constant Field Values

PARENT_TASK_ID_KEY

public static final String PARENT_TASK_ID_KEY

The atrribute key for parent task id.

See Also:

Constant Field Values

CHILD_TASK_ID_KEY

public static final String CHILD_TASK_ID_KEY

The atrribute key for child task id.

See Also:

Constant Field Values

JOB_EVENT_NAME

public static final String JOB_EVENT_NAME

The event name for a job

See Also:

Constant Field Values

JOB_ID_KEY

public static final String JOB_ID_KEY

The attribute key for job id

See Also:

Constant Field Values

JOB_SUBMIT_FILE_KEY

public static final String JOB_SUBMIT_FILE_KEY

Teh attribute key for the submit file

See Also:

Constant Field Values

JOB_CLUSTERED_KEY

public static final String JOB_CLUSTERED_KEY

The attribute key for whether a job is clustered or not

See Also:

Constant Field Values

JOB_MAX_RETRIES_KEY

public static final String JOB_MAX_RETRIES_KEY

The attribute key for how many times a job is retried

See Also:

Constant Field Values

JOB_TASK_COUNT_KEY

public static final String JOB_TASK_COUNT_KEY

The attribute key for the number of tasks in the job

See Also:

Constant Field Values

JOB_EXECUTABLE_KEY

public static final String JOB_EXECUTABLE_KEY

The attribute key for the executable

See Also:

Constant Field Values

JOB_EDGE_EVENT_NAME

public static final String JOB_EDGE_EVENT_NAME

The event name for job edge

See Also:

Constant Field Values

PARENT_JOB_ID_KEY

public static final String PARENT_JOB_ID_KEY

The atrribute key for parent job id.

See Also:

Constant Field Values

CHILD_JOB_ID_KEY

public static final String CHILD_JOB_ID_KEY

The atrribute key for child job id.

See Also:

Constant Field Values

TASK_MAP_EVENT_NAME

public static final String TASK_MAP_EVENT_NAME

The event name for task map event

See Also:

Constant Field Values

mLogFormatter

private LogFormatter mLogFormatter

The handle to the netlogger log formatter.

mBag

protected PegasusBag mBag

The bag of initialization objects.

mSubmitFileDir

protected String mSubmitFileDir

The directory where all the submit files are to be generated.

mProps

protected PegasusProperties mProps

The object holding all the properties pertaining to Pegasus.

mPOptions

protected PlannerOptions mPOptions

The object containing the command line options specified to the planner at runtime.

mLogger

protected LogManager mLogger

The handle to the logging object.

Constructor Detail

Stampede

public Stampede()

Method Detail

initialize

public void initialize(PegasusBag bag)
                throws CodeGeneratorException

Initializes the Code Generator implementation.

Specified by:

initialize in interface CodeGenerator

Parameters:

bag - the bag of initialization objects.

Throws:

CodeGeneratorException - in case of any error occuring code generation.

generateCode

public Collection<File> generateCode(ADag dag)
                              throws CodeGeneratorException

Generates the code for the executable workflow in terms of a braindump file that contains workflow metadata useful for monitoring daemons etc.

Specified by:

generateCode in interface CodeGenerator

Parameters:

dag - the concrete workflow.

Returns:

the Collection of File objects for the files written out.

Throws:

CodeGeneratorException - in case of any error occuring code generation.

generateEventsForDAXTask

protected void generateEventsForDAXTask(PrintWriter writer,
                                        ADag workflow,
                                        Job job)
                                 throws CodeGeneratorException

Generates stampede events corresponding to jobs/tasks in the DAX

Parameters:

writer - the writer stream to write the events too

workflow - the workflow.

job - the job for which to generate the events.

Throws:

CodeGeneratorException

generateEventsForExecutableJob

protected void generateEventsForExecutableJob(PrintWriter writer,
                                              ADag dag,
                                              Job job)
                                       throws CodeGeneratorException

Generates stampede events corresponding to an executable job

Parameters:

writer - the writer stream to write the events too

workflow - the workflow.

job - the job for which to generate the events.

Throws:

CodeGeneratorException

generateTaskMapEvents

protected void generateTaskMapEvents(PrintWriter writer,
                                     ADag dag,
                                     Job job)

Generates the task.map events that link the jobs in the DAX with the jobs in the executable workflow

Parameters:

writer - the writer stream to write the events too

workflow - the workflow.

job - the job for which to generate the events.

generateCode

public void generateCode(ADag dag,
                         Job job)
                  throws CodeGeneratorException

Method not implemented. Throws an exception.

Specified by:

generateCode in interface CodeGenerator

Parameters:

dag - the workflow

job - the job for which the code is to be generated.

Throws:

CodeGeneratorException

getTaskCount

private int getTaskCount(Job job)

Returns the task count for a job. The task count is the number of tasks/jobs in the DAX that map to this job. jobs inserted by Pegasus, which do not have a mapped task from the DAX, will have its task_count set to 0.

Parameters:

job - the executable job.

Returns:

task count

booleanToInt

public String booleanToInt(boolean value)

Returns boolean as an integer

Parameters:

value - the boolean value

Returns:

0 for false and 1 for true

startMonitoring

public boolean startMonitoring()

Description copied from interface: CodeGenerator

Starts monitoring of the workflow by invoking a workflow monitor daemon. The monitoring should start only after the output files have been generated. FIXME: It should actually happen after the workflow has been submitted. Eventually should be a separate monitor interface, and submit writers should be loaded by an AbstractFactory.

Specified by:

startMonitoring in interface CodeGenerator

Returns:

boolean indicating whether could successfully start the monitor daemon or not.

reset

public void reset()
           throws CodeGeneratorException

Description copied from interface: CodeGenerator

Resets the Code Generator implementation.

Specified by:

reset in interface CodeGenerator

Throws:

CodeGeneratorException - in case of any error occuring code generation.