edu.isi.pegasus.planner.transfer
Interface Implementation

All Known Implementing Classes:

Abstract, AbstractMultipleFTPerXFERJob, AbstractSingleFTPerXFERJob, GUC, Stork, T2, TPTGUC, Transfer

public interface Implementation

The interface defines the functions that a particular Transfer Implementation should implement. The functions deal with the creation of a TransferJob that can transfer files using the transfer tool to which it refers to.

Version:

$Revision: 4720 $

Author:

Karan Vahi, Gaurang Mehta

Field Summary

static String	TRANSFER_UNIVERSE The universe that applies for the transfer jobs.
static String	VERSION The version number associated with this API

Method Summary

boolean	addSetXBitJobs(Job computeJob, String txJobName, Collection execFiles, int transferClass, int xbitIndex) Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
void	applyPriority(TransferJob job) Applies priorities to the transfer jobs if a priority is specified in the properties file.
boolean	checkAndTransferProxy(TransferJob job) Determines if there is a need to transfer proxy for the transfer job or not.
Job	createSetXBitJob(Job computeJob, Collection<FileTransfer> execFiles, int transferClass, int xbitIndex) Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
TransferJob	createTransferJob(Job job, String site, Collection files, Collection execFiles, String txJobName, int jobClass) This constructs the Job object for the transfer node.
boolean	doesPreserveXBit() Returns a boolean indicating whether the transfer protocol being used by the implementation preserves the X Bit or not while staging.
String	getDescription() Returns a textual description of the transfer implementation.
String	getSetXBitJobName(String name, int counter) Generates the name of the setXBitJob , that is unique for the given workflow.
TransformationCatalogEntry	getTransformationCatalogEntry(String siteHandle, int jobClass) Retrieves the transformation catalog entry for the executable that is being used to transfer the files in the implementation.
void	setRefiner(Refiner refiner) Sets the callback to the refiner, that has loaded this implementation.
boolean	useThirdPartyTransferAlways() Return a boolean indicating whether the transfers to be done always in a third party transfer mode.

Field Detail

VERSION

static final String VERSION

The version number associated with this API

See Also:

Constant Field Values

TRANSFER_UNIVERSE

static final String TRANSFER_UNIVERSE

The universe that applies for the transfer jobs. Used for querying to the Site Catalog.

See Also:

Constant Field Values

Method Detail

setRefiner

void setRefiner(Refiner refiner)

Sets the callback to the refiner, that has loaded this implementation.

Parameters:

refiner - the transfer refiner that loaded the implementation.

createTransferJob

TransferJob createTransferJob(Job job,
                              String site,
                              Collection files,
                              Collection execFiles,
                              String txJobName,
                              int jobClass)

This constructs the Job object for the transfer node. The transfer is supposed to occur at job execution site. It should lead to the creation of the setup chmod jobs to the workflow, that appear as parents to compute jobs in case the transfer implementation does not preserve the X bit on the file being transferred. This is required for staging of executables as part of the workflow.

Parameters:

job - the Job object for the job, in relation to which the transfer node is being added. Either the transfer node can be transferring this jobs input files to the execution pool, or transferring this job's output files to the output pool.

site - the site where the transfer job should run.

files - collection of FileTransfer objects representing the data files and staged executables to be transferred.

execFiles - subset collection of the files parameter, that identifies the executable files that are being transferred.

txJobName - the name of transfer node.

jobClass - the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer

Returns:

the created TransferJob.

doesPreserveXBit

boolean doesPreserveXBit()

Returns a boolean indicating whether the transfer protocol being used by the implementation preserves the X Bit or not while staging. If it does not, then it should extend the Abstract implementation of this interface, that allows for adding of a setup job after the stagein job that changes the X Bit.

addSetXBitJobs

boolean addSetXBitJobs(Job computeJob,
                       String txJobName,
                       Collection execFiles,
                       int transferClass,
                       int xbitIndex)

Adds the dirmanager job to the workflow, that do a chmod on the files being staged.

Parameters:

computeJob - the computeJob for which the files are being staged.

txJobName - the name of the transfer job that is staging the files.

execFiles - the executable files that are being staged.

transferClass - the class of transfer job

xbitIndex - index to be used for creating the name of XBitJob.

Returns:

boolean indicating whether any XBitJobs were succesfully added or not.

createSetXBitJob

Job createSetXBitJob(Job computeJob,
                     Collection<FileTransfer> execFiles,
                     int transferClass,
                     int xbitIndex)

Adds the dirmanager job to the workflow, that do a chmod on the files being staged.

Parameters:

computeJob - the computeJob for which the files are being staged.

execFiles - the executable files that are being staged.

transferClass - the class of transfer job

xbitIndex - index to be used for creating the name of XBitJob.

Returns:

the job object for the xBitJob

getSetXBitJobName

String getSetXBitJobName(String name,
                         int counter)

Generates the name of the setXBitJob , that is unique for the given workflow. If the implementation preserve the X bit, then it should return null.

Parameters:

name - the name of the compute job for which the executable is being staged.

counter - the index for the setXBit job.

Returns:

the name of the setXBitJob, null in case the implementation preserves the XBit.

getTransformationCatalogEntry

TransformationCatalogEntry getTransformationCatalogEntry(String siteHandle,
                                                         int jobClass)

Retrieves the transformation catalog entry for the executable that is being used to transfer the files in the implementation.

Parameters:

siteHandle - the handle of the site where the transformation is to be searched.

jobClass - the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer stage-in worker transfer

Returns:

the transformation catalog entry if found, else null.

useThirdPartyTransferAlways

boolean useThirdPartyTransferAlways()

Return a boolean indicating whether the transfers to be done always in a third party transfer mode. A value of false, results in the direct or peer to peer transfers being done.

A value of false does not preclude third party transfers. They still can be done, by setting the property "pegasus.transfer.*.thirdparty.sites".

Returns:

boolean indicating whether to always use third party transfers or not.

See Also:

PegasusProperties.getThirdPartySites(String)

applyPriority

void applyPriority(TransferJob job)

Applies priorities to the transfer jobs if a priority is specified in the properties file.

Parameters:

job - the transfer job .

checkAndTransferProxy

boolean checkAndTransferProxy(TransferJob job)

Determines if there is a need to transfer proxy for the transfer job or not. If there is a need to transfer proxy, then the job is modified to create the correct condor commands to transfer the proxy. Proxy is usually transferred if the VDS profile TRANSFER_PROXY is set, or the job is being run in the condor vanilla universe. The proxy is transferred from the submit host (i.e site local). The location is determined from the value of the X509_USER_PROXY profile key associated in the env namespace.

Parameters:

job - the transfer job .

Returns:

boolean true job was modified to transfer the proxy, else false when job is not modified.

getDescription

String getDescription()

Returns a textual description of the transfer implementation.

Returns:

a short textual description