edu.isi.pegasus.planner.catalog.transformation.impl
Class File

java.lang.Object
  edu.isi.pegasus.planner.catalog.transformation.impl.Abstract
      edu.isi.pegasus.planner.catalog.transformation.impl.File

All Implemented Interfaces:

Catalog, TransformationCatalog

public class File
extends Abstract
implements TransformationCatalog

This is the new file based TC implementation storing the contents of the file in memory. For the old tc file implementation see OldTC.java

Version:

$Revision: 4800 $

Author:

Gaurang Mehta, Karan Vahi

Field Summary

private boolean	mFlushOnClose Boolean indicating whether to flush the contents back to the file on close.
protected LogManager	mLogger The LogManager object which is used to log all the messages.
private boolean	modifyFileURL Boolean indicating whether to modify the file URL or not
private PegasusProperties	mProps The handle to the properties object.
private String	mTCFile The path to the file based TC.
private static File	mTCFileHandle The singleton handler to the contents of the transformation catalog.
private Map	mTreeMap The Tree Map which stores the contents of the file.
protected List	mvExecPools The List containing the user specified list of pools on which he wants the dag to run.

Fields inherited from interface edu.isi.pegasus.planner.catalog.TransformationCatalog

FILE_URL_SCHEME, MODIFY_FOR_FILE_URLS_KEY, VERSION

Fields inherited from interface edu.isi.pegasus.planner.catalog.Catalog

DB_ALL_PREFIX

Constructor Summary

File()

Method Summary

int	addLFNProfile(String namespace, String name, String version, List profiles) Add additional profile to a logical transformation .
int	addPFNProfile(String pfn, TCType type, String resourcename, List profiles) Add additional profile to a physical transformation.
protected boolean	addTCEntry(String namespace, String name, String version, String physicalname, TCType type, String resourceid, List pfnprofiles, List lfnprofiles, SysInfo system, Notifications invokes, boolean write) Add an single entry into the transformation catalog.
int	clear() Deletes the entire transformation catalog.
void	close() Explicitely free resources before the garbage collection hits.
boolean	connect(Properties props) Establishes a link between the implementation and the thing the implementation is build upon.
int	deleteLFNProfiles(String namespace, String name, String version, List profiles) Delete a list of profiles or all the profiles associated with a logical transformation.
int	deletePFNProfiles(String physicalname, TCType type, String resourceid, List profiles) Delete a list of profiles or all the profiles associated with a pfn on a resource and of a type.
void	File() The default constructor.
List<TransformationCatalogEntry>	getContents() List all the contents of the TC
String	getDescription() Returns a textual description of the transformation mode.
static TransformationCatalog	getInstance() Deprecated.
List<String[]>	getTCLogicalNames(String resourceid, TCType type) Get the list of LogicalNames available on a particular resource.
void	initialize(PegasusBag bag) Initialize the implementation, and return an instance of the implementation.
int	insert(List<TransformationCatalogEntry> entries) Add multiple TCEntries to the Catalog.
int	insert(String namespace, String name, String version, String physicalname, TCType type, String resourceid, List pfnprofiles, List lfnprofiles, SysInfo system) Add an single entry into the transformation catalog.
int	insert(TransformationCatalogEntry entry) Add single TCEntry to the Catalog.
int	insert(TransformationCatalogEntry entry, boolean write) Add single TCEntry object temporarily to the in memory Catalog.
boolean	isClosed() Predicate to check, if the connection with the catalog's implementation is still active.
protected void	logMessage(String msg) Logs the message to a logging stream.
List<TransformationCatalogEntry>	lookup(String namespace, String name, String version, List resourceids, TCType type) Returns TC entries for a particular logical transformation and/or on a number of resources and/or of a particular type.
List<TransformationCatalogEntry>	lookup(String namespace, String name, String version, String resourceid, TCType type) Returns TC entries for a particular logical transformation and/or on a particular resource and/or of a particular type.
List<Profile>	lookupLFNProfiles(String namespace, String name, String version) Get the list of Profiles associated with a particular logical transformation.
List<TransformationCatalogEntry>	lookupNoProfiles(String namespace, String name, String version, String resourceid, TCType type) Get the list of PhysicalNames for a particular transformation on a site/sites for a particular type/types;
List<Profile>	lookupPFNProfiles(String pfn, String resourceid, TCType type) Get the list of Profiles associated with a particular physical transformation.
List<String>	lookupSites(String namespace, String name, String version, TCType type) Get the list of Resource ID's where a particular transformation may reside.
private boolean	populateTC() Populates the internal copy of the transformation catalog from the file containing the transformation catalog in the 6 column format.
private boolean	populateTC(InputStream reader) Populates the internal copy of the transformation catalog from a byte stream (input stream).
private boolean	populateTC(Reader reader) Adds multiple entries into the TC.
int	removeByLFN(String namespace, String name, String version, String resourceid, TCType type) Delete all entries in the transformation catalog for a give logical tranformation and/or on a resource and/or of a particular type
int	removeByPFN(String physicalname, String namespace, String name, String version, String resourceid, TCType type) Delete all entries in the transformation catalog for pair of logical and physical transformation.
int	removeBySiteID(String resourceid) Delete all entries on a particular resource from the transformation catalog.
int	removeBySysInfo(SysInfo sysinfo) Deletes entries from the catalog which have a particular system information.
int	removeByType(TCType type, String resourceid) Delete a particular type of transformation, and/or on a particular resource
private void	writeTC()

Methods inherited from class edu.isi.pegasus.planner.catalog.transformation.impl.Abstract

modifyForFileURLS, modifyForFileURLS

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mTCFileHandle

private static File mTCFileHandle

The singleton handler to the contents of the transformation catalog.

mLogger

protected LogManager mLogger

The LogManager object which is used to log all the messages. It's values are set in the CPlanner (the main toolkit) class.

mvExecPools

protected List mvExecPools

The List containing the user specified list of pools on which he wants the dag to run.

mTreeMap

private Map mTreeMap

The Tree Map which stores the contents of the file. The key is the transformationname.

mTCFile

private String mTCFile

The path to the file based TC.

mProps

private PegasusProperties mProps

The handle to the properties object.

mFlushOnClose

private boolean mFlushOnClose

Boolean indicating whether to flush the contents back to the file on close.

modifyFileURL

private boolean modifyFileURL

Boolean indicating whether to modify the file URL or not

Constructor Detail

File

public File()

Method Detail

getInstance

public static TransformationCatalog getInstance()

Deprecated.

Returns an instance of the File TC.

Returns:

TransformationCatalog

File

public void File()

The default constructor.

initialize

public void initialize(PegasusBag bag)

Initialize the implementation, and return an instance of the implementation.

Specified by:

initialize in interface TransformationCatalog

Parameters:

bag - the bag of Pegasus initialization objects.

getDescription

public String getDescription()

Returns a textual description of the transformation mode.

Specified by:

getDescription in interface TransformationCatalog

Returns:

String containing the description.

lookup

public List<TransformationCatalogEntry> lookup(String namespace,
                                               String name,
                                               String version,
                                               List resourceids,
                                               TCType type)
                                        throws Exception

Returns TC entries for a particular logical transformation and/or on a number of resources and/or of a particular type.

Specified by:

lookup in interface TransformationCatalog

Parameters:

namespace - String The namespace of the logical transformation.

name - String the name of the logical transformation.

version - String The version of the logical transformation.

resourceids - List The List resourceid where the transformation is located. If NULL it returns all resources.

type - TCType The type of the transformation to search for. If NULL it returns all types.

Returns:

List Returns a list of TransformationCatalogEntry objects containing the corresponding entries from the TC. Returns null if no entry found.

Throws:

Exception

See Also:

TCType, edu.isi.pegasus.planner.catalog.TransformationCatalogEntry

lookup

public List<TransformationCatalogEntry> lookup(String namespace,
                                               String name,
                                               String version,
                                               String resourceid,
                                               TCType type)
                                        throws Exception

Returns TC entries for a particular logical transformation and/or on a particular resource and/or of a particular type.

Specified by:

lookup in interface TransformationCatalog

Parameters:

namespace - String The namespace of the logical transformation.

name - String the name of the logical transformation.

version - String The version of the logical transformation.

resourceid - String The resourceid where the transformation is located. If NULL it returns all resources.

type - TCType The type of the transformation to search for. If NULL it returns all types.

Returns:

List Returns a list of TransformationCatalogEntry objects containing the corresponding entries from the TC. Returns null if no entry found.

Throws:

Exception

See Also:

TCType, edu.isi.pegasus.planner.catalog.TransformationCatalogEntry

lookupSites

public List<String> lookupSites(String namespace,
                                String name,
                                String version,
                                TCType type)
                         throws Exception

Get the list of Resource ID's where a particular transformation may reside.

Specified by:

lookupSites in interface TransformationCatalog

Parameters:

namespace - String The namespace of the transformation to search for.

name - String The name of the transformation to search for.

version - String The version of the transformation to search for.

type - TCType The type of the transformation to search for.
(Enumerated type includes SOURCE, STATIC-BINARY, DYNAMIC-BINARY, PACMAN, INSTALLED, SCRIPT)
If NULL it returns all types.

Returns:

List Returns a list of Resource Id's as strings. Returns NULL if no results found.

Throws:

Exception - NotImplementedException if not implemented

See Also:

TCType

lookupNoProfiles

public List<TransformationCatalogEntry> lookupNoProfiles(String namespace,
                                                         String name,
                                                         String version,
                                                         String resourceid,
                                                         TCType type)
                                                  throws Exception

Get the list of PhysicalNames for a particular transformation on a site/sites for a particular type/types;

Specified by:

lookupNoProfiles in interface TransformationCatalog

Parameters:

namespace - String The namespace of the transformation to search for.

name - String The name of the transformation to search for.

version - String The version of the transformation to search for.

resourceid - String The id of the resource on which you want to search.
If NULL then returns entries on all resources

type - TCType The type of the transformation to search for.
(Enumerated type includes source, binary, dynamic-binary, pacman, installed)
If NULL then returns entries of all types.

Returns:

List Returns a List of objects with the profiles not populated.

Throws:

Exception - NotImplementedException if not implemented.

See Also:

TCType

getTCLogicalNames

public List<String[]> getTCLogicalNames(String resourceid,
                                        TCType type)
                                 throws Exception

Get the list of LogicalNames available on a particular resource.

Specified by:

getTCLogicalNames in interface TransformationCatalog

Parameters:

resourceid - String The id of the resource on which you want to search

type - TCType The type of the transformation to search for.
(Enumerated type includes source, binary, dynamic-binary, pacman, installed)
If NULL then return logical name for all types.

Returns:

List Returns a list of String Arrays. Each array contains the resourceid, logical transformation in the format namespace::name:version and type. Returns NULL if no results found.

Throws:

Exception - NotImplementedException if not implemented.

lookupLFNProfiles

public List<Profile> lookupLFNProfiles(String namespace,
                                       String name,
                                       String version)
                                throws Exception

Get the list of Profiles associated with a particular logical transformation.

Specified by:

lookupLFNProfiles in interface TransformationCatalog

Parameters:

namespace - String The namespace of the transformation to search for.

name - String The name of the transformation to search for.

version - String The version of the transformation to search for.

Returns:

List Returns a list of Profile Objects containing profiles assocaited with the transformation. Returns NULL if no profiles found.

Throws:

Exception - NotImplementedException if not implemented.

See Also:

org.griphyn.cPlanner.classes.Profile

lookupPFNProfiles

public List<Profile> lookupPFNProfiles(String pfn,
                                       String resourceid,
                                       TCType type)
                                throws Exception

Get the list of Profiles associated with a particular physical transformation.

Specified by:

lookupPFNProfiles in interface TransformationCatalog

Parameters:

pfn - The physical file name to search the transformation by.

resourceid - String The id of the resource on which you want to search.

type - TCType The type of the transformation to search for.
(Enumerated type includes source, binary, dynamic-binary, pacman, installed)

Returns:

List Returns a list of Profile Objects containing profiles assocaited with the transformation. Returns NULL if no profiless found.

Throws:

Exception - NotImplementedException if not implemented.

See Also:

org.griphyn.cPlanner.classes.Profile

getContents

public List<TransformationCatalogEntry> getContents()
                                             throws Exception

List all the contents of the TC

Specified by:

getContents in interface TransformationCatalog

Returns:

List Returns a List of TransformationCatalogEntry objects.

Throws:

Exception

insert

public int insert(List<TransformationCatalogEntry> entries)
           throws Exception

Add multiple TCEntries to the Catalog.

Specified by:

insert in interface TransformationCatalog

Parameters:

tcentry - List Takes a list of TransformationCatalogEntry objects as input

Returns:

number of insertions On failure,throw an exception, don't use zero.

Throws:

Exception

See Also:

edu.isi.pegasus.planner.catalog.TransformationCatalogEntry

insert

public int insert(TransformationCatalogEntry entry)
           throws Exception

Add single TCEntry to the Catalog.

Specified by:

insert in interface TransformationCatalog

Parameters:

tcentry - Takes a single TransformationCatalogEntry object as input

Returns:

number of insertions, should always be 1. On failure, throw an exception, don't use zero.

Throws:

Exception

See Also:

edu.isi.pegasus.planner.catalog.TransformationCatalogEntry

insert

public int insert(TransformationCatalogEntry entry,
                  boolean write)
           throws Exception

Add single TCEntry object temporarily to the in memory Catalog. This is a hack to get around for adding soft state entries to the TC

Specified by:

insert in interface TransformationCatalog

Parameters:

tcentry - Takes a single TransformationCatalogEntry object as input

write - boolean enable write commits to backed catalog or not.

Returns:

number of insertions, should always be 1. On failure, throw an exception, don't use zero.

Throws:

Exception

See Also:

edu.isi.pegasus.planner.catalog.TransformationCatalogEntry

insert

public int insert(String namespace,
                  String name,
                  String version,
                  String physicalname,
                  TCType type,
                  String resourceid,
                  List pfnprofiles,
                  List lfnprofiles,
                  SysInfo system)
           throws Exception

Add an single entry into the transformation catalog.

Specified by:

insert in interface TransformationCatalog

Parameters:

namespace - String The namespace of the transformation to be added (Can be null)

name - String The name of the transformation to be added.

version - String The version of the transformation to be added. (Can be null)

physicalname - String The physical name/location of the transformation to be added.

type - TCType The type of the physical transformation.

resourceid - String The resource location id where the transformation is located.

lfnprofiles - List The List of Profile objects associated with a Logical Transformation. (can be null)

pfnprofiles - List The List of Profile objects associated with a Physical Transformation. (can be null)

sysinfo - SysInfo The System information associated with a physical transformation.

Returns:

number of insertions, should always be 1. On failure, throw an exception, don't use zero.

Throws:

Exception

See Also:

edu.isi.pegasus.planner.catalog.TransformationCatalogEntry, SysInfo, org.griphyn.cPlanner.classes.Profile

addTCEntry

protected boolean addTCEntry(String namespace,
                             String name,
                             String version,
                             String physicalname,
                             TCType type,
                             String resourceid,
                             List pfnprofiles,
                             List lfnprofiles,
                             SysInfo system,
                             Notifications invokes,
                             boolean write)
                      throws Exception

Add an single entry into the transformation catalog.

Parameters:

namespace - the namespace of the transformation to be added (Can be null)

name - the name of the transformation to be added.

version - the version of the transformation to be added. (Can be null)

physicalname - the physical name/location of the transformation to be added.

type - the type of the physical transformation.

resourceid - the resource location id where the transformation is located.

lfnprofiles - the List of Profile objects associated with a Logical Transformation. (can be null)

pfnprofiles - the list of Profile objects associated with a Physical Transformation. (can be null)

system - the System information associated with a physical transformation.

invokes - the Notifications associated with the transformation.

write - boolean to commit changes to backend catalog

Returns:

boolean true if succesfully added, returns false if error and throws exception.

Throws:

Exception

See Also:

org.griphyn.common.catalog.TransformationCatalogEntry, SysInfo, org.griphyn.cPlanner.classes.Profile

addLFNProfile

public int addLFNProfile(String namespace,
                         String name,
                         String version,
                         List profiles)
                  throws Exception

Add additional profile to a logical transformation .

Specified by:

addLFNProfile in interface TransformationCatalog

Parameters:

namespace - String The namespace of the transformation to be added. (can be null)

name - String The name of the transformation to be added.

version - String The version of the transformation to be added. (can be null)

profiles - List The List of Profile objects that are to be added to the transformation.

Returns:

number of insertions. On failure, throw an exception, don't use zero.

Throws:

Exception

See Also:

org.griphyn.cPlanner.classes.Profile

addPFNProfile

public int addPFNProfile(String pfn,
                         TCType type,
                         String resourcename,
                         List profiles)
                  throws Exception

Add additional profile to a physical transformation.

Specified by:

addPFNProfile in interface TransformationCatalog

Parameters:

pfn - String The physical name of the transformation

type - TCType The type of transformation that the profile is associated with.

resourcename - String The resource on which the physical transformation exists

profiles - The List of Profile objects that are to be added to the transformation.

Returns:

number of insertions. On failure, throw an exception, don't use zero.

Throws:

Exception

See Also:

org.griphyn.cPlanner.classes.Profile

removeByLFN

public int removeByLFN(String namespace,
                       String name,
                       String version,
                       String resourceid,
                       TCType type)
                throws Exception

Delete all entries in the transformation catalog for a give logical tranformation and/or on a resource and/or of a particular type

Specified by:

removeByLFN in interface TransformationCatalog

Parameters:

namespace - String The namespace of the transformation to be deleted. (can be null)

name - String The name of the transformation to be deleted.

version - String The version of the transformation to be deleted. ( can be null)

resourceid - String The resource id for which the transformation is to be deleted. If NULL then transformation on all resource are deleted

type - TCType The type of the transformation. If NULL then all types are deleted for the transformation.

Returns:

the number of removed entries.

Throws:

Exception

See Also:

TCType

removeByPFN

public int removeByPFN(String physicalname,
                       String namespace,
                       String name,
                       String version,
                       String resourceid,
                       TCType type)
                throws Exception

Delete all entries in the transformation catalog for pair of logical and physical transformation.

Specified by:

removeByPFN in interface TransformationCatalog

Parameters:

physicalname - String The physical name of the transformation

namespace - String The namespace assocaited in the logical name of the transformation.

name - String The name of the logical transformation.

version - String The version number of the logical transformation.

resourceid - String The resource on which the transformation is to be deleted. If NULL then it searches all the resource id.

type - TCType The type of transformation. If NULL then it search and deletes entries for all types.

Returns:

the number of removed entries.

Throws:

Exception

See Also:

TCType

removeByType

public int removeByType(TCType type,
                        String resourceid)
                 throws Exception

Delete a particular type of transformation, and/or on a particular resource

Specified by:

removeByType in interface TransformationCatalog

Parameters:

type - TCType The type of the transformation

resourceid - String The resource on which the transformation exists. If NULL then that type of transformation is deleted from all the resources.

Returns:

the number of removed entries.

Throws:

Exception

See Also:

TCType

removeBySysInfo

public int removeBySysInfo(SysInfo sysinfo)
                    throws Exception

Deletes entries from the catalog which have a particular system information.

Specified by:

removeBySysInfo in interface TransformationCatalog

Parameters:

sysinfo - SysInfo The System Information by which you want to delete

Returns:

the number of removed entries.

Throws:

Exception

See Also:

SysInfo

removeBySiteID

public int removeBySiteID(String resourceid)
                   throws Exception

Delete all entries on a particular resource from the transformation catalog.

Specified by:

removeBySiteID in interface TransformationCatalog

Parameters:

resourceid - String The resource which you want to remove.

Returns:

the number of removed entries.

Throws:

Exception

clear

public int clear()
          throws Exception

Deletes the entire transformation catalog. CLEAN............. USE WITH CAUTION.

Specified by:

clear in interface TransformationCatalog

Returns:

the number of removed entries.

Throws:

Exception

deletePFNProfiles

public int deletePFNProfiles(String physicalname,
                             TCType type,
                             String resourceid,
                             List profiles)
                      throws Exception

Delete a list of profiles or all the profiles associated with a pfn on a resource and of a type.

Specified by:

deletePFNProfiles in interface TransformationCatalog

Parameters:

physicalname - String The physical name of the transformation.

type - TCType The type of the transformation.

resourceid - String The resource of the transformation.

profiles - List The list of profiles to be deleted. If NULL then all profiles for that pfn+resource+type are deleted.

Returns:

the number of removed entries.

Throws:

Exception

See Also:

org.griphyn.cPlanner.classes.Profile

deleteLFNProfiles

public int deleteLFNProfiles(String namespace,
                             String name,
                             String version,
                             List profiles)
                      throws Exception

Delete a list of profiles or all the profiles associated with a logical transformation.

Specified by:

deleteLFNProfiles in interface TransformationCatalog

Parameters:

namespace - String The namespace of the logical transformation.

name - String The name of the logical transformation.

version - String The version of the logical transformation.

profiles - List The List of profiles to be deleted. If NULL then all profiles for the logical transformation are deleted.

Returns:

the number of removed entries.

Throws:

Exception

See Also:

org.griphyn.cPlanner.classes.Profile

connect

public boolean connect(Properties props)

Description copied from interface: Catalog

Establishes a link between the implementation and the thing the implementation is build upon.

FIXME: The cause for failure is lost without exceptions.

Specified by:

connect in interface Catalog

Parameters:

props - contains all necessary data to establish the link.

Returns:

true if connected now, or false to indicate a failure.

isClosed

public boolean isClosed()

Description copied from interface: Catalog

Predicate to check, if the connection with the catalog's implementation is still active. This helps determining, if it makes sense to call close().

Specified by:

isClosed in interface Catalog

Returns:

true, if the implementation is disassociated, false otherwise.

See Also:

Catalog.close()

close

public void close()

Description copied from interface: Catalog

Explicitely free resources before the garbage collection hits.

Specified by:

close in interface Catalog

writeTC

private void writeTC()

populateTC

private boolean populateTC(InputStream reader)

Populates the internal copy of the transformation catalog from a byte stream (input stream). Used in webservices, when clients upload their files. It uses the default character encoding.

Parameters:

reader - the InputStrean containing the bytes to be read.

Returns:

boolean

populateTC

private boolean populateTC()

Populates the internal copy of the transformation catalog from the file containing the transformation catalog in the 6 column format.

Returns:

boolean

populateTC

private boolean populateTC(Reader reader)

Adds multiple entries into the TC. Calls the above api multiple times.

Parameters:

reader - the input stream from where to read the contents of the transformation catalog.

Returns:

boolean

logMessage

protected void logMessage(String msg)

Logs the message to a logging stream. Currently does not log to any stream.

Parameters:

msg - the message to be logged.