| Table of Contents | ||
|---|---|---|
|
Overview
SAGE is a non-domain specific application framework that is built upon Bard, PTPFlow, Tupelo, and MyProxy for setting up, launching and managing HPC application workflows through an easy to use set of user interfaces. This document is intended to lay the foundation of the core components and views provided by the SAGE application framework and inform users how they can extend the various parts for their domain specific application.
...
This scenario bean will evolve as the application framework is built and more final documentation will be put here as the design matures. The main parts of this bean are: DatasetBean's will be used to manage all of the input/output datasets, the RMIServiceBean (described later) will contain the service information and the WorkflowBean will contain the workflows associated with this scenario. A user might extend the ScenarioBean if their application has other things that logically belong to their scenarios.
Service Manager
RMI Service Registry View
This view shows all of the machines defined as available to the user for installing the RMI service and PTPFlow plugins required to run HPC jobs and return status information to the client.
RMIService Info Bean
The information about each service installation will be stored in an RMIServiceBean and will be used to launch and start the service. All of this information is currently used in PTPFlow and is stored in xml files. Bringing in tupelo to the service stack will allow us to store this information in tupelo.
| Code Block | ||
|---|---|---|
| ||
// Service Info
private String name;
private String platform;
private String deployUsingURI; // e.g. file:/
private String launchUsingURI;
private String installLocation; // e.g. /home/user_home/ptpflow
private String rmiContactURI;
private int rmiPortLowerBound;
private int rmiPortUpperBound;
private int gridftpPortLowerBound;
private int gridftpPortUpperBound;
private Date installedDate;
private boolean running;
private Set<HostResourceBean> knownHosts; // all of the known hosts associated with this service
|
...
Workflows Bean
Each workflow is described in an XML file that outlines the steps in the process including which resource to run on, executables that will be launched, input files to use, etc. Initially, we will simply store the workflow information in a single WorkflowStepBean that has a reference to the file containing the xml and the DatasetBeans. Ogrescript xml files can be complex, but if we can logically separate out the steps or parts into individual beans that can be used to generate the full workflow xml file required by the HPC machines, then we can include workflow steps as separate beans and provide a UI for adding steps dynamically.
...
| Code Block | ||
|---|---|---|
| ||
private String title;
private PersonBean creator;
private Date date;
private List<DatasetBean> inputDatasets; // all data inputs associated with this step
private DatasetBean workflow; // initially our steps will only include a single step, the entire workflow
|
Service Manager
...
RMI Service Registry View
This view shows all of the machines defined as available to the user for installing the RMI service and PTPFlow plugins required to run HPC jobs and return status information to the client.
RMIService Info Bean
The information about each service installation will be stored in an RMIServiceBean and will be used to launch and start the service. All of this information is currently used in PTPFlow and is stored in xml files. Bringing in tupelo to the service stack will allow us to store this information in tupelo.
| Code Block | ||
|---|---|---|
| ||
// Service Info private String name; private String platform; private String deployUsingURI; // e.g. file:/ private String launchUsingURI; private String installLocation; // e.g. /home/user_home/ptpflow private String rmiContactURI; private int rmiPortLowerBound; private int rmiPortUpperBound; private int gridftpPortLowerBound; private int gridftpPortUpperBound; private Date installedDate; private boolean running; private Set<HostResourceBean> knownHosts; // all of the known hosts associated with this service |
Repository Manager
...
Repository View
Rather than a single repository view, this will be multiple views that are configured to show a particular type of bean(s) coming from a content provider. The content provider would get the data required from the configured tupelo context(s). For example, we will need a "Dataset Repository View" that shows all datasets (e.g. input/output datasets) and a way to manipulate them (e.g. add tags, annotations, etc), "Workflow Repository View" that shows all imported workflows, "Scenario Repository View" that shows all saved scenarios, "Service Repository View" that shows defined RMI service endpoints for launching jobs, and a "Known Hosts View" for showing known hosts that can accept jobs. This seems like too much disparate information to display in a single view. All Repository views will descend from BardFrameView since the BardFrame will be required to get the data required for each view.
...
A critical requirement is that repositories can be both remote and local and users might use more than one simultaneously. Input data for workflows that is managed by Tupelo will need to move from the users machine to a location that the HPC machine can access. Datasets should also be returned to the users scenario or made available to it.
Host Manager
...
Known Host View
This view contains a list of defined HPC hosts that the user can launch jobs on. This view will provide the user with the ability to change/view/add properties such as environment settings, user information for the host (username, user home, etc), host operating system, node properties, new hosts, etc. These changes should be propogated to the defined RMI services so they can be used immediately.
Known Host Bean
Below is the bean structure that is anticipated:
...