...
The Workflow Broker (formerly "Ensemble Broker") is a GSI web service designed to orchestrate and manage complex sequences of large computational jobs on production hostsresources.
Features
- Expressive (XML) description logic which allows for
- Structuring workflow in two independent layers:
- Top level (broker-centric): a directed acyclical graph of executable nodes;
- Individual node payloads: scripts to be run on the production resource, usually inside an Elf container;
- Configuration of scheduling and execution properties on a node-by-node basis;
- Structuring workflow in two independent layers:
- Integration with scheduling systems, such as MOAB;
- (Potentially recursive) service-side parameterized expansion of nodes (done lazily, as the nodes become ready to run);
- Fully asynchronous staging, submission and monitoring of jobs;
- State management through a persistent object-relational layer (Hibernate);
- Inspection of workflow state enabled via event bus as well as API calls on the borker service itself;
- Cancellation and restart of nodes or entire workflows.
...