You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 60 Next »

Purpose

The Workflow Broker (formerly "Ensemble Broker") is a GSI web service designed to manage large computational jobs on production hosts; its features include

  • Highly expressive description logic which allows for
    • Structuring workflow in two independent layers:
      • Top level (broker-centric): a directed acyclical graph of executable nodes;
      • Individual node payloads: scripts to be run on the production resource, usually inside an Elf container;
    • Configuration of scheduling and execution properties on a node-by-node basis;
  • Integration with scheduling systems, such as MOAB;
  • (Potentially recursive) Parameterized expansion of nodes;
  • Staging, submission and monitoring of jobs;
  • Cancellation and restart of nodes or entire workflows.

The broker is designed to be extensible in terms of

  • Type of node payload (currently ELF/Ogrescript and C-Shell are available);
  • Type of launching and monitoring protocols used (currently pre-WS GRAM, GSI-SSH to the native batch system, and local submission to a MOAB peer are available).

  • No labels