| Table of Contents |
|---|
Overview
...
A study of cost and performance of the application of cloud computing to Astronomy 1
The performance of three workflow applications with different I/O, memory and CPU requirements are investigated on Amazon EC2 and the performance of cloud are compared with that of a typical HPC (Abe in NCSA).
The goal is to determine which type of scientific workflow applications are cheaply and efficiently run on the Amazon EC2 cloud.
Also the application of cloud computing to the generation of an atlas of periodograms for the 210,000 light curves is described.
...
|
| Result |
|---|---|---|
Runtimes | Tasks | 631,992 |
| Mean Task Runtime | 6.34 sec |
| Jobs | 25,401 |
| Mean Job Runtime | 2.62 min |
| Total CPU Time | 1,113 hr |
| Total Wall Time | 26.8 hr |
Inputs | Input Files | 210,664 |
| Mean Input Size | 0.084 MB |
| Total Input Size | 17.3 GB |
Outputs | Output Files | 1,263,984 |
| Mean Output Size | 0.124 MB |
| Total Output Size | 76.52 GB |
Cost | Compute Cost | $291.58 |
| Transfer Cost | $11.48 |
| Total Cost | $303.06 |
...
Seeking Supernovae in the Clouds: A Performance Study
...
2
Summary
Nearby Supernova Factory(SNfactory) experiment measures the expansion history of the Universe to explore the nature of Dark Energy with Type Ia supernovae. SNfactory is a pipeline of serial processes executing various image processing algorithms in parallel on ~10TBs of data. SNfactory is ported to Amazon Web Services environment.
...
- Need to replicate HPC cluster environment in EC2 or the application must be modified.
- Mean rate of failure is higher in EC2 than in traditional cluster environments which needs to be handled.
- Inability to acquire all of the VMI requested because insufficient resources are available, so need to modify the application to adapt this.
- Transient errors.
Application of Cloud computing to the creation of image mosaic and management of their provenance 3
Summary
Workflow
Data
Cloud platform
Cloud performance
Issues/Gaps
Scientific workflow applications on Amazon EC2 4
Summary
Workflow
Data
Cloud platform
Cloud performance
Issues/Gaps
References
- Berriman, G.B. et al. Sixth IEEE International Conference on e-Science, 1-7 (2010)
- Jackson, K.R. et al. Proc. ACM Int. Symp. HPDC, 421-429 (2010)
- Berriman, G.B. et al. SPIE Conference 7740: Software and Cyberinfrastructure for Astronomy (2010)
- Juve, G. et al. Cloud Computing Workshop in Conjunction with e-Science Oxford, UK: IEEE (2009)
- Juve, G. et al. SC(2010)
- Wiley, K. et al. Publications of the Astronomical Society of the Pacific 123 366-380 (2011)
- Gaudet, SJackson, K.R. et al. Proc . ACM Int. Symp. HPDCSPIE (2010)
- Ramakrishnan, L. et al. Wands '10 , 421-429 (2010)
Notes and other links
a. Workflow: loosely coupled parallel applications that consist of a set of computational tasks linked by data- and control-flow dependencies.
b. A parallel file system and high-speed interconnect would make dramatic performance upgrades. Recently Amazon released a new resource type including a 10Gb interconnect.
c. There is a movement towards providing academic clouds, such as FutureGrid or Magellan.
d. Only true for intra-zone transfer (before July 1st, 2011). Also the request for data transfer is not free.