...
- EBS vs S3
- In the 80-core experiment, a run of processing took ~7 hours for EBS variants and only 3 hours for S3.
- Output data loading time into S3 is an order of magnitude smaller than into EBS.
- Cost: data transfers between EC2 and S3 are free d. S3 storage is better than EBS for SNfactory.
- Input data and application data will be stored in EBS and output data will be writen to S3.
Issues/Gaps
- Need to replicate HPC cluster environment in EC2 or the application must be modified.
- Mean rate of failure is higher in EC2 than in traditional cluster environments which needs to be handled.
- Inability to acquire all of the VMI requested because insufficient resources are available, so need to modify the application to adapt this.
- Transient errors.
References
- Berriman, G.B. et al. Sixth IEEE International Conference on e-Science, 1-7 (2010)
- Berriman, G.B. et al. SPIE Conference 7740: Software and Cyberinfrastructure for Astronomy (2010)
- Juve, G. et al. Cloud Computing Workshop in Conjunction with e-Science Oxford, UK: IEEE (2009)
- Jackson, K.R. et al. Proc. ACM Int. Symp. HPDC, 421-429 (2010)
...