MODELING THE PERFORMANCE OF DIRECT NUMERICAL SIMULATION ON PARALLEL SYSTEMS
Abstract
Direct Numerical Simulation (DNS) is an important application area that is expected to use large fractions of future large-scale simulations. In this work we develop, validate and use a performance model of the combustion code, DNS3D, to explore achieved performance on current parallel systems. The performance model is developed from a thorough analysis of the application. Its key computation characteristics are coupled with the performance characteristics of the system using an parameterized analytical model. The model is validated on three parallel systems: a muti-core AMD Opteron based system with an Infiniband fat-tree network, an IBM Power5+ system with an HPS fat-tree network, and an IBM Power7 system with a direct connect network. The performance model is shown to achieve high prediction accuracy on all three systems. We illustrate how the model can be used to explore impact of changes in either the system or the application. It is used to both analyze the achieved performance on these systems as well as to explore the possible benefits of further optimizing DNS3D's main computational kernel of one-dimensional FFTs, or in possibly overlapping communication with computation.
References
- J. Turbulence 4, (2003). Google Scholar
-
D. A. Donzis , P. K. Yeung and D. Pekurovsky , Turbulence simulations on O(104) processors , Proc. TeraGrid . Google Scholar - National Science Foundation, Leadership-Class System Acquisition Creating a Petascale Computing Environment for Science and Engineering, NSF06573 (2006) . Google Scholar
- Los Alamos Science 29, 142 (2005). Google Scholar
- IEEE Computer 42(11), 42 (2009). Crossref, ISI, Google Scholar
-
D. J. Kerbyson , Predictive Performance and Scalability Modeling of a Large-Scale Application , proc. IEEE/ACM SuperComputing (SC01) . Google Scholar - int. J. of High Performance Computing Applications 14(4), 330 (2000). Crossref, ISI, Google Scholar
- Int. J. High Performance Computing Applications 19(5), 261 (2005). Crossref, ISI, Google Scholar
D. J. Kerbyson , K. J. Barker and K. Davis , Analysis of the Weather Research and Forcasting (WRF) Model on Large-scale Systems, Parallel Computing: Architectures, Algorithms and Applications38,NIC (IOS Press, Juelich, Germany, 2007) pp. 89–98. Google Scholar-
A. Hoisie , A Performance Comparison through Benchmarking and Modeling of Three Leading Supercomputers: Blue Gene/L, Red Storm, and Purple , proc. IEEE/ACM Supercomputing (SC06) . Google Scholar -
K. J. Barker , Entering the Petaflop Era: The Architecture and Performance of Roadrunner , proc. IEEE/ACM Supercomputing (SC08) . Google Scholar - D. J. Kerbyson and K. J. Barker, A Performance Model of Direct Numerical Simulation for Analyzing Large-Scale Systems, in proc. workshop on Large-Scale Parallel Processing (LSPP), Int. Parallel and Distributed Processing Symp. (IPDPSW), Anchorage (2011) . Google Scholar
B. Arimilli , The PERCS High-Performance Interconnect, proc. 18th IEEE Symp. on High Performance Interconnects pp. 75–85. Google Scholar-
D. J. Kerbyson and K. J. Barker , Analyzing the Performance Bottlenecks of the POWER7-IH Network , proc. IEEE Cluster . Google Scholar - Blue Waters Sustained Petascale Computing, Project Office, National Center for Supercomputing Applications, IL (2011) , http://www.ncsa.illinois.edu/BlueWaters . Google Scholar


