Maximizing Throughput on a Dragonfly Network

Interconnection networks are a critical resource for large supercomputers. The dragonfly topology, which provides a low network diameter and large bisection bandwidth, is being explored as a promising option for building multi-Petaflop/s and Exaflop/s systems. Unlike the extensively studied torus networks, the best choices of message routing and job placement strategies for the dragonfly topology are not well understood. This paper aims at analyzing the behavior of an interconnect based on the dragonfly topology for various routing strategies, job placement policies, and application communication patterns. Our study is based on a novel model that predicts traffic on individual links for direct, indirect, and adaptive routing strategies. We analyze results for individual communication patterns and some common parallel job workloads. The predictions presented in this paper are for a 100+ Petaflop/s prototype machine with 92,160 high-radix routers and 8.8 million cores.

@inproceedings{jain_sc:2014,
 author = {Jain, Nikhil and Bhatele, Abhinav and Ni, Xiang and Wright, Nicholas J. and Kale, Laxmikant V.},
 title = {Maximizing Throughput on a Dragonfly Network},
 booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis},
 series = {SC '14},
 year = {2014},
 isbn = {978-1-4799-5500-8},
 location = {New Orleans, Louisana},
 pages = {336--347},
 numpages = {12},
 url = {http://dx.doi.org/10.1109/SC.2014.33},
 doi = {10.1109/SC.2014.33},
 acmid = {2683631},
 publisher = {IEEE Press},
 address = {Piscataway, NJ, USA},
 keywords = {dragonfly networks, job placement, modeling, prediction, routing},
}