Dr. Marrinan Presents Research at SuperComputing Conference
CISC Assistant Professor Thomas Marrinan recently gave a talk at the International Conference for High Performance Computing, Networking, Storage and Analysis (SuperComputing) in Denver, Colorado. He also attended the Early Career Program, which aimed to help participants secure a better understanding of the issues and challenges they are likely to face while navigating a successful research career.
Since its inception, the SuperComputing Conference has been dedicated to showcasing work in high performance computing, networking, storage, and analysis by the international HPC community. The core of the Conference is the Technical Program with peer-reviewed content for every track; a series of Student Programs to engage and foster the next generation of HPC professionals; community awards to honor researchers in the field; and the fastest research network, SCinet, built to support high performing applications and demonstrations during the Conference. The full title and abstract are below.
Parallel Streaming for In Transit Analysis with Heterogeneous Data Layout
Performing analysis or generating visualizations concurrently with high performance simulations can yield great benefits compared to post-processing data. Writing and reading large volumes of data can be reduced or eliminated, thereby producing an I/O cost savings.
One such method for concurrent simulation and analysis is in transit - streaming data from the resource running the simulation to a separate resource running the analysis. In transit analysis can be beneficial since computational resources may not have certain resources needed for visualization and analysis (e.g. GPUs) and to reduce the impact of performing analysis tasks to the run time of the simulation. When sending and receiving data in transit, data redistribution mechanisms are needed in order to support heterogeneous data layouts that may be required by the simulation and analysis applications.
The work described in this paper compares two mechanisms for on-the-fly data redistribution when streaming data in parallel between two distributed memory applications. Our results show that it is often advantageous to stream data in the same layout as the sender and redistribute data amongst processes on the receiving end than to stream data in the final layout needed by the receiver.