Publications

A novel shard-based approach for asynchronous many-task models for in situ analysis*

Pébaÿ, P.P.; Borghesi, G.; Kolla, Hemanth K.; Bennett, Janine C.; Treichler, S.

We present the current status of our work towards a scalable, asynchronous many-task, in situ statistical analysis engine using the Legion runtime system, expanding upon earlier work, that was limited to a prototype implementation with a proxy mini-application as a surrogate for a full-scale scientific simulation code. In contrast, we have more recently integrated our in situ analysis engines with S3D, a full-size scientific application, and conducted numerical tests therewith on the largest computational platform currently available for DOE science applications. The goal of this article is thus to describe the SPMD-Legion methodology we devised in this context, and compare the data aggregation technique deployed herein to the approach taken within our previous work.