Presentation
Universal Data Junction: A Transport Layer for Data Driven Workflows
Presenter
DescriptionA novel transport library for the efficient coupling of applications through their data dependencies is presented. The design is driven by the intent to require minimal changes to existing scientific applications and to declare the data objects that are meaningful for other applications for read and write as well as to perform transparent transport including automatic redistribution of parallel data structures, thus permitting seamless coupling of applications in workflows. The actual transport can be selected at run time, and can exploit a variety of data exchange methods, including MPI, Dataspaces, Ceph Rados, CRAY Datawarp, and a POSIX file system. For the case of MPI transport, the library is used to implement the first stage of a co-working visualization pipeline for CP2K and results show a significant advantage compared to a filesystem based approach.
TimeTuesday, June 2715:00 - 15:30 CEST
LocationDischma
SessionAP2A - ACM Papers Session 2A
Session Chair
Event Type
Paper
Chemistry and Materials
Life Sciences
