Adaptable Research and Data Management Platform for High Data-Rate Experiments

At the High-Field High-Repetition-Rate Terahertz facility (TELBE) [1] at the Helmholtz-Zentrum Dresden-Rossendorf, ultrafast terahertz-induced dynamics can be probed in various states of matter with highest precision. The TELBE source is available for external users who are not familiar with the experiments's data management pipelines. This is especially a challenge as the measurements at TELBE are data intensive, producing as much as 20GB per experiment over few minutes.

In this contribution, we present the guidance system HELIPORT [3] which manages the metadata of the associated project proposal and integrates systems for workflow management, electronic lab documentation and others. We showcase the integrated workflow for post-processing of the experimental data at TELBE with in-built exchange of metadata between our proposal submission service, the experiment control software and workflows managed and executed using UNICORE [2].

Among other information, HELIPORT integrates documentation, scientific workflows, and the final publication of the research results - all via already established solutions for proposal management, electronic lab notebooks, software development and devops tools, and other additional data sources. The integration is accomplished by presenting the researchers with a high-level overview to keep all aspects of the experiment in mind, and automatically exchanging relevant metadata between the experiment's life cycle steps.

By visualizing all aspects of large-scale research experiments, HELIPORT enables deeper insights into a comprehensible data provenance with the chance of raising awareness for FAIR data management.

[1] https://doi.org/10.1063/1.4978042
[2] https://doi.org/10.1109/HPCSim.2016.7568392
[3] https://doi.org/10.1145/3456287.3465477