Creating reproducible, scalable, portable and adaptable CFD validation workflows with Snakemake

The modelling and simulation of nuclear reactors is increasingly carried out with open-source tools. To researchers, the most appealing benefits are the possibility to study and verify existing implementations and the freedom to add code for prototyping new concepts and models. Further, there is no direct dependency on a software manufacturer, which generally makes open-source codes a solid basis for collaboration. An important downside however is that the responsibility of controlling quality and reliability lies with the users. For actively developed code this is a continuous task and demands considerable resources which should be bundled.
In the area of Computational Fluid Dynamics, a well-established open-source solution is OpenFOAM. Its development follows agile principles with a strong focus on maintainability and usability. This increases the demands on users to keep their setups functional. Since 2020, HZDR is creating an IT environment that supports the centralized and continuous maintenance of OpenFOAM code and simulation setups developed by members of the German nuclear safety research community. The project focuses on work relevant for the reactor cooling system and is referred to as OpenFOAM_RCS. Both, an addon to the OpenFOAM release from The OpenFOAM Foundation and a dedicated repository for simulation setups are supplied. The basis for the project is the web-based software development environment GitLab provided by the Helmholtz Federated IT Services (HIFIS), which allows for a high degree of automation with the help of continuous integration and deployment pipelines. Part of the project is the creation of a validation workflow
Validation workflows should, among other things: (1) allow for continuous and automated checks of the setup input syntax; (2) frequently validate the simulation results by comparison against reference solutions; (3) be able to evaluate the impact of model substitutions on the overall “goodness” of results; (4) automatically create self-contained reports; (5) be executable on different environments, i.e. be portable and scalable; (5) allow for easy integration of new setups, i.e. be adaptable; (6) enable design point studies; (7) be comprehensible, ideally be self-documenting; (8) follow well-defined standards.
All aforementioned requirements can be fulfilled with the popular and actively developed Python-based workflow management system Snakemake (Mölder et al., 2021), which was originally written for Bioinformatics workflows, but is far more versatile. In this work Snakemake is configured to embed all setups archived in OpenFOAM_RCS in a single workflow. Execution of simulation setups is controlled via so-called Snakefiles, which list inputs and outputs of a simulation, the required resources as well as the commands to run them. Within a top-level configuration file utilizing the easy to read markup language YAML, all setups to be incorporated in a workflow are listed. Once properly configured, users can run the workflow in four steps: (1) configuration, i.e. assembly of selected setups; (2) running the simulations; (3) post-processing the simulations; (4) creating HTML reports gathering all generated plots together with runtime statistics and provenance information. The third step can be executed on a workstation or a cluster. For the latter case, a top-level job organizes the workflow and automatically submits sub-jobs to the cluster.
The proposed framework provides the opportunity to reduce cost and effort for CFD validation and verification. It also is of great help for complex model development such as multi-phase CFD where a large range of test cases and test conditions is mandatory to judge the performance of a given model choice.

This work is carried out in the frame of a current research project funded by the German Federal Ministry for Environment, Nature Conservation, Nuclear Safety and Consumer Protection, project number 1501604.