Software Variability for Replicable Science

The ability to recreate computational results provides a solid foundation for scientific research. Yet, reproducibility and getting identical results with the original data, code, and environment are challenging, due to many uncontrolled or undocumented variability factors. When reproduction is achieved, replicability can be envisioned to check whether conclusions still hold or generalize when well-controlled changes—created by inevitable software variability—are introduced. In this talk, I will first provide evidence that deep software variability — spanning operating systems, compilers, input data, versions, and configurations, etc. — is impacting reproducibility and replicability in numerous fiels of computational science. I will then present an approach based on "modelling, sampling, measuring, learning" to systematically explore variability spaces of neuroimaging pipelines. I will also illustrate how Masters` students et INSA Rennes leverage variability to reproduce and replicate studies in soccer, chess, and energy consumption.
Throughout this talk, I will try to convince the audience that software-engineering and variability researchers have a key role to play for truly replicable science.

Mathieu Acher is Professor at University of Rennes/IRISA/Inria, France. His research focuses on modelling, reverse engineering, and learning (deep) variability of software-intensive systems. Beyond its applicability, his research is original in combining software engineering and artificial intelligence techniques (symbolic reasoning, machine learning, generative AI). He is the author of more than 150 peer-reviewed publications in international journals and conferences. His work has received Most Influential Paper Award (SLE’19) and Best Paper Awards (SPLC’13, ICPE’19, SPLC’21, ICSR’22, MODELS’23, AST'24, ISSTA'24). Since 2021, he is a junior research fellow at Institut Universitaire de France (IUF). He is also co-chairing the défi Inria LLM4Code (2024-2028).