Software Quality Evaluation of AI/ML-Based Neuroimaging Tools - A Simulation Study Using BRAPH 2

Abstract

The increasing integration of artificial intelligence (AI) and machine learning (ML) into neuroimaging research has amplified concerns regarding the quality, reproducibility, and trustworthiness of such software systems. Existing evaluation practices often lack standardized and systematic frameworks, limiting their ability to test software quality, especially for AI/ML-based neuroimaging software system. This study addresses this gap by developing and applying a simulation-based framework to evaluate key quality attributes—transparency, functional correctness, and robustness— in AI/ML-based brain imaging analysis pipelines. The framework leverages the Watts–Strogatz network model to generate controlled, simulated brain connectivity datasets, enabling rigorous and repeatable testing. Two analysis pipelines within BRAPH 2 (Brain Analysis using Graph Theory 2), an open-source MATLABbased neuroimaging software for brain network analysis, are tested: a graph theory– based pipeline and a deep learning–based analysis pipeline. Both pipelines are evaluated against simulated datasets, successfully identifying the predefined salient brain regions and maintaining stable performance across repeated runs and random noise situations. Transparency was further enhanced through a graphical user interface and visualization modules that allow inspection of intermediate and final outputs. These results demonstrate that the proposed framework can effectively verify critical quality attributes in a controlled environment. The established methodology provides a robust and accessible foundation for extending validation to real-world neuroimaging datasets and for guiding future st andards in the quality evaluation of AI/ML-based neuroscience software.