Automated Change Impact Analysis for Deep Learning Programs

<p dir="ltr">Deep learning (DL) is the core technology underlying many Artificial Intelligence (AI) applications. Maintaining deep learning software programs presents unique challenges due to their complex, interdependent structures. Traditional Change Impact Analysis (CIA) methods, while effective for conventional software, fall short when they are applied to deep learning programs due to the intricate interdependencies within neural networks, particularly those related to the unique architectural layers and hyperparameters. As AI deep learning applications continue to expand in many fields such as healthcare, finance and autonomous systems, maintaining these software programs becomes challenging since any modification can have far-reaching impacts on overall performance and stability. This research presents a novel approach to CIA tailored specifically for deep learning programs. We introduce a comprehensive taxonomy that systematically categorizes changes in deep learning architectures, hyperparameters, and regularization techniques.</p><p dir="ltr">Our approach leverages this taxonomy to map changes automatically, thereby enabling structured and efficient impact analysis in complex deep learning programs. To predict the potential impact of modifications, the framework incorporates co-change pattern mining to identify and generate co-change rules, capturing dependencies based on historical co-change patterns. We also designed domain-expert rules derived from relevant literature, which encode specific co-change patterns pertinent to neural network structures. The framework’s hybrid approach combines these co-change patterns and domain-expert rules to generate accurate predictions, aligning data-driven insights with domain-specific knowledge. Additionally, we incorporate a Large Language Model (LLM) to enhance the semantic understanding of these changes, improving recall by detecting relevant impacts while addressing limitations in conventional CIA methods.</p><p dir="ltr">To validate this approach, we constructed a dataset from GitHub commits in Keras-based projects, using real-world modifications to test the predictive accuracy of our CIA framework. Our evaluation shows that the hybrid approach significantly enhances recall, achieving an average recall of 0.932 (93.2%) at optimal support and confidence thresholds. The LLM-based component demonstrated an average recall of 0.656 (65.6%) but with limited precision. These results highlight the utility of a combined approach, capturing a broad range of impactful changes while minimizing irrelevant suggestions.</p><p dir="ltr">The contributions of this thesis are multi-faceted: a deep learning-specific taxonomy for Change Impact Analysis, an automated mapping of changes within this taxonomy, and a recommendation system for impact prediction. The findings suggest that the proposed CIA approach has significant potential to support AI software developers in managing deep learning model evolution by providing actionable insights and recommendations, consequently reducing the risks associated with software updates.</p>