Agent HAnS: Using feature traceability to improve comprehension and trust in LLM-based coding

Loading...
Thumbnail Image

Date

Type

Examensarbete för masterexamen
Master's Thesis

Model builders

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

As LLM-based coding projects grow large comprehension and in turn trust of generated code is reduced. Feature models and embedded feature annotations are established concepts for getting an overview of the features present in a codebase. Both of these however require constant maintenance to reflect their associated codebase. Through a three cycle Design Science Research methodology a MCP-based tool, Agent HAnS, is iteratively implemented and evaluated. Agent HAnS automatically generates and maintains feature models and embedded feature annotations in a LLM-based software development workflow. In addition Agent HAnS presents a graphical dashboard between prompts highlighting changes made to the source code from a feature-oriented perspective. To evaluate the tool, a user study and a correctness evaluation was conducted. The results of the user study show no significant effect on trust and comprehension but that Agent HAnS is usable and does not impact the overall effort needed. The correctness evaluation shows that feature models generated by Agent HAnS varies in quality, sometimes equivalent to a human defined baseline and other times misses critical features. A more extensive study would be better positioned to assess the potential benefits of Agent HAnS.

Description

Keywords

LLM, features, feature models, embedded feature annotations, vibe cod ing, agentic coding

Citation

Architect

Location

Type of building

Build Year

Model type

Scale

Material / technology

Index

Endorsement

Review

Supplemented By

Referenced By