https://hdl.handle.net/10388/18675

Alam, Farhana (2026) Generative AI in Programming Education_Examining Behavioral Strategies, Critical Thinking, and Retention, Harvest - Theses & Dissertations, https://hdl.handle.net/10388/18675

The integration of Generative Artificial Intelligence (GenAI) tools into Computer Science (CS) education raises fundamental questions about how AI-mediated assistance shapes student reasoning, comprehension, and retention. This thesis presents an exploratory, pilot-scale investigation into how GenAI use may be associated with differences in Critical Thinking (CT), debugging effectiveness, and knowledge retention among senior (3rd and 4th year) undergraduate CS students. To investigate these questions, this thesis employs a multi-method design combining behavioral performance measures, AI interaction log analysis, a validated CT instrument, and a delayed no-AI retention test. This combination is largely absent from prior programming education research. A controlled experiment was conducted with senior undergraduate CS students (N = 13) randomly assigned to an AI-assisted group (n = 6) or an unassisted control group (n = 7). In the first session, participants completed a coding task under their assigned condition, followed by a no-AI debugging task, a structured reflection questionnaire, and the WatsonGlaser Critical Thinking Appraisal (WGCTA). Two weeks later in the second session, all participants returned to complete only the same coding task independently, without AI access for both groups. Data were analyzed through quantitative performance comparisons, and reflexive thematic coding of participant reflections and AI prompt logs. Several findings emerged, though all must be interpreted cautiously given the small sample size and exploratory nature of the study. Although the AI-assisted group achieved significantly higher mean baseline coding scores (M = 9.50 vs. M = 6.00, p = 0.033), this advantage did not carry over to debugging, where both groups scored identically (M = 1.00 out of 4). In the two-week retention session, four of six AI-assisted participants showed notable coding score declines once support was removed, while five of seven unassisted participants held steady or improved, a pattern that raises questions about whether AI-assisted performance consistently reflects independent competence, though the small sample size prevents firm conclusions. WGCTA scores did not differ significantly between groups, and the AI group's stronger coding ability did not correspond to higher CT scores, pointing to a partial separation between programming competence and domain-general analytical reasoning. Thematic analysis further revealed that AI-assisted participants more frequently described cognitive offloading and minimal verification of AI outputs, whereas unassisted participants more often reported iterative, self-directed problem-solving. Notably, the one AI participant who used iterative prompting showed markedly higher cognitive engagement, suggesting that how students interact with AI may matter as much as whether they use it at all. Taken together, these patterns suggest that strong task performance may not reflect deep understanding or durable learning, a distinction worth keeping in mind as AI tools become more embedded in these workflows. The central argument of this thesis is that the question 'Does AI improve student performance?' is too narrow. The more consequential question is whether AI improves the reasoning processes that underlie performance, and answering that requires examining how students learn, not only what they produce. This thesis offers that framework as a replicable scaffold for future, larger-scale work, built to make the examination of AI's effect on reasoning processes empirically tractable.

https://hdl.handle.net/10388/18675