Coding with AI: How are tools like ChatGPT being used by students in foundational programming courses

Tools based on generative artificial intelligence (AI), such as ChatGPT, have quickly become commonplace in education, particularly in tasks like programming. We report on a study exploring how students use a tool similar to ChatGPT, powered by GPT-4, while working on Introductory Computer Programming (CS1) assignments, addressing a gap in empirical research on AI tools in education. Utilizing participants from two CS1 class sections, our research employed a custom GPT-4 tool for assignment assistance and the ShowYourWork plugin for keystroke logging. Prompts, AI replies, and keystrokes during assignment completion were analyzed to understand the state of students’ programs when they prompt the AI, the types of prompts they create, and whether and how students incorporate the AI responses into their code. The results indicate distinct usage patterns of ChatGPT among students, including the finding that students ask the AI for help on debugging and conceptual questions more often than they ask the AI to write code snippets or complete solutions for them. We hypothesized that students ask conceptual questions near the beginning and debugging help near the end of program development do not find statistical evidence to support it. We find that large numbers of AI responses are immediately followed by the student copying and pasting the response into their code. The study also showed that tools like these are widely accepted and appreciated by students and deemed useful according to a student survey - suggesting that the integration of AI tools can enhance learning outcomes and positively impact student engagement and interest in programming assignments.