Formative assessment using diagrammatic code queries: engaging students with a notional machine

Introductory Computer Science classes often start by teaching students to write simple programs in the language being taught. The ability to competently write code that solves problems encompasses many skills and foundational concepts. Students need a basic understanding of how the programming language is structured and how the underlying machine executes the instructions. This mental model of execution of a specific programming language is called a `notional machine' in computer science. My research investigates how helping students form accurate mental models early affects their course performance and experience. I used a framework we created to guide Code Diagram Queries (CDQs) development and then deployed the questions in an introductory programming class. Code Diagram Queries (CDQs) are a visual formative tool that I designed to aid students in developing accurate mental models of programming language NMs. I hypothesized that these questions would act as a forcing function to engage students with diagrammatic material that challenged and corrected their current mental models and addressed misconceptions about the notional machine of the language they learned. I describe my experiment and how I indirectly measured student mental model development through course performance and examined question-type effectiveness. Results from my study suggest CDQs helped to reveal misconceptions in the topics related to the flow of execution programming concepts and a slight increase in PythonTutor usage for the section with frequent exposure to it. These findings suggest that CDQs may enhance understanding of some concepts and indicate that CDQs could have a place in post-secondary classrooms.