Influence of Science Text Reading Difficulty and Hands-On Manipulation on Science Learning: An Eye-Tracking Study

This study used eye‐movement tracking to investigate how students engage with the learning process of reading science articles with or without hands‐on manipulation of a pulley system and their influences on learning outcomes. This experiment used a 2 (reading easy or difficult articles) × 2 (with or without hands‐on manipulation) between‐subject design. Seventy‐nine undergraduate students participated. They first read a science article about rotational mechanics and then completed a problem‐solving task. While completing the problem‐solving task, the two pure reading groups could reread the article to write down their answers, and the two read‐and‐manipulation groups were allowed to manipulate the pulley system in addition to reading the article. The participants' eye movements were recorded during the completion of this task. Results showed that hands‐on manipulation benefited those learners who read the difficult article, since they had better scores regarding the problem‐solving task; this was not the case for those who read the easy article. Eye movement data showed that the two pure reading groups spent more processing time rereading the article as well as writing their answers to the problem‐solving task than did the other two read‐and‐manipulation groups. Furthermore, the easy‐article group spent significantly longer processing time on performing the experiment but not on preparing or planning for manipulating the pulley system than did the difficult‐article group. The transition proportion of eye fixations reflected the looking and acting order of the participants during the problem‐solving task and indicated groups' similarities (e.g., seldom did students refer to the article to manipulate the pulley system and in reverse) and differences (e.g., the difficult‐article group preferred to check back‐and‐forth if their manipulation resolved the testing questions, but the easy‐article group did so relatively seldom).

Keywords: article difficulty; eye‐movement tracking; hands‐on manipulation; science learning

Manipulation (i.e., experiments) and reading are the two main approaches to learning in science. Doing experiments has typically been emphasized in science education, and there is empirical evidence demonstrating the advantages of manipulation in science learning (Klahr et al., 2007; Lazonder & Ehrenhard, 2014; Olympiou & Zacharia, 2012). However, there is a growing body of evidence showing medium to high strength correlations between reading and academic achievement in science (August et al., 2009; Cromley et al., 2010). Recently, there has been an increasing tendency to combine reading and manipulation to help students learn science (Cervetti et al., 2012; Hattan & Lupo, 2020; Romance & Vitale, 1992). Before doing experiments, there are various scientific concepts that learners need to be able to grasp. Texts are very useful in conveying scientific knowledge and can be seen as a form of second‐hand investigation in conducting scientific inquiry, along with designing experiments and collecting data (Palincsar & Magnusson, 2001; Sullivan & Puntambekar, 2019). As Osborne (2002) has claimed, "science without literacy is like a ship without a sail."

Various studies have confirmed the importance and effectiveness of combining reading and practicing science (Cervetti et al., 2012; Hattan & Lupo, 2020; Romance & Vitale, 1992; Sullivan & Puntambekar, 2019). In an outstanding study, Romance and Vitale (1992) developed a year‐long implementation strategy by combining an in‐depth science teaching program (e.g., hands‐on activities, science process skills) and science‐content reading instruction (e.g., identify cause‐and‐effect relations in science texts) for fourth grade students to investigate the effect of science learning and attitude. The results indicated that students who received the integrative curriculum strategy had better reading comprehension, more positive science attitudes, and greater self‐confidence in science learning. Cervetti et al. (2012) also developed an instruction of integrating science and literacy for teachers to teach fourth grade students and investigate the instruction effect on science reading comprehension. The integrated science‐literacy instruction engaged students in reading science text, writing notes, conducting firsthand investigations, and discussing the key concepts and processes to acquire inquiry skills and knowledge about science concepts. The results showed that the fourth‐grade students who received this integrated instruction had better performance on science understanding, science vocabulary, and science reading comprehension than the control group. Recently, Sullivan and Puntambekar (2019) further investigated how teachers can prepare their students to conduct scientific investigations with scientific texts as part of the process of scientific inquiry in the classroom. The results showed that sixth‐grade students who received the instruction having access to multiple scientific texts had better physics test scores than the control group who received the scientific instruction without combining text reading. In addition, this study also indicated that having profound discussions related to the implied higher learning outcomes in physics concepts tests. Thus, reading texts is an important part of classroom science learning (Cervetti et al., 2012; Palincsar & Magnusson, 2001; Sullivan & Puntambekar, 2019; Varelas et al., 2014).

Despite these findings, we still do not understand what happens during the "learning process" involved in combining reading and hands‐on manipulations. This study aims to bridge this gap, considering a large body of educational research regards learning processes and outcomes as equally important (Jian, 2018, 2019; Rodrigues & Rosa, 2017; Solheim & Uppstad, 2011). In educational research, interviews and self‐report measures have been frequently used to assess cognitive activities during learning (Daley et al., 2014; Rodrigues & Rosa, 2017), but these approaches may suffer validity issues. The contents that participants self‐report or report in interviews after task completion may not totally reflect what they think about during the learning task. Eye‐movement tracking technology that records on‐line processes of learning (real‐time information processing) can overcome the limitation and has been extensively used to investigate the reading processes (Huang & Chen, 2016; Jian, 2019, 2021; Liao et al., 2020; Tsai et al., 2019; Yang, 2017; Yang et al., 2016). The on‐line reading process is usually measured by the eye‐movement indicators of "fixations" and "saccades." Fixation refers to the maintenance (average 200–25000A0ms in a fixation) of the eye gaze on a certain area of the learning material to encode information, and saccade refers to a quick movement (average 20–35 ms) between two continuous fixations (Rayner, 1998). By using eye tracking technology, we can know what happens (e.g., where and how long) during a learning task rather than know the learning products. Some interesting findings of illustrated science text reading were revealed. For example, whether adults, teenagers, or children are text‐driven readers, they spend much more time reading texts than on scientific diagrams in science text reading (Jian, 2018, Jian, 2021; Jian, 2019; Hegarty & Just, 1993; Mason et al., 2013; Yang, 2017); high‐ability readers with good reading comprehension spend a much longer time on scientific diagrams and made many more eye saccades between texts and diagrams in the science articles than the low‐ability students (Hannus & Hyönä, 1999; Jian & Ko, 2017); high‐knowledge students transferred their eye fixations across thematically relevant contents in scientific diagrams, but low‐knowledge students focused on superficial features such as color differences (Cook et al., 2008).

Using an eye tracker to record the learning processes of combining science text reading and manipulating experiments helped us reveal the mind operations of learners. For example, how do learners allocate their visual attention on text reading and hands‐on manipulation to complete a scientific problem‐solving task, and how does their learning sequence work during this task? Investigations in this direction seem to be absent in the literature. In sum, the current study used an eye‐movement tracker to investigate the learning processes involved in reading a science article combined with hands‐on manipulation of a pulley system as well as their influences on learning outcomes.

Science learning typically involves the activities of reading and manipulating experiments (Pearson et al., 2010). Theories based on combining reading and manipulations proposed in science education and cognitive psychology can be explained as follows.

In the research of science education, Hodson (1992) described the multidimensionality of scientific literacy with three major elements: (1) acquiring and developing conceptual and theoretical knowledge; (2) developing an understanding of science's methods; and (3) engaging in and developing expertise in scientific inquiry and problem solving. Hodson also claimed that using conceptual knowledge (e.g., scientific facts and principles) and procedural knowledge (e.g., measurement) in combined learning activities (e.g., reading and doing) is important for science learning. Suppose a student is learning a concept of pully mechanics by reading and doing experiments, he/she might experience learning both conceptual and procedural knowledge. In text reading, the student needs to learn some conceptual knowledge such as the different functions of fixed and movable pulleys, and some procedural knowledge such as the steps of hanging up the pulleys and weights on the ropes. In doing experiments, the students may also experiment with both types of knowledge, such as preparing (e.g., selecting ropes, pulleys, and weights), planning (e.g., gesticulating or thinking about where and how to hang the hook or rope in the pulley system), and performing the experiment of manipulating the pulley system for verifying the content written in the text or for completing a problem‐solving task (e.g., to calculate how many weights are needed in a pulley system).

In cognitive psychology research, a series of studies have noted that manipulation has a positive influence on text memory and comprehension (Glenberg et al., 2007) based on the following two theories. The first is dual‐coding theory (Paivio, 1986), in which manipulating a behavior or object described in the text introduces a visuomotor component in addition to the verbal code from the text. These dual codes are assumed to be separate but interdependent information‐processing systems. Therefore, the dual‐coded information is more durable and easier to retrieve than individually coded information. The second is the embodied theory of cognition (Shapiro, 2011). It emphasizes the role that the body and environment play in cognitive processing and the ways in which gestures, actions, and analogical mapping can be leveraged to improve learning. Weisberg and Newcombe (2017) indicate that embodied cognitive ways provide a unique opportunity for linking sensory representations (e.g., touching an object or visualization of data or information) with abstract principles to augment science education. For example, Varelas et al. (2014) executed instruction research that combined children's science books and related hands‐on explorations to confirm that incorporation of both informational texts and hands‐on explorations can maximize the richness of children's science learning experiences.

The literature review revealed controversial results for the effects of hands‐on manipulation on science learning, with some being positive (e.g., Klahr et al., 2007; Lazonder & Ehrenhard, 2014; Olympiou & Zacharia, 2012; Triona & Klahr, 2003; Zacharia et al., 2012) and some being negative (Stull & Mayer, 2007; Zhang, 2019; Zhang & Van Reet, 2021). A more balanced view was reported in the following statements.

The positive view of hands‐on manipulation demonstrated several advantages for science learning: (1) It involves touch, activating the brain to form abstract concepts through interactions between real objects (Klahr et al., 2007; Lazonder & Ehrenhard, 2014; Zacharia et al., 2012); (2) students experience the physical qualities of the experimental components (Klahr et al., 2007; Olympiou & Zacharia, 2012; Zacharia et al., 2012); and (3) students are able to conduct experiments with real objects to develop experimental skills (Bumbacher et al., 2018). Physicality brings some benefits to students in science learning. Lazonder and Ehrenhard (2014) investigated whether physical and virtual materials are equally effective in science inquiry learning. The elementary school students engaged in an inquiry task about falling objects (a common misconception for children [Howe et al., 2012]), and were assigned to one of the three instructional conditions: (1) physical manipulation, engaged in hands‐on experiments with concrete, tangible materials; (2) virtual manipulation, using a computer simulation; and (3) physical demonstration, watch a teacher to conduct the experiments with concrete materials. The results showed that students in the physical manipulation condition had a great improvement for revising their misconception of falling objects. Therefore, Lazonder and Ehrenhard claimed that physicality provides several modalities, including tactile and visual, to help students understand concepts, and students can integrate this information into new concepts. This integration could help students remember what they have learned and also positively affect students' confidence. For example, Triona and Klahr (2003) note that students should draw conclusions from experiments and then justify those conclusions. Their study found that students using hands‐on manipulation have more confidence when reporting their conclusions. The above findings reveal the importance of touch and how it influences students' science learning.

However, some studies showed that hands‐on manipulation did not strengthen science‐learning outcomes. Zhang and Van Reet (2021) tested the effects of two instructional elements, physically manipulating investigation materials (called "playing") and being presented with answers (called "telling") by a teacher, on learning the concept of light. The children aged between 5.5 and 7.5 years old, either in kindergarten or first grade, were randomly assigned to one of four conditions: either playing or telling, playing combined with telling, and the control group without playing or telling. The participants completed a 12‐question interview test that measured light concepts. The results showed that providing the young children with opportunities to physically interact with the investigation materials did not help them learn science concepts, but directly giving them answers had an impact on the science concept learning performance. The results also demonstrated better understanding for these students when they were provided with answers, regardless of whether or not they were allowed to play. Similar results were found for fourth and fifth grade students: participants in the hands‐on inquiry condition gained less knowledge of the science of energy compared with participants given direct instructions (Zang, 2019). In another study, Stull and Mayer (2007) asked undergraduate students to read a long‐page scientific text to construct their own graphic organizers (i.e., learning by doing) or the graphic organizers were provided (i.e., learning by viewing). The result demonstrated that the former group had worse learning performance than the latter group, and supported the cognitive load theory, which posits that excessive activity can create extraneous cognitive load, disrupting generative processing.

In sum, the advantages and disadvantages of hands‐on manipulation on science learning are controversial and need to be investigated further.

Several factors may influence the performance of science learning, such as conceptual difficulty of the learning materials, learners' prior knowledge, and spatial ability. First, this study explored if hands‐on manipulation benefits scientific concept learning mediated by science texts of varying difficulty levels. Scientific texts have unique properties that make reading comprehension challenging for readers such as lexical density (the number of content words within a text; Halliday & Martin, 1993), technical vocabulary and academic language (scientific words have specific meanings within a science discipline; Snow, 2010; Wellington & Osborne, 2001), and multimodality (graphs, tables, and mathematics that express relationships; Kress & Van Leeuwen, 2001). Science texts determine students' reading comprehension (Osborne et al., 2016), and texts are supported by performing experiments that help students to understand abstract concepts (van den Broek, 2010). The relationship between reading science texts and performing experiments may influence students' conceptual understanding. Patterson et al. (2018) and van den Broek (2010) indicated that if students misunderstand science texts, they demonstrate poor conceptual understanding. Previous research has shown that reading easy texts requires less cognitive capacity, leaving more capacity for other cognitive tasks (Roy‐Charland et al., 2016).

In addition, prior knowledge and spatial ability may influence science reading and science learning (Hodgkiss et al., 2018; Kozhevnikov & Thornton, 2006; van den Broek, 2010; Vosniadou & Skopeliti, 2018; Yang, 2017). As for prior knowledge, Patterson et al. (2018) indicated that reading comprehension of science texts influences learning of scientific concepts. In science reading comprehension, students need to organize various pieces of scientific information to form a new concept, and their prior knowledge can influence how they read texts and link it to science propositions (van den Broek, 2010). Vosniadou and Skopeliti (2018) indicate that poor prior knowledge leads to the forming of misconceptions, and Yang (2017) found that it is associated with weaker scientific reasoning. As for spatial ability, Kozhevnikov and Thornton (2006) show that spatial ability could help students imagine physics phenomena, and students with low spatial awareness need the assistance of visual materials to learn physics concepts. Hodgkiss et al. (2018) adopted longitudinal methods to understand 7th‐ to 11th‐grade students' science achievements, finding that spatial ability determines students' science achievements and mental folding, a kind of spatial ability, could explain students' understanding of physics. Therefore, the factors of learners' prior knowledge and spatial ability that may influence the performance of science learning were measured and treated as covariate variables in this study.

The current study used eye‐movement tracking technology and paper‐and‐pencil tests to investigate the influence of reading a science article (easy vs. difficult) and hands‐on manipulation (with vs. without) on learning processes and learning outcomes for the pulley system concept. The experiment used a 2 (reading easy or difficult article) × 2 (with or without hands‐on manipulation) between‐subject design. Before the experiment, the participants' spatial abilities and prior knowledge of the material were measured. The experimental procedure had two stages, and the participants wore an eye tracker (Tobbi Pro Glasses) throughout the experiment. The participants first read an easy or difficult science article describing the rotational mechanics and then completed a problem‐solving, paper‐and‐pencil task. During the 25‐min problem‐solving task, the two pure reading groups (one reading the article but not manipulating the pulley system) could reread the article, and the other two groups were asked to manipulate the pulley system (see Figure 1) as well as reread the articles when writing their answers.

The three study questions and hypotheses are as follows:

• Does article difficulty and hands‐on manipulation of the pulley system influence learning outcomes? As reflected by the problem‐solving task, the researcher speculated that reading the article accompanied by hands‐on manipulation would cause better learning performance than only reading the article (Hypothesis 1a) because the former offered the learners multiple modes of representations and engagement with the ideas contained in static (textual) and dynamic (hands‐on operation) modalities (Varelas et al., 2014). This is also predicted by embodied cognition theories (Shapiro, 2011) as stated in the literature review. Further, the researchers speculated that the benefit effect of hands‐on manipulation for learning the pulley system concept would be mediated by difficulty (Hypothesis 1b). For instance, learning a difficult concept by reading a difficult article should be affected by less familiarity with academic terms and higher lexical density (Halliday & Martin, 1993), hindering reading comprehension (Osborne et al., 2016 ; Patterson et al., 2018). Thus, hands‐on manipulation may be needed to make abstract concepts concretely accessible.
• Does article difficulty and hands‐on manipulation influence learners' mental efforts (reflected by processing time of eye fixations) upon rereading the article and writing down answers to the problem‐solving task? Previous research has found that integration of hands‐on activities and science‐content reading instruction benefits learners' science learning (Cervetti et al., 2012 ; Hattan & Lupo, 2020 ; Romance & Vitale, 1992), but has not determined the learning processes of this integration activity. The second research question of this study aimed to analyze this process using eye‐movement data and further investigate whether the learning process was mediated by varying article difficulty. The researcher speculated that reading the article, accompanied by hands‐on manipulation, would reduce processing time upon rereading the article because learners would spend time manipulating the pulley system within their allotted time (Hypothesis 2).
• Do the two groups using hands‐on manipulation with easy and difficult articles make different mental efforts (reflected by processing time of eye fixations) for preparing (e.g., selecting ropes, pulleys, and weights), planning (e.g., gesticulating or thinking where and how to hang the hook or rope in the pulley system), reading, and performing the experiment of manipulating the pulley system for the problem‐solving task? Each group should perform different eye movement patterns (Hypothesis 3).

The participants were 84 undergraduate students (Mage = 20.78 years, SD = 1.84) majoring in education, management, the arts, or social sciences. Students who majored in science or engineering had not been recruited for this study as they might have had high prior knowledge relevant to the learning material. All participants were native speakers of the language used in the reading material and had normal or corrected‐to‐normal vision. The experimental design was a 2 × 2 between‐subjects design, and all participants were randomly assigned to each of the four groups: read an easy (or a difficult) article with (or without) hands‐on manipulation. The data of five participants were excluded due to poor eye calibration and eye‐tracking computer or recording failures. Therefore, the study sample comprised 79 participants: 17 in the group that read an easy article with hands‐on manipulation, 18 in the group that read a difficult article with hands‐on manipulation, 21 in the group that read an easy article without hands‐on manipulation, and 23 in the group that read a difficult article without hands‐on manipulation.

The learning material included two science articles (one easy and one difficult) about rotational mechanics and a problem‐solving task. Two ways were adopted to confirm the manipulation of article difficulty in this study. First, to distinguish article difficulty, the easy article was selected from Conceptual Physics (Hewitt, 2017), with target readers being middle‐school students, and the difficult article was selected from College Physics (Giambattista et al., 2004), which targeted college‐level readers. Then, two middle‐school physics teachers and three undergraduate students (who did not participate in this study) were invited to judge article difficulty individually by completing a 5‐point‐scale questionnaire (ranging from 1, very easy, to 5, very difficult). In this study, all raters rated the article selected from Conceptual Physics as a 2 or 3 and the article selected from College Physics as 4 to 5. The content analysis of both the articles was displayed in Appendix and confirmed by a middle‐school teacher with a master's degree in physics and more than 10 years of experience in teaching physics. Both articles covered the topics of pulley system and conservation of energy, and both were reset as four pages with the same word size in the reading materials. They both had a similar number of words (easy article: 1883; difficult article: 1985) and diagrams (four each). Briefly, the easy article's content included the law of conservation of energy, the principle of leverage, the functions of a pulley (the difference between fixed and movable pulleys and a focus on the principles), and a simple principle for calculating force in a pulley system. The difficult article's content included the law of conservation of energy, the functions of a pulley system (the difference between fixed and movable pulleys and a focus on the calculation), the components of force associated with a pulley system, and the relationships between normal force and work. In addition, the teacher also checked if the concepts included in both the articles were commonly taught in middle schools, and the results revealed that some knowledge of rotational mechanics described in both articles were new for the participants. The learning material of the pulley system included a scaffold, four pulleys, a cotton thread, and several weights (1 g, 2 g, 5 g, 10 g, 20 g, and 100 g).

Prior to the experiment, the researchers collected individual variable data with a spatial ability test and a prior knowledge test for each student. The Purdue Visualization of Rotations Test (ROT) (Bodner & Guay, 1997), consisting of 20 items, assessed students' spatial ability. There were 20 points altogether. The ROT represented the value of Kuder–Richarson 20 as the reliability, and the value was from 0.78 to 0.80. The criterion‐related validity of the ROT was 0.61 (p < 0.001) with Shepard–Metzler tests and 0.25 (p < 0.01) with the Revised Minnesota Paper Form Board. The prior knowledge test included five multiple‐choice questions (see Appendix) measuring "forces." These questions were derived from middle‐school level textbooks, and a high‐school physics teacher was invited to confirm if the concepts measured in the prior knowledge test were taught in middle‐school science courses. Each correct answer was worth one point, and there were five points in total.

There were four questions in the problem‐solving task (please see Appendix). These questions were about the concepts of law of conservation of energy and pully systems which contained both in the easy and difficult articles. Each correct answer was worth one point, a partially correct answer received one‐half point (e.g., wrote the correct calculation of weight for Question 2 but without an explanation or with an incorrect explanation). Two independent raters who did not know the experimental design rated answers and rationalized any inconsistent scores. Cohen's Kappa coefficient was 0.875.

This study used temporal (e.g., total visit time) and spatial (e.g., transition proportions of eye‐fixations sequences) measures to reflect learning processes during the problem‐solving task.

Total visit time was used in the study to reflect the processing time and mental effort paid to a specific event (e.g., reading, manipulation). Total visit time can be divided into several areas of interest (AOIs) according to the research questions. The manual for Tobbi Studio (2016) states that "Total visit time is defined as the sum of visit durations of an active AOI (or AOI group). An individual visit is defined as the time interval between the first fixation on the active AOI and the end of the last fixation within the same active AOI where there have been no fixations outside the AOI."

The problem‐solving task was divided into five AOIs (see Figure 2): reading—read the science article, testing—write down answers, and three AOIs (preparing, planning, and performing) of manipulating the pulley system. The three detailed AOIs within the movable pulley system were defined as: (1) prepare (select materials, e.g., pick up the ropes, pulleys, and weights), (2) plan (e.g., gesticulate or think about where and how to hang up the hook or rope in the pulley system), and (3) perform (e.g., manipulate the pulley system).

The transition proportions were calculated for understanding the learning processes during the problem‐solving task. For example, transition from testing to reading and to performing may suggest that students encounter difficulties during testing, and they seek information from article reading to manipulate the pulley system. Another example, transition from planning to performing and to preparing may suggest that after pausing and considering how to hang the weights to the pulley system, students may decide to execute the pully experiment and to reconsider which tools they need to choose. One starting AOI had four transferring (end) AOIs, for example, if all participants in one group had 100 times of fixation transitions from testing AOI, and among them, 50 were to performing AOI, 30 to reading AOI, 20 to preparing AOI, and 10 to planning AOI, then the transition proportions were 0.5, 0.3, 0.2, 0.1, respectively, and 1 in total.

Given the large quantity of data for eye‐movement tracking behavior and to answer the research questions directly, only eye‐tracking movement data collected during the problem‐solving task are reported here.

Tobii Pro Glasses 2 recorded the learners' eye movements. The sampling rate was 100 Hz. A bridle was used to fix the eye tracker system to the participants' head. The visual field of view (frame obstruction) was more than 160° horizontally and 70° vertically according to the official report of Tobii Pro Glasses. There were some lens of different nearsightedness provided in the eye tracker system. If participants had corrected vision with glasses, their glasses were removed, and the corresponding‐degree leg was selected to act the part of the eye tracker glasses. Eye movements data of participants' two eyes were recorded and analyzed.

The study comprised an individual experiment with a two‐stage procedure and a researcher told the participants how to do in each procedure. The first stage was to collect the participants' demographic data, including a spatial ability test and a prior knowledge test. The researcher demonstrated and asked the participants to complete the ROT (Bodner & Guay, 1997) within 15 min and the prior knowledge test relevant to the learning material within 10 min. The second stage was to collect the participants' eye movement data in a university lab while they used the research materials (including reading the science article, manipulating the pulley system, and writing answers to the problem‐solving task). The participants wore the eye‐movement tracker while reading a science article within 10 min and completing the problem‐solving task within 25 min. The eyes' calibration was conducted before the formal experiment. The participants were instructed to wear the eye tracker and look around to get used to the equipment. During the problem‐solving task, the two pure reading (easy or difficult article) groups could reread the article and write down their answers, and the two read‐and‐manipulate groups could manipulate the pulley system and reread the article to write down their answers within the same given time. The groups allowed to manipulate the pully system were instructed to freely use all elements (pulleys, a cotton thread, and several weights) in the box, as illustrated in Figure 2. The entire experimental procedure lasted about 60 min. A pilot study was executed to make sure undergraduate students were capable of completing each task within the time limit.

To confirm that the participants groups had similar levels in the demographic survey, an ANOVA was run with the article difficulty and hands‐on manipulation as independent variables, and the scores of the spatial ability test and the prior knowledge test as dependent variables. The results showed that there were no main effects of article difficulty and hands‐on manipulation nor interaction of them on the scores of the spatial ability test and the prior knowledge test, ps > 0.05.

To answer the first research question that does article difficulty and hands‐on manipulation of the pulley system influence learning outcomes, a multivariate analysis of a covariance (MANCOVA) was run with the article difficulty and hands‐on manipulation as independent variables, the scores of the problem‐solving task as the dependent variable, and prior knowledge test score and spatial ability test score as covariates. A MANCOVA adopted in this study was to use mathematical principle to do statistical control for excluding the possible influence of the covariate variables on the dependent variable. Therefore, it was possible to determine how much variation of the scores of the problem‐solving task was attributed to article difficulty and hands‐on manipulation but not to the prior knowledge test score and spatial ability test score. The means and standard errors of the scores for the prior knowledge test, spatial ability test, and problem‐solving task are shown in Table 1.

1 TABLEThe means and SD of the tests scores of spatial ability, prior knowledge, and problem‐solving task

Regarding the total score of the problem‐solving task, the results showed that there was a main effect of manipulation, F(1, 73) = 5.54, p < 0.05, η2 = 0.07, and an interaction of article difficulty with manipulation on the total score of the problem‐solving task, F(1, 73) = 9.09, p < 0.01, η2 = 0.11. Conducting a simple main effect showed that manipulation benefited the group reading a difficult article, t (39) = 3.80, p < 0.001; the readers who read a difficult article with hands‐on manipulation had better problem‐solving test performance than the readers without manipulation. However, the manipulation did not benefit easy article learning, p > 0.05.

To answer the second research question that does article difficulty and hands‐on manipulation of the pulley system influence learning outcomes, MANCOVAs were run with the article difficulty and hands‐on manipulation as independent variables, the total visit time of reading the article during the problem‐solving task and of writing answers as two dependent variables, and prior knowledge test score and spatial ability test score as covariates. The results are presented in Figure 3.

Regarding the total visit time of reading the article during the problem‐solving task, the results showed that there was a main effect from hands‐on manipulation, F(1, 73) = 11.28, p < 0.01, η2 = 0.13; the pure reading (without manipulation) groups spent significantly longer total visit time on reading the article to write their answers to the problem‐solving task than did the reading‐and‐manipulation groups, with a marginal main effect of article difficulty, F(1, 73) = 3.48, p = 0.06, η2 = 0.05, but article difficulty had no main effect nor interaction of hands‐on manipulation with article difficulty, ps > 0.05.

Regarding the total visit time for writing answers during the problem‐solving task, the results showed that there was a main effect for hands‐on manipulation, F(1, 73) = 70.23, p < 0.001, η2 = 0.49. The pure reading groups spent significantly longer total visit time on writing their answers to the problem‐solving task than did the read‐and‐manipulation groups, but article difficulty had no main effect, nor did interaction of hands‐on manipulation with article difficulty, ps > 0.05.

To answer the third research question that does article difficulty and hands‐on manipulation of the pulley system influence learning outcomes, MANCOVAs were run with the article difficulty as an independent variable, the total visit time for selecting materials (prepare AOI), for manipulating the pulley system (perform AOI), and for other actions (e.g., picking up the weights and considering where and how to hang the hook or rope in the pulley system; plan AOI) as three dependent variables and prior knowledge test score and spatial ability test score as covariates. Figure 4 indicates that the easy‐article reading with manipulation group spent a significantly longer total visit time for manipulating the pulley system than the difficult‐article reading with manipulation group, F (1, 31) = 6.51, p < 0.05, ŋ2 = 0.17. However, the two groups had similar total visit times for preparing and planning where and how to hang up the hook or rope in the pulley system, ps > 0.05. These results are visualized in Figure 2, which depicts two heat maps of eye fixations converged with all participants in each group during the problem‐solving test. Opaque colors (e.g., testing AOI for the two groups) indicate the longest fixations convergence of all participants. There was more opaque yellow and green during performing for the easy‐article group than the difficult‐article group. Transparent green color indicates relatively less fixation convergence of the participants.

Further, to depict the different processes during the problem‐solving task for the two groups of reading easy versus difficult articles with hands‐on manipulation, tests of homogeneity were run with the article difficulty and groups of AOIs (pairwise of the starting area and the subsequent transfer area) as independent variables and the transition numbers of eye fixations as dependent variables. Figure 5 depicts the overall appearances of fixations sequences by reporting the transition proportions of eye fixations across AOIs during the problem‐solving task for the two groups of reading easy versus difficult articles with hands‐on manipulation.

For the preparing area as the starting AOI, the results showed that the relation of the two independent variables was significant, χ2(3) = 13.54, p < 0.01; posttests showed that the easy‐article group had a higher proportion of transitions of "preparing to planning" AOIs than the difficult‐article group, but the difficult‐article group had a higher proportion of transitions of "preparing to performing" AOIs than the easy‐article group, ps < 0.05. Neither group had significant differences on the proportion of transitions of "preparing to reading" and "preparing to testing" AOIs, ps > 0.05.

For the planning area as the starting AOI, the results showed that the relation of the two independent variables was significant, χ2(3) = 14.98, p < 0.01; posttests showed that the easy‐article group had a higher proportion of transitions of "planning to performing" AOIs than the difficult‐article group, but the difficult‐article group had a higher proportion of transitions of "planning to testing" AOIs than the easy‐article group, ps < 0.05. Neither group had significant differences on the proportion of transitions of "planning to preparing" and "planning to reading" AOIs, ps > 0.05.

For the performing area as the starting AOI, the results showed that the relation of the two independent variables was significant, χ2(3) = 90.66, p < 0.001; posttests showed that the easy‐article group had a higher proportion of transitions of "performing to planning" AOIs than the difficult‐article group, but the difficult‐article group had a higher proportion of transitions of "performing to preparing" and "performing to testing" AOIs than the easy‐article group, ps < 0.05. Neither group had significant differences on the proportion of transitions of "performing to reading" AOIs, p > 0.05.

For the reading area as the starting AOI, the results showed that the relation of the two independent variables was not significant, χ2(3) = 5.76, p > 0.05.

For the testing area as the starting AOI, the results showed that the relation of the two independent variables was significant, χ2(3) = 18.57, p < 0.001; posttests showed that the easy‐article group had a higher proportion of transitions of "testing to preparing" and "testing to planning" AOIs than the difficult‐article group, but the difficult‐article group had a higher proportion of transitions of "testing to performing" AOIs than the easy‐article group, ps < 0.05. Neither group had significant differences on the proportion of transitions of "testing to reading" AOIs, p > 0.05.

Reading science articles and hands‐on manipulation are two important methods of science education, and current tendencies combine the two approaches (Cervetti et al., 2012; Hattan & Lupo, 2020; Romance & Vitale, 1992). These excellent studies used tests to measure the learning products rather than learning processes. Since many educational researchers regard the processes and outcomes of learning both as important (Jian, 2018, 2019; Rodrigues & Rosa, 2017; Solheim & Uppstad, 2011; Wu et al., 2021), this study was the first to use eye‐movement tracking technology and paper‐and‐pencil tests to investigate the influence of article reading and hands‐on manipulation on learning processes and learning outcomes for the pulley system concept. The major findings are as follows.

First, this study found that hands‐on manipulation had different effects on learning easy and learning difficult science concepts. The benefit was greater for processing difficult articles than for processing easy articles. This was supported by higher test scores on the problem‐solving task for those students who read the difficult article accompanied by hands‐on manipulation than for those who read the same article without hands‐on manipulation. One possible explanation was because the difficult article contains more unfamiliar academic language and higher lexical density (Halliday & Martin, 1993; Snow, 2010; Wellington & Osborne, 2001) and requires hands‐on manipulation to concretize abstract concepts (Osborne et al., 2016; Patterson et al., 2018). Although reading difficult texts might occupy learners' cognitive capacity (Roy‐Charland et al., 2016), using hands‐on manipulation could help them free up some cognitive capacity. The tactile sense could assist the visual sense in understanding phenomena (Lazonder & Ehrenhard, 2014). However, this study indicated that hands‐on manipulation cannot strengthen science learning during easier scientific text reading. It corresponds to the findings of previous research (Stull & Mayer, 2007; Zhang, 2019; Zhang & Van Reet, 2021) that claimed that hands‐on manipulation had no effect on science learning.

Second, the read‐and‐manipulation groups spent significantly shorter time on reading the articles and less processing time for writing their answers in the problem‐solving task than the pure reading groups; the article difficulty did not moderate this result. One possible explanation is that the learners using the difficult article and hands‐on manipulation skipped to the hands‐on activity without reading the science article thoroughly enough for them to comprehend the content. This may be supported by the eye movement data (see Figure 5) showing the increased number (and proportion) of fixations back to the article. Based on the comparison with the difficult article/read‐only group, it seems that even if the participants in the difficult article/hands‐on group did not read the article thoroughly, the opportunity to engage in hands‐on manipulation made up for their less thorough reading. Another possible explanation for the result indicated that the read‐and‐manipulation groups completed the problem‐solving task more quickly than the pure reading groups was the former group's feeling that the task was easier based on the embodied theory of cognition (Shapiro, 2011) and the hypothesis of physicality (Klahr et al., 2007; Lazonder & Ehrenhard, 2014; Olympiou & Zacharia, 2012; Zacharia et al., 2012) as mentioned above. Further research could add a subjective questionnaire of difficulty rating to confirm this possible explanation.

Third, the participants who read the easy article accompanied by hands‐on manipulation spent a significantly longer time in manipulating the pulley system than did those with the difficult article with manipulation. One possible explanation may be students' prior science experiences result in this observational learning behavior. Another possible explanation is to correspond to the statement that people who read easy texts use less cognitive capacity, leaving more capacity for processing other tasks (Roy‐Charland et al., 2016). However, the two groups spent a similar amount of time on preparing and planning the pulley system. In addition, this study depicted the learning processes (e.g., looking and acting order) of the participants during the problem‐solving task. Overall, Figure 5 indicated some similarities and differences between the two groups of reading easy and difficult articles. The similarities were: (1) both groups dominantly used the manipulation of the pulley system to write their answers of the problem‐solving test rather than reading the science articles. This is supported by the evidence that the transition proportion from testing‐to‐performing AOIs for the easy‐article and difficult‐article groups were 41% and 47%, respectively but only 23% and 27% from testing‐to‐reading AOIs for the easy‐article and difficult‐article groups, respectively. (2) Very few students in either group read the entire science article and then manipulated the pulley system or vice versa. The transition proportion of eye fixations from reading‐to‐performing AOIs were 15% and 12% for the easy‐article and difficult‐article groups, respectively; the reverse transition proportions from performing‐to‐reading AOIs were 4% and 6% for the easy‐article and difficulty‐article groups, respectively. Furthermore, there were several interesting differences between the two groups: (1) After pausing and considering how to hang the weights to the pulley system, the easy‐article group tended to manipulate the pulley system directly. The transition proportion of planning‐to‐performing AOIs reached 48% for the easy group, and the relatively higher AOIs confirm this statement. However, besides manipulating the pulley system directly (planning‐to‐performing AOIs was 37%), a few students in the difficult‐article group needed to check back and consider the question statements of the test while holding the pulley weights in their hands before manipulating the pulley. This was supported by the evidence that the difficult‐article group (39%) had significantly higher transition proportions of planning‐to‐testing AOIs than did the easy group (27%). (2) Hands‐on manipulation seemed to be more helpful in learning pulley concepts and prompted the students in the difficult‐article group to write down their answers. This was supported by the evidence that 53% of the eye fixation transitions were in performing‐to‐testing AOIs for the difficult‐article group. However, for the easy‐article group, the learning behaviors of pausing and considering how to do the task (planning AOI, 33%) and of writing down answers (testing AOI, 38%) were similar.

In sum, this study had some contributions for research methods and science education implications. In a novel manner, this study used eye‐movement tracking technology to gain in‐depth insight into how students learn science by manipulation. Embodied cognition (Shapiro, 2011) emphasizes the role that the body and environment play in cognitive processing and the ways that gestures, actions, and analogical mapping can be leveraged to improve learning. The eye‐movement data of this study revealed mental aspects involved in embodied cognition. As for science education implications, this study suggests that teachers should understand how learners perform different strategies for learning different levels of science concepts, as this study provides empirical evidence of transition proportion analyses. Also, that teachers can combine hands‐on manipulation and article reading for help students learn science, especially for difficult and abstract concepts involved in the learning of science. In addition, the eye‐movement tracking data showed that both the manipulation groups spent considerable time trying to hang the pulleys and weights on the pulley system. Some participants had trouble fixing the fixed pulley and circling the rope in the movable pulleys. The transition proportions of eye fixations data also supported this speculation. Figure 5 shows that nearly one‐third (24% for easy‐article group, and 29% for difficult‐article group) of the participants' eye fixation transfers were from performing (manipulating the pulley system) AOI to preparing (e.g., selecting ropes, pulleys, and weights) AOI. This data suggests that even though some concepts of the pulley system were taught in the middle‐school student, they seemed to have limited procedural knowledge regarding how to manipulate the pully system. Since procedural and conceptual knowledge were important for science learning, it is worth noting that science teachers should check to make sure students have enough procedural knowledge before doing experiments.

It needs to be noted was that the manipulations in this study can be scaffolding to learning because they slowed down reading to allow learners to process information and visually represented/tested the information in correspondence with the articles. The whole process gave learners the opportunity to interact with the concepts conveyed in the articles, which was quite different from physical manipulations involved in conducting hands‐on investigations in most science instructions in normal classrooms, where not only is the reading article part often not available to learners but also the concepts that are directly explained in the article are withheld from learners.

This research was financially supported from the grant MOST 110‐2636‐H‐003‐003 and MOST 109‐2636‐H‐003‐003 under Young Scholar Fellowship Program by Ministry of Science and Technology in Taiwan, and the "Institute for Research Excellence in Learning Sciences" and "Higher Education Deep Cultivation Project" of National Taiwan Normal University (NTNU), sponsored by the Ministry of Education, Taiwan. I also thank Dr. Tsu‐Nan Lee for collecting the data.

APPENDIX

Content analysis of the easy and difficult physics articles in this study

Questions in the prior knowledge test

• If a rock is carried from the ground onto a table at a constant speed, which type of energy of the rock has increased?
• Chemical energy
• Gravitational potential energy
• Kinetic energy
• Electrical energy
• W1 is the work done to move an object of mass m by displacement S on a horizontal surface using force F; W2 is the work done to move an object of mass 2 m by displacement S up an inclined surface using the same force as before (F); W3 is the work done to move an object of mass 3 m upwards by displacement S, using a force that is directed upwards. How are W1, W2, and W3 related to each other?
• W1 = W2 = W3
• W1 > W2 > W3
• W1 < W2 < W3
• W1 = W2 > W3
• If the input energy of a machine is A joules, the magnitude of the external work done by the machine is B joules, and the thermal energy produced by the machine during this process is C joules, how are A, B, and C related to each other?
• A = B = C
• A + B = C
• A + C = B
• A = B + C
• If an object with a mass of 5 kg free falls from a height of 20 m, what is the height of the object from the ground when its potential energy is equal to its kinetic energy?
• A person is carrying a 1.5 kg heavy briefcase and has walked 10 m on a horizontal road with an acceleration of 2 m/s 2. If the briefcase is static relative to Xuanxuan and gravitational acceleration is g = 10 m/s 2 , the work done by gravity on the briefcase during this walk was __ joules.

Examples of the questions in the problem‐solving task (for the group of the easy article with manipulate)

Vignette: Consider a combination of pulleys with four hooks, A, B, C, and D. The distances between A and B, C and D, and B and C are 9 cm, 9 cm, and 12.5 cm, respectively. The pulley system consists of a fixed pulley and movable pulley (25 g), which are connected by a string. The string is tied to hook A on one end, and it first goes under the movable pulley before going over the fixed pulley. The fixed pulley is tied to hook B, and the string falls naturally after going over the fixed pulley. A heavy object is suspended by the movable pulley.

• In this scenario, if the mass of the pulley may be overlooked and a 100‐g mass is being suspended by the movable pulley g, how many grams of mass must be hung on the string to balance the pulley system? Please, explain your answer.
• However, the mass of the pulley must also be considered in practical terms. If the pulley has a mass of 25 g, how many grams of mass must be hung on the string to balance the pulley system? Please, explain your answer. (_B_Please, perform experiments using the materials on the table to answer questions 2, 3, and 4).
• Now, please hang a pair of fixed pulleys on Hooks A and C, and let a string go over these fixed pulleys, with its ends hanging off naturally from them. Next, hang a movable pulley between A and C, and hang a 100‐g mass on this movable pulley. How many grams of mass are required on both sides of the string to balance this pulley system? Choose an answer from the five following options, and explain your choice.
• 40–49 g
• 50–59 g
• 60–69 g
• 70–79 g
• 80–89 g
• Now, hang a pair of fixed pulleys on Hooks B and D, and fasten one end of a string to Hook A. Make this string go under a movable pulley, and then over the fixed pulleys on Hooks B and D before falling off naturally. Add another movable pulley between Hooks B and D so that there are two movable pulleys: one between Hooks A and B, and another between Hooks B and D. Next, hang a 50‐g weight on each pulley. How many grams of mass are required to balance this pulley system? Please, select a possible answer from the five options that were given in the previous question, and explain your choice.