Retro-Cue Effect: The Retro-Cue Is Effective When and Only When Working Memory Consolidation Is Inadequate

Acknowledgement: The work described in this paper was supported by the National Natural Science Foundation of China (32200847), the Research Grants Council of the Hong Kong Special Administrative Region (CUHK 14610520), and the Key Project of Philosophy and Social Science Research in Colleges and Universities in Jiangsu Province (2022SJZD100). Data and script for data analysis are available on the Open Science Framework project page (https://osf.io/8qgvx/).

Visual working memory (VWM) is the online system for maintaining and manipulating visual information in the short term (Awh et al., 2007; Brady et al., 2009; Cowan, 2010; Luck & Vogel, 1997; Pashler, 1998). In a typical working memory task, a set of stimuli is presented and then disappears. The observers try to memorize them and are then tested after a “retention interval” (RI).

A retro-cue effect (RCE) suggested that the representations maintained in VWM can be further improved in the RI. Specifically, previous studies found that participants’ VWM performance was enhanced when their attention was directed to the to-be-tested position during the RI (Griffin & Nobre, 2003; Landman et al., 2003; for a review, see Souza & Oberauer, 2016).

A typical retro-cue paradigm starts with a simultaneous presentation of multiple to-be-memorized items (e.g., six colors). The stimuli display is followed by a RI of about 1 s to exclude the influence of iconic memory. Then, a spatial retro-cue is presented on the screen for around 200 ms. After a post-cue interval that usually ranges from 300 to 1,000 ms, memory performance is tested on the cued (and sometimes un-cued) location. A robust retro-cue benefit of 5–15% in WM task performance was reported across a wide range of tasks and various types of stimuli (Backer et al., 2015; Katus et al., 2014; LaRocque et al., 2015; Lepsien et al., 2011; Oberauer, 2005; Q. Li & Saiki, 2015; Sligte et al., 2008).

Although the RCE is empirically well-established, the root of this effect is still a matter of debate. There are currently four major explanations. First, the strengthening hypothesis suggests that retro-cue can strengthen the memory item being cued (Rerko & Oberauer, 2013; Souza et al., 2015). This hypothesis is backed by the findings that focusing attention on the cued item does not impair memory for the un-cued items (Berryhill et al., 2012; Gunseli et al., 2015). Moreover, the RCE appears to be additive, as when the target item is cued twice, its recognition is superior to when it is cued only once (Rerko & Oberauer, 2013). Second, the removal hypothesis posits that retro-cue aids participants in recognizing irrelevant information and removing them from VWM (Williams et al., 2013). Removing irrelevant information in VWM decreases the interference between representations and frees the capacity to process the relevant information (Souza et al., 2014). Third, the protection hypothesis suggests that the retro-cued item is in the focus of attention and thus insulated from visual interference of irrelevant visual information or time-based decay (Makovski et al., 2008; Matsukura et al., 2007; Pertzov et al., 2013). Fourth, the head-start retrieval hypothesis first assumes a two-stage model of short-term recognition (Souza et al., 2016). The first stage involves gradually accumulating evidence in WM for the target item. The second stage involves the decision-making process. Earlier and longer time for accumulating evidence presumably leads to more accurate responses (Niklaus et al., 2019). Therefore, the head-start retrieval hypothesis generally regards the retro-cue as an advanced indicator for entering the first stage (Souza & Oberauer, 2016).

Although each of these hypotheses has its merit in explaining some aspects of the RCE findings, they have trouble explaining two null RCE results reported recently: Shepherdson et al. (2018) sequentially presented six words as stimuli for 500 ms each, followed by a retro-cue indicating the position of the test probe. However, no RCE was observed in this experiment. Similarly, Niklaus et al. (2019) also found a null RCE in response accuracy when five visual or verbal stimuli were sequentially displayed, for 450 ms each, followed by a 50-ms interstimulus interval (ISI). There are two major differences between these null RCE studies and the typical ones: First, typical retro-cue paradigms generally adopt a simultaneous display of stimuli, whereas these two null RCE studies both used a sequential display of stimuli. Second, the average encoding time for each stimulus in a typical retro-cue paradigm is around 100–200 ms, whereas the encoding time for each stimulus is 450–500 ms for the null RCE studies. Why a longer encoding time or a sequential display of stimuli could diminish the RCE? The previously mentioned hypotheses all focused on the maintenance or retrieval process of the WM system and thus did not provide any prediction for these observations.

Here, we propose a consolidation hypothesis as an alternative explanation of the null RCE findings and the underlying mechanism for the RCE.

Upon viewing a visual stimulus, a transient and fragile sensory memory is first created (Cowan, 1984; Pinto et al., 2017). To stabilize the sensory memory traces for maintenance and carrying out further cognitive processes, a short-term consolidation process is needed to transform the traces into a more durable working memory state (Jolicœur & Dell’Acqua, 1998). Attention is required during short-term consolidation. While memory items are being consolidated, other attentional-demanding tasks are either disrupted or delayed (Nieuwenstein & Wyble, 2014; Stevanovski & Jolicœur, 2007). Studies have also shown that the short-term consolidation process can be completed without the presence of visual stimuli (Jolicœur & Dell’Acqua, 1998; Ricker, 2015). The estimates of the time course for short-term consolidation ranged from 400 to 1,600 ms depending on the type of stimuli examined (Bayliss et al., 2015; Jolicœur & Dell’Acqua, 1998; Ricker & Cowan, 2014).

Masking is generally considered an important factor affecting the short-term consolidation process. While some researchers regard a visual mask as the termination of the short-term consolidation process (Vogel et al., 2006), others suggest that the short-term consolidation process could proceed beyond the mask (Nieuwenstein & Wyble, 2014). In a recent study, Ricker and Sandry (2018) unveiled two aspects of the masking effect to reconcile the debate. On the one hand, a visual mask produces interference based on the temporal proximity to the visual stimuli. This effect explains why when holding the total stimulus onset asynchrony (SOA) equal, shorter visual mask onset time leads to worse WM task performance (Vogel et al., 2006). On the other hand, visual mask slows the rate of short-term consolidation. That is, when a visual mask is presented immediately following the visual stimuli, longer retention time after the mask presence results in better memory outcomes (Nieuwenstein & Wyble, 2014; Ricker & Cowan, 2014; Ricker & Hardman, 2017). In another study, using a sequential whole-report paradigm, Ricker and Hardman (2017) demonstrated that the short-term consolidation process for visual stimuli completes at around 600–700 ms, and fully consolidated stimuli were found to be protected from time-based decay or interference. Longer consolidation time (CT) leads to better memory performance, and this beneficial effect was found to be primarily driven by an increase in the probability that an item is in memory at the test (Ricker & Hardman, 2017). When the time allowed for consolidation exceeds 700 ms, VWM performance no longer benefits from the increasing CT. While the CT is shorter than 600 ms, the memory items are in a highly volatile state and prone to time-based decay or interference.

The short-term consolidation research brings forward a tentative explanation for the previous null RCE findings: By directing one's attention to the target memory representation, the retro-cue induces a continuation of the short-term consolidation process, but the relatively long CT in their design (500 ms presentation time + post-cue interval) leads to well-consolidated memory representations, which nullify the beneficial effect of retro-cue. Besides accounting for the doubts from the null RCE studies, the consolidation hypothesis also offers novel explanations for two well-established findings in retro-cue research: First, why a certain length of the cue-test interval (CTI) is needed for retro-cue to exert its effect; second, why probabilistic retro-cue generally leads to smaller RCE.

Previous retro-cue studies have constantly found that a CTI of at least 300 ms was needed to obtain a significant RCE (Pertzov et al., 2013, 2017; Souza & Oberauer, 2016). For example, Tanoue and Berryhill (2012) manipulated the CTI from 100 to 700 ms in a retro-cue paradigm and observed a gradual increase in the magnitude of RCE from 100 to 400 ms. However, a significant RCE emerged only at CTIs greater than 300 ms (Tanoue & Berryhill, 2012). Similarly, other researchers also found the magnitude of RCE to be stabilized when the CTI reached 400 ms (Gressmann & Janczyk, 2016; Souza et al., 2014). Although it is well accepted that a CTI of at least 300 ms is needed for the retro-cue to take effect, the ongoing cognitive process during this period is still unknown. Therefore, the CTI is usually referred to as the time to “make use of” the retro-cue (Souza & Oberauer, 2016). Here, the consolidation hypothesis provides a straightforward explanation for these observations: The CTI acts as a continuation of the short-term consolidation process. Once the consolidation process completes, memory performance no longer benefits from the subsequent periods.

Studies have shown that compared to deterministic retro-cue (always pointing to the position of the test probe), probabilistic retro-cue (70% validity) would lead to smaller RCE (Dube et al., 2019; Gunseli et al., 2015; Matsukura et al., 2007; Pertzov et al., 2013). Furthermore, on the invalid cue trials, observers’ memory performance was significantly reduced compared to the valid cue trials (Dube et al., 2019; Pertzov et al., 2013). This invalid cue cost is usually attributed to the disengagement of attentional resources from the target memory representation (Dube et al., 2019; Gunseli et al., 2015). The misguided attention subsequently leads to the failure of protection or false removal of the target memory representation. However, the consolidation hypothesis would provide a quite disparate explanation. When the CT is short, directing observers’ attention to the wrong position should result in worse memory performance because observers cannot use the CTI to continue the WM consolidation process of the target item. However, fully consolidated memory items should be protected from the invalid retro-cue because no additional CT is needed.

The general design of the present set of experiments followed the paradigm of Ricker and Hardman (2017), except that only one memory item was probed. Therefore, the current design differs from a typical retro-cue paradigm in three ways: First, four memory items are presented sequentially for the precise manipulation of each item's CT; second, a visual mask is presented after each stimulus to eliminate sensory memory traces and minimize its potential effect on the memory performance, especially for the last items in sequence; third, in the no-cue condition, the test probe is presented without delay. The rationale for this change is to equate the time when memory representations are prone to interference or decay. Since attention is directed to the target item in the presence of the cue, participants could either continue the WM consolidation process or retain only the target item during the CTI. However, in the no-cue condition, participants must retain all the memory representations during the RI. In other words, an additional RI would impair participants’ memory performance in the no-cue condition and potentially inflate the observed RCE.

In the following sections of this article, we investigated four research questions: first, whether longer CT erases the RCE; second, whether the interaction between CT and the RCE is limited to sequentially presented display; third, whether the CTI serves as a continuation of the WM consolidation process; and fourth, whether fully consolidated memory representations are protected from invalid retro-cue.

According to the consolidation hypothesis, the RCE could be detected only if the memory items were not fully consolidated. Therefore, in the first two experiments (Experiments 1A and 1B), we varied the total CT for each memory item to either 200 or 700 ms in both retro-cue and no-cue conditions. It is hypothesized that the RCE can only be observed in the 200 ms condition. Experiment 2 examined whether this interaction between RCE and CT could be generalized to the typical retro-cue paradigm with a simultaneously displayed array. It is hypothesized that the interaction could also be observed when stimuli are simultaneously displayed. Previous studies have consistently found that a post-cue interval of 300–400 ms was required to “make use of” the retro-cue to improve response accuracy (Pertzov et al., 2013; Souza et al., 2014; Tanoue & Berryhill, 2012). However, the underlying mechanism for this post-cue interval is still unclear. In Experiment 3, we sought to investigate whether the CTI served as the continuation of the short-term consolidation process. Accordingly, we varied both the CT and CTI to assess whether the time needed to “make use of” the retro-cue proposed in previous studies is associated with the CT for memory items. It is hypothesized that memory items with shorter CT before the retro-cue presence would benefit more from lengthened CTI. In Experiment 4, we sought to test a novel prediction generated by the consolidation hypothesis that fully consolidated memory items are protected from the invalid retro-cue. In this last experiment, we manipulated cue validity (70% or 100%) and CT (200 or 700 ms) to assess whether fully consolidated items were protected from the invalid retro-cue. It is hypothesized that RCE in the long-CT condition would be less affected by invalid retro-cue. In five experiments, we sought to confirm a consolidation hypothesis of the RCE and hopefully provide a unified account for previous retro-cue research.

Experiment 1A was designed to test whether longer CT led to diminished RCE. As shown in Figure 1, we sequentially presented four colored squares. In the cue condition, after the display of four stimuli, a retro-cue was presented on the center of the screen, validly indicating the position of the test probe. In the no-cue condition, no cue was presented, and the test probe was displayed immediately after the disappearance of the last stimulus. The total CT was set at 200 ms in the short-CT condition and 700 ms in the long-CT condition. As Ricker and Hardman (2017) had shown that the WM consolidation process was completed at 600–700 ms, a short CT of 200 ms in the present experiment cannot fully consolidate the memory traces. Therefore, we expect to observe the standard RCE in the short-CT condition (Griffin & Nobre, 2003; Makovski & Jiang, 2007). While in the long-CT condition, the consolidation hypothesis predicts that little RCE could be observed since fully consolidated memory items no longer benefit from the extended consolidation process.

Participants

The sample of this 1-hr experiment included 30 student volunteers (16 females; age range from 18 to 22; Mage = 20.23 years) from Nanjing Normal University who participated for a monetary reward (40 CNY). All participants were naïve to the experimental paradigm and reported normal color vision and normal or corrected-to-normal eyesight.

Apparatus and Stimuli

Visual stimuli for this experiment were displayed on a 1,024 × 768 pixels CRT color monitor with a gray background and a refresh rate of 60 Hz. All the experiments reported here were programmed in MATLAB 2016a with the Psychtoolbox-3 extension (Borgo et al., 2012; Kleiner et al., 2007). Participants were seated in a comfortable position and at a distance of about 60 cm from the computer screen. Responses were collected using a computer keyboard.

The memory set consisted of a series of four items, each presented individually on one of the four corners (i.e., left-top, left-bottom, right-top, and right-bottom), which were 4 cm away (about 3.82° of visual angle) from the center of the screen. Each memory item was a colored square with the color randomly selected without replacement from a set of nine colors (red, purple, yellow, blue, green, cyan, magenta, orange, and brown). The visual mask consisted of a 5 × 5 multicolored checkboard pattern composed of 25 randomly generated colors. The size of colored squares and masks was 2.6 cm × 2.6 cm (about 2.48° × 2.48° of visual angle).

Procedure

Participants first received a consent form and a detailed description of this study at the beginning of the experiment. After the briefing session, the participants started to perform the task. The sequence of presentations is shown in Figure 1. At the start of each trial, a white fixation cross was presented at the center of the screen for 500 ms. Next, a colored square was presented on one of the four possible positions for 100 ms, immediately followed by a visual mask, which was presented for 100 ms. In the long-CT condition, the visual mask was followed by a 500 ms ISI, whereas in the short-CT condition, the ISI was set to 0. This memory item–visual mask–ISI sequence was repeated four times in total with different memory items and at different locations. The same location was never used for multiple memory items in the same trial. In the retro-cue condition, after the presentation of all four stimuli, a white arrow extending 2 cm (about 1.91° of visual angle) from the center of the screen (retro-cue) was presented for 200 ms. The retro-cue always validly pointed to the location to be probed later. One second following the disappearance of the retro-cue, the test display was presented and remained on the screen until a response was made. At the test phase, a new colored square probe was presented on the cued location. The probe stimulus remained on the screen until participants responded whether the probe color matched the color of the square presented previously at the same position. For half of the trials, the test probe matched the color of the previously presented square (match trials). In the other half of the trials, the probe was either a color from another location in the memory display or a new color from the color set (mismatch trials). In the no-cue condition, the test probe was directly presented on the screen after the disappearance of the last visual mask. Participants were instructed to respond as accurately and as fast as possible by pressing the “J” button to indicate a match and the “F” button to indicate a mismatch. After responding, participants would hear either a cheering sound for a correct choice or a negative buzzer sound indicating an incorrect response. The next trial began 1,000 ms later. Each participant completed 16 practice trials followed by 16 blocks of 32 experimental trials, including four retro-cue long-CT blocks, four retro-cue short-CT blocks, four no-cue long-CT blocks, and four no-cue short-CT blocks. The order of the four types of blocks was counterbalanced across participants.

To avoid participants using verbal coding to aid their performance in the present task, they were required to repeat the phrase Coca-Cola at a speed of two times per second during the stimuli presentation and the RI (Wheeler & Treisman, 2002). They could stop repeating the phrase once entering the test phase.

Data and script for data analysis are available on the Open Science Framework project page (https://osf.io/8qgvx/).

The response accuracy and reaction time (RT) data were submitted to two separate 2 (CT) × 2 (cue conditions) × 4 (serial positions) three-way repeated-measures analysis of variance (ANOVA).

Analysis of Response Accuracy

Figure 2 depicts the mean accuracy for responses in each condition across the four serial positions (SPs). The ANOVA analysis yielded significant main effects of CT (F1,29 = 21.00, p < .001, partial η2 = 0.42), cue condition (F1,29 = 19.54, p < .001, partial η2 = 0.40), and SP (F3,87 = 64.18, p < .001, partial η2 = 0.69). A significant CT × SP interaction was detected (F3,87 = 5.79, p = .001, partial η2 = 0.17). Bonferroni post hoc test revealed that this interaction was mainly driven by the same response accuracy for short and long-CT conditions at SP1 (0.76 vs. 0.76). Importantly, the analysis also detected a significant CT × Cue interaction (F1,29 = 14.31, p = .001, partial η2 = 0.3), indicating that the RCE was modulated by CT. Moreover, Bonferroni post hoc tests revealed significant accuracy differences between the retro-cue and no-cue short-CT conditions across four SPs (SP1: 0.78 vs. 0.73, p = .02; SP2: 0.68 vs. 0.63, p = .04; SP3: 0.76 vs. 0.67, p < .001; SP4: 0.88 vs. 0.79, p < .001), but insignificant accuracy differences between the retro-cue and no-cue long-CT conditions (SP1: 0.77 vs. 0.75, p = .46; SP2: 0.74 vs. 0.71, p = .10; SP3: 0.80 vs. 0.81, p = .44; SP4: 0.90 vs. 0.91, p = .63); for a clearer illustration, the magnitude of RCE across 4 SPs is depicted in Figure 3. Interestingly, a significant CT × Cue × SP three-way interaction was also detected (F3,87 = 3.06, p = .03, partial η2 = 0.10). Bonferroni post hoc test showed that this interaction was mostly driven by a marginally significant RCE in long-CT condition at SP2 (0.74 vs. 0.71, p = .10).

Analysis of Response Time

The analysis of RT was limited to correct response trials only. Before analyzing the data of RT, the outliers of each participant were first removed by first excluding response time values shorter than 100 ms or longer than 10,000 ms, followed by excluding all the values that deviated from the mean for more than three standard deviations. Figure 4 depicts the mean response time in each condition across the four SPs. The ANOVA analysis yielded significant main effects of cue condition (F1,29 = 51.76, p < .001, partial η2 = 0.64), and SP (F3,87 = 23.63, p < .001, partial η2 = 0.45), but not of CT (F1,29 = 0.47, p = .50, partial η2 = 0.02). A significant CT × SP interaction was detected (F3,87 = 8.24, p < .001, partial η2 = 0.22). Bonferroni post hoc test revealed that this interaction was mainly driven by a significant difference between short and long-CT conditions at SP4 (897.21 ms vs. 812.98 ms, p = .014). However, no CT × Cue interaction (F1,29 = 3.32, p = .08, partial η2 = 0.10) was observed for the RT data. Planned comparisons showed that the RCEs were significant in both long-CT, t(29) = 5.94, p < .001, and short-CT, t(29) = 6.24, p < .001, condition, thus indicating a comparable RCE for long and short-CT conditions. Moreover, a significant CT × Cue × SP three-way interaction was detected (F3,87 = 5.15, p = .003, partial η2 = 0.15). Bonferroni post hoc test showed that this interaction was mostly driven by significant differences between no-cue short and no-cue long CT conditions at SP3 (1,105.12 ms vs. 995.22 ms, p = .048) and SP4 (899.10 ms vs. 1,069.47 ms, p = .002).

In Experiment 1A, we varied the CT and retro-cue presence across conditions. We observed a stable RCE regarding response accuracy in the short-CT condition across four SPs. This replicated past results measuring the RCE. Critically, we observed no RCE in the long-CT condition. This agrees with the consolidation hypothesis of the RCE, which predicts that longer CT would nullify the beneficial effect of retro-cue in terms of response accuracy.

Interestingly, the RT data showed a different pattern from response accuracy data: a significant and comparable RCE was found in both short and long-CT conditions. These results align with previous findings of a dissociation between response accuracy and RT in terms of RCE (Souza et al., 2014). The RT results suggest that participants actively used the retro-cue in both short and long-CT conditions to direct their attention to the to-be-tested memory representation, resulting in faster response speed. However, regarding response accuracy, only the memory representations under the short-CT condition benefited from this focused attention.

One question remains unclear as to why a three-way interaction of response accuracy result was detected. Therefore, we replicated this paradigm in Experiment 1B to assess the stability of this interaction.

Experiment 1B was a direct replication of Experiment 1A to assess the stability of the CT × Cue × SP three-way interaction. More specifically, we sought to investigate whether a stable RCE can be observed at SP2 in the long-CT condition.

Participants

The sample of this 1-hr experiment included 30 student volunteers (12 females; age range from 17 to 22; Mage = 18.93 years) from Nanjing Normal University who participated for a monetary reward (40 CNY). All participants were naïve to the experimental paradigm and reported normal color vision and normal or corrected-to-normal eyesight.

Apparatus and Stimuli

All the apparatus and stimuli were the same as in Experiment 1A.

Procedure

The procedure was the same as in Experiment 1A.

The response accuracy is the primary dependent measure discussed. The RT data are not discussed because they are consistent with Experiment 1A.

Figure 5 depicts the mean accuracy for responses in each condition across the four SPs. The accuracy data were analyzed by a 2 (CT) × 2 (cue conditions) × 4 (SPs) three-way repeated-measures ANOVA. The analysis again yielded significant main effects of CT (F1,29 = 22.22, p < .001, partial η2 = 0.43), cue condition (F1,29 = 25.22, p < .001, partial η2 = 0.47), and SP (F3,87 = 115.96, p < .001, partial η2 = 0.80). A significant CT × SP interaction was also detected (F3,87 = 8.73, p < .001, partial η2 = 0.23). Bonferroni post hoc test revealed that this interaction was mainly driven by the same response accuracy for short and long CT conditions at SP1 (0.72 vs. 0.71). As in Experiment 1A, the analysis detected a significant CT × Cue interaction (F1,29 = 19.20, p < .001, partial η2= 0.40), indicating that the RCE was modulated by CT. Moreover, Bonferroni post hoc tests revealed significant accuracy differences between the cue short CT and no-cue short CT conditions across four SPs (SP1: 0.76 vs. 0.68, p = .02; SP2: 0.67 vs. 0.61, p = .01; SP3: 0.77 vs. 0.69, p = .001; SP4: 0.89 vs. 0.83, p = .001), but insignificant accuracy differences between the cue long CT and no-cue long CT conditions (SP1: 0.71 vs. 0.71, p = .88; SP2: 0.70 vs. 0.68, p = .43; SP3: 0.82 vs. 0.83, p = .61; SP4: 0.92 vs. 0.91, p = .34); for a clearer illustration, the magnitude of RCE across 4 SPs is depicted in Figure 6. However, no CT × Cue × SP three-way interaction was detected in Experiment 2B (F3,87 = 0.57, p = .64, partial η2 = 0.02), suggesting that the significant 3-way interaction observed in Experiment 1A could be due to a fluctuation of data at SP2 in long CT condition. We combined the data from Experiments 1A and 1B to further confirm this notion. All other main and interaction effects were unchanged. Again, we observed an insignificant CT × Cue × SP three-way interaction (F3,177 = 2.21, p = .09, partial η2 = 0.04), suggesting that the result was not caused by the lack of statistical power in the design. We also submitted the combined data to Bayesian ANOVA (BANOVA) using the BayesFactor 0.9.12 package (Morey & Rouder, 2022) in R (R Core Team, 2022). The comparison of the models with and without the CT × Cue × SP three-way interaction yielded a Bayes factor of 0.12, indicating that the model without the three-way interaction is preferred.

In Experiment 1B, we replicated the key finding in Experiment 1A, which was a significant CT × Cue interaction, suggesting that longer CT nullified the standard RCE observed in the short-CT condition. Furthermore, we confirmed that the CT × Cue × SP three-way interaction observed in Experiment 1A was not a stable effect but possibly due to a fluctuation of data at SP2 in its long-CT condition.

To conclude, in two experiments, we demonstrated a stable interaction between CT and RCE, suggesting that sufficient CT completely vanishes the standard RCE observed in previous studies. These results provide clear evidence that the magnitude of RCE is modulated by the WM consolidation process, thus supporting the consolidation hypothesis.

The conclusion from Experiments 1A and B is clear: Sufficient CT erased the RCE in a sequential display paradigm. However, whether this interaction between CT and RCE effect could be generalized to other retro-cue paradigms is unclear. In Experiment 2, we tested whether different presentation method affect the interaction between CT and RCE. As shown in Figure 7, in this experiment, the simultaneous presentation condition resembles the typical retro-cue paradigm (Griffin & Nobre, 2003; Souza & Oberauer, 2016). We presented four colored squares simultaneously on screen for 400 ms, followed by a 100-ms visual mask. The RI was either 300 ms in the short-CT or 2,300 ms in the long-CT condition. In the simultaneous-repeat condition, we presented four colored squares simultaneously on the screen twice, each for 200 ms and followed by a 100-ms visual mask and 100 ms or 1,100 ms RI in the short-CT or long-CT conditions. In the sequential condition, all the time parameters were the same as in the simultaneous-repeat condition, except that we only presented two colored squares at a time. In this way, the average total CT for each item is held constant across simultaneous, simultaneous-repeat, and sequential conditions (200 ms/item in the short-CT and 700 ms/item in the long-CT conditions).

Participants

The sample of this 1.5-hr experiment included 30 student volunteers (21 females; age range from 22 to 26; Mage = 23.80 years) from Nanjing Normal University who participated for a monetary reward (60 CNY). All participants were naïve to the experimental paradigm and reported normal color vision and normal or corrected-to-normal eyesight.

Apparatus and Stimuli

All the apparatus and stimuli were the same as in Experiment 1A.

Procedure

The sequence of presentations is shown graphically in Figure 7. At the start of each trial, a white fixation cross was presented at the center of the screen for 500 ms. In the simultaneous condition, all four colored squares were presented together on the four possible positions for 400 ms, immediately followed by a 100-ms visual mask. After the visual mask offset, the RI began (300 ms in the short-CT or 2,300 ms in the long-CT condition). In the simultaneous-repeat condition, all four colored squares were presented together for 200 ms, followed by a 100-ms visual mask and then a 100-ms (short-CT) or 1,100-ms (long-CT) blank RI. This memory item–visual mask–RI sequence was repeated twice. In the sequential condition, two colored squares were first presented on one side of the screen for 200 ms, followed by a 100-ms visual mask and a 100-ms (short-CT) or 1,100-ms (long-CT) blank RI. Then, the other two colored squares were presented on the other side of the screen, again followed by a 100-ms visual mask and a 100-ms (short-CT) or 1,100-ms (long-CT) blank RI.

After the RI, in the retro-cue condition, a white arrow cue was presented for 200 ms, always validly pointing to the location to be probed later. One second following the disappearance of the retro-cue, the test display was presented and remained on the screen until a response was made. In the no-cue condition, the test probe was directly presented on screen following the last RI. The test phase in Experiment 2 remained the same as in Experiment 1A. Articulatory suppression was also implemented as in Experiment 1A. The next trial began 1,000 ms later.

The three presentation conditions (simultaneous, simultaneous-repeat, and sequential) were run in blocks of 32 trials. Each presentation condition included eight blocks: two retro-cue long-CT blocks, two retro-cue short-CT blocks, two no-cue long-CT blocks, and two no-cue short-CT blocks, resulting in 24 blocks in total. The order of the 12 types of blocks was counterbalanced across participants. Participants completed 16 practice trials at the beginning of each presentation condition.

Figure 8 depicts the mean accuracy for responses in each condition. The response accuracy data were submitted to a 2 (CT) × 2 (cue conditions) × 3 (presentation method) three-way repeated-measures ANOVA. The ANOVA analysis yielded significant main effects of cue condition (F1,29 = 140.86, p < .001, partial η2 = 0.83) and presentation method (F2,58 = 12.35, p < .001, partial η2 = 0.30). A significant CT × Cue interaction was detected (F1,29 = 24.77, p < .001, partial η2 = 0.46), indicating that the RCE was modulated by CT. Bonferroni post hoc tests revealed significantly larger RCE in the short-CT condition than in the long-CT condition (0.098 vs. 0.052, p < .001). In addition, planned comparisons showed that the CT × Cue interaction was significant across the Simultaneous (F1,29 = 7.35, p = .011, partial η2 = 0.20), Simultaneous-Repeat (F1,29 = 6.64, p = .015, partial η2 = 0.19), and Sequential Conditions (F1,29 = 6.53, p = .016, partial η2 = 0.18). Importantly, the analysis also detected a significant Presentation Method × Cue interaction (F2,58 = 5.97, p = .004, partial η2 = 0.17), indicating that the magnitude of RCE was affected by different presentation method. Bonferroni post hoc tests showed that the retro-cue led to significantly better performance across the three presentation conditions (Simultaneous: 0.86 vs. 0.77, p < .001; Simultaneous-Repeat: 0.89 vs. 0.80, p < .001; Sequential: 0.83 vs. 0.78, p < .001) and the significant Presentation Method × Cue interaction was attributed to the smaller RCE observed in the Sequential condition. For a clearer illustration, the magnitude of RCE across three presentation conditions is depicted in Figure 9. No CT × Cue × Presentation method 3-way interaction was detected (F2,58 = 0.005, p = .99, partial η2 < 0.001).

In Experiment 2, we varied the presentation method, CT, and retro-cue presence across conditions and found that the magnitude of RCE was modulated by CT across different presentation method. This result demonstrates that the interaction between CT and RCE is not restricted to the one-by-one sequential display adopted in Experiment 1 but also occurs in a standard RCE paradigm.

One distinction between the results of Experiment 2 and Experiment 1 is that longer CT did not completely erase the RCE in Experiment 2. The similar RCE results between the simultaneous and simultaneous-repeat conditions suggest that simply repeating the stimuli display has little effect on the magnitude of RCE. However, when the number of stimuli in each display is reduced to two in the sequential condition, the magnitude of RCE dramatically drops to Experiment 1's level. We propose that this could reflect the parallel consolidation of color features into WM (Hao et al., 2018; Mance et al., 2012; Miller et al., 2014). While one or two stimuli presented in a display could be efficiently consolidated into WM, adding more items into the display could introduce extra interference and disrupt the WM consolidation process. However, it is also worth noting that the RCE remains significant in the current sequential presentation condition when CT is long. This could indicate that the WM consolidation process of color feature does not work in a strict parallel manner or that the two stimuli interfered with each other during the process. Further research would be needed to distinguish between these two possibilities.

If observers can consolidate two items in parallel, would this finding affect the conclusion we reached in Experiment 1? Here we argue that whether or not two items can be consolidated in parallel will not affect the conclusion reached in the present study. The CT was set to 200 ms/item in the short-CT conditions of current sets of experiments, which means that even if participants effectively consolidated two items at once in the one-by-one sequential display paradigm, the CT (400 ms) was still insufficient for them to fully consolidate the items. In the long-CT conditions, the CT was set to 700 ms/item, which means that even if participants strictly consolidated one item at a time, the CT was enough to fully consolidate the items.

One small but important effect should also be noted. In the no-cue condition, participants were found to perform better when CT is longer across Experiment 1 (0.79 vs. 0.70, p < .001), Sequential condition (0.80 vs. 0.76, p = .017), and Simultaneous-Repeat condition (0.82 vs. 0.78, p = .033). However, no significant effect was observed in the Simultaneous condition (0.78 vs. 0.77, p = .465). Therefore, it seems that the WM consolidation process in the Simultaneous condition is already completed when CT is short. We agree with the notion that with a simultaneous presentation, the WM consolidation process stops at a certain time point in the no-cue long-CT condition, leading to the attenuation of task performance improvement. Therefore, at least the latter phase of the RI in the no-cue long-CT condition should be characterized as the maintenance process. However, here we argue that the cessation of the WM consolidation process in the Simultaneous condition does not mean that the memory representations are fully consolidated.

One distinction between the fully consolidated and partially consolidated memory representations is that fully consolidated memory representation is little affected by time-based decay. With a simultaneous display, Ricker and Hardman (2017) demonstrated that participants’ memory performance was not affected by time-based decay even when the ISI was increased to 2.2 s. A similar effect is also observed in previous studies adopting a sequential display paradigm (Barrouillet et al., 2004; Lewandowsky et al., 2004). However, other studies found that with a simultaneous display, lengthening the RI to around 5 s in the neutral-cue, post-cue, or no-cue condition significantly deteriorated observers’ memory performance (Astle et al., 2012; Ricker & Cowan, 2010; Sligte et al., 2008). Therefore, the different pattern of result observed in the present no-cue Simultaneous condition could possibly reflect some distinctions between the simultaneous and sequential presentation method. It is likely that the WM consolidation process terminates at a certain point during the retention period in a simultaneous presentation paradigm, leaving the memory representations only partially consolidated and thus susceptible to time-based decay. Furthermore, the present Simultaneous-Repeat condition demonstrated that repeating the simultaneous display did not reduce the magnitude of RCE. Hence, this phenomenon could possibly reflect a constraint to the WM consolidation process for simultaneously encoded memory representations, which deserves further investigation.

Previous studies have consistently found that a CTI of 300–400 ms was required to “make use of” the retro-cue to increase response accuracy (Gressmann & Janczyk, 2016; Pertzov et al., 2013; Souza et al., 2014; Tanoue & Berryhill, 2012). However, one question remains to be answered: why a certain length of CTI is needed for the retro-cue to be effective? According to the consolidation hypothesis proposed in the present study, the CTI serves as an extension of the WM consolidation process. This leads to the prediction that only the memory representations that received insufficient CT before the retro-cue presence would benefit from lengthened CTI. Therefore, in Experiment 3, we varied the total CT (200 ms or 500 ms) and CTI (100 ms or 400 ms) to examine whether stimuli with short CT would benefit more from lengthened CTI. The logic for this design was that under both short and long-CT conditions, memory representations should benefit from the increase of CTI. However, since the WM consolidation process completes at around 600–700 ms, the magnitude of CTI benefit in the short-CT condition should be comparable to a 300 ms WM consolidation process but only comparable to a 100 ms WM consolidation process in the long-CT condition. In other words, the consolidation hypothesis would predict a larger improvement of memory performance in the short-CT condition than in the long-CT condition when the CTI increases from 100 to 400 ms. However, the strengthening hypothesis would predict a comparable improvement of memory performance in the two conditions as CTI increases because it does not encompass a point of saturation for the strengthening process. Similarly, the head-start retrieval hypotheses would also predict a comparable improvement of memory performance in the two conditions as CTI increases because the same amount of time is increased in both conditions for participants to retrieve the target representation. To avoid participants initiating the WM consolidation process during the retro-cue display time, the presentation time of retro-cue was reduced to 100 ms in Experiment 3.

Participants

The sample of this 1-hr experiment included 30 student volunteers (25 females; age range from 18 to 24; Mage = 20.13 years) from Nanjing Normal University who participated for a monetary reward (40 CNY). All participants were naïve to the experimental paradigm and reported normal color vision and normal or corrected-to-normal eyesight.

Apparatus and Stimuli

All the apparatus and stimuli were the same as in Experiment 1A.

Procedure

The sequence of presentations is shown graphically in Figure 10. The retro-cue paradigm used in the present study was similar to the one used in Experiment 1A with three exceptions. First, in the long-CT condition, the total CT for each stimulus was reduced to 500 ms (100 ms presentation + 100 ms mask + 300 ms RI). Second, we varied the CTI to be either 100 ms or 400 ms in both long and short-CT conditions. The different CTI trials were randomly intermixed in each condition. Third, the retro-cue display time was reduced to 100 ms. Each participant completed 16 practice trials followed by 16 blocks of 32 experimental trials, including eight long-CT blocks and eight short-CT blocks. The order of the two types of blocks was counterbalanced across participants. Articulatory suppression was implemented as in Experiment 1A.

Figure 11 depicts the mean accuracy for responses in each condition across the four SPs. The response accuracy data from retro-cue long-CT and retro-cue short-CT conditions were submitted to a 2 (CT) × 2 (CTIs) × 4 (SPs) three-way repeated-measures ANOVA. The ANOVA analysis yielded significant main effects of CT (F1,29 = 5.32, p = .028, partial η2 = 0.16), CTI (F1,29 = 19.67, p < .001, partial η2 = 0.40), and SP (F3,87 = 53.33, p < .001, partial η2 = 0.65). A significant CT × SP interaction was detected (F3,87 = 3.67, p = .015, partial η2 = 0.11). Bonferroni post hoc test revealed that this interaction was mainly driven by a significant difference in response accuracy for short and long-CT conditions at SP3 (0.80 vs. 0.74, p = .024) and SP4 (0.91 vs. 0.87, p = .004). Importantly, the analysis also detected a significant CT × CTI interaction (F1,29 = 5.26, p = .029, partial η2 = 0.15), indicating that increasing the CTI had a different effect on the magnitude of RCE in short- and long-CT conditions. Furthermore, Bonferroni post hoc tests showed a significant accuracy difference between the 100- and 400-ms CTI trials in the short-CT condition (0.72 vs. 0.77, p < .001) but only a marginally significant difference in the long-CT condition (0.76 vs. 0.78, p = .055). For a clearer illustration, the response accuracy data for short- and long-CT conditions are depicted in Figure 12. No CT × CTI × SP 3-way interaction was detected (F3,87 = 0.057, p = .98, partial η2 = 0.002).

In Experiment 3, we varied the CT and CTI across conditions and identified a significant improvement in memory performance in the short-CT condition when CTI was increased from 100 to 400 ms. This result replicated previous RCE studies (Pertzov et al., 2013; Tanoue & Berryhill, 2012). In contrast, only a marginally significant improvement was observed in the long-CT condition. The pattern of these results is in agreement with the consolidation hypothesis of RCE, which predicts the interaction between CTI and CT. However, the strengthening and head-start retrieval hypotheses cannot explain these findings adequately since they would both predict a comparable improvement of memory performance in the cue-short and cue-long conditions as CTI increases.

Based on the experimental results discussed so far, one alternative explanation needs to be considered. As the retro-cue always accurately directs attention to the target, it is possible that it accelerates the normal WM consolidation process since interference from other stimuli is reduced. Here we argue that this acceleration account is not likely to hold for two reasons. First, one previous study has demonstrated that the increase in the magnitude of RCE is not apparent in the initial stage but gradually develops over 300–400 ms (Tanoue & Berryhill, 2012). In addition, if we add up the average CT before the retro-cue (325 ms/item) and the time needed to develop the full RCE (300–400 ms) in this study, the total CT still falls into the range of 600–700 ms. Second, there is no significant performance difference between the current retro-cue short-CT + 400 ms CTI (200 ms CT before the cue + 400 ms CT after the cue) and no-cue long-CT (500 ms before the cue) conditions (0.771 vs. 0.766, p = .51), suggesting that the retro-cue did not significantly affect the speed of WM consolidation process. Hence, we conclude that the retro-cue induces the continuation of the WM consolidation process without altering its speed.

Previous studies have shown that compared to the deterministic retro-cue (always validly pointing to the to-be-tested item), probabilistic retro-cue (70% validity) led to smaller RCE (Dube et al., 2019; Gunseli et al., 2015). Furthermore, on the invalid cue trials, observers’ memory performance was significantly reduced compared to the valid cue trials (Dube et al., 2019; Pertzov et al., 2013). The invalid cue cost has been viewed as support for all previous explanations of the RCE. Since attention is directed to the irrelevant stimulus, participants cannot use the post-cue time to strengthen the to-be-tested memory representation, correctly remove the irrelevant representations, protect the to-be-tested representation from interferences, or start ahead to retrieve the to-be-tested representation. However, the consolidation hypothesis provides a widely different view of the mechanism of invalid retro-cue. According to the consolidation hypothesis, an invalid retro-cue should not affect the WM task performance if the memory items are fully consolidated because no additional CT is needed for the memory representations. Therefore, in Experiment 4, we manipulated the total CT (200 ms or 700 ms) and cue validity (70% or 100%) to examine the hypothesis that given enough CT, participants’ memory performance would be protected from the invalid retro-cue. The consolidation hypothesis predicts an interaction between CT and cue validity.

Participants

The sample of this 1-hr experiment included 30 student volunteers (20 females; age range from 18 to 27; Mage = 20.37 years) from Nanjing Normal University who participated for a monetary reward (40 CNY). All participants were naïve to the experimental paradigm and reported normal color vision and normal or corrected-to-normal eyesight.

Apparatus and Stimuli

All the apparatus and stimuli were the same as in Experiment 1A.

Procedure

The procedure was the same as Experiment 1A, except that we changed the no-cue condition to a probabilistic cue condition. In the probabilistic cue condition, the retro-cue correctly pointed to the to-be-tested item in 70% of the trials. Each participant completed 16 practice trials followed by 12 blocks of 32 experimental trials, including three deterministic-cue long-CT blocks, three deterministic-cue short-CT blocks, three probabilistic-cue long-CT blocks, and three probabilistic-cue short-CT blocks. Participants were informed about the validity of the cues at the beginning of each block. The order of the four types of blocks was counterbalanced across participants. Articulatory suppression was implemented as in Experiment 1A.

Figure 13 depicts the mean accuracy for responses in each condition across the four SPs. The response accuracy data were submitted to a 2 (CT) × 2 (cue validity) × 4 (SPs) three-way repeated-measures ANOVA. The ANOVA analysis yielded significant main effects of CT (F1,29 = 5.85, p = .022, partial η2 = 0.17), cue validity (F1,29 = 11.10, p = .002, partial η2 = 0.28), and SP (F3,87 = 74.11, p < .001, partial η2 = 0.72). Importantly, a significant CT × Cue validity interaction was detected (F3,87 = 5.41, p = .027, partial η2 = 0.16), indicating that the probabilistic cue exerted different effects on short and long-CT conditions. Bonferroni post hoc tests revealed that the probabilistic cue led to significantly worse memory performance in the short-CT condition (0.75 vs. 0.79, p < .001) but had little adverse effect in the long-CT condition (0.80 vs. 0.81, p = .29). Interestingly, a CT × Cue validity × SP 3-way interaction was also detected (F3,87 = 3.00, p = .035, partial η2 = 0.094). Bonferroni post hoc tests showed that the three-way interaction was mainly driven by an insignificant response accuracy difference between probabilistic and deterministic-cue conditions at SP1 when CT was short (0.74 vs. 0.74, p = .81) together with a numerical decrease in response accuracy at SP1 in the probabilistic-cue long-CT condition (0.74 vs. 0.77, p = .18).

To further examine our hypothesis that long CT can protect memory representations from the invalid retro-cue, we specifically extracted the response accuracy data of the invalid cue trials from the probabilistic cue condition. Although the inclusion of the SP1 data did not change the pattern of the current results, considering the different cue validity effects found at SP1, we included only the data from SP2 to SP4 in the current analysis. Figure 14 depicts the response accuracy data from invalid and deterministic-cue trials. The data were submitted to a 2 (CT) × 2 (cue validity) repeated-measures ANOVA. The analysis yielded significant main effects of CT (F1,29 = 8.47, p = .007, partial η2 = 0.23) and cue validity (F1,29 = 29.98, p < .001, partial η2 = 0.51). Critically, a significant CT × Cue validity interaction was detected (F1,29 = 9.84, p = .004, partial η2 = 0.25), indicating that the adverse effect of invalid cue was modulated by CT. Bonferroni post hoc tests showed that although invalid retro-cue led to a significant decrease in response accuracy in both short-CT (0.70 vs. 0.81, p < .001) and long-CT (0.78 vs. 0.82, p = .013) conditions, the magnitude of this invalid retro-cue cost was significantly larger in the short-CT condition than long-CT condition (0.11 vs. 0.04).

Experiment 4 demonstrated an apparent interaction effect between CT and cue validity. While the memory performance was comparable in the short- and long-CT conditions when the retro-cue was deterministic, participants’ memory performance significantly suffered from invalid retro-cue when CT was short. Moreover, the examination of invalid cue trials indicates that longer CT protected the memory representations from SP2 to SP4. These results provide clear evidence that memory representations were protected from invalid retro-cue in the long-CT condition.

Two points deserve some further comments: First, although the memory representations from SP2 to SP4 in the long-CT condition were protected from invalid retro-cue compared to the short-CT condition, we still observed a decrease in response accuracy in the long-CT condition. We propose that this effect could be attributed to a removal process of the target representation (Souza et al., 2014; Williams et al., 2013), which was misinformed by the invalid retro-cue (Gunseli et al., 2015). Second, no adverse effect of invalid retro-cue was found for memory representations at SP1 in the short-CT condition. Previous studies have shown that the consolidation process of the stimulus at SP 1 proceeds even after the presentation of the second stimulus (Ricker & Hardman, 2017; Ricker & Sandry, 2018). This better-consolidated memory representation may be less affected in the probabilistic cue condition, leading to protection from the invalid retro-cue. Nevertheless, future studies are needed to examine these possible explanations further.

In the present work, we examined the consolidation hypothesis of the RCE with a sequential display paradigm. In Experiments 1A and 1B, we demonstrated that sufficient CT completely erased the standard RCE regarding response accuracy. In Experiment 2, we again observed a significant interaction between CT and the RCE in a standard retro-cue paradigm, proving that this interaction effect is not restricted to sequentially presented display. In Experiment 3, we showed further evidence that the CTI served as a continuation of the WM consolidation process. When CT for memory items was long, observers required a shorter time to “make full use of” the retro-cue. In the last experiment, we found that sufficient CT effectively protected memory representations from invalid retro-cue. The pattern of these results is well predicted and explained by the current consolidation hypothesis of RCE and is difficult to accommodate by previous hypotheses.

The current sets of experiments delineate a very different role for the retro-cue compared to previous RCE research (for a review, see Souza & Oberauer, 2016). Instead of exerting its effect during the maintenance and retrieval process of a WM task, the efficacy of the retro-cue should be traced back to the WM consolidation process: only the memory representations with insufficient CT benefit from the lengthened consolidation period induced by the retro-cue. Besides explaining the experimental results in the current study, the consolidation account is compatible with several well-established phenomena observed in previous RCE studies. For example, a lower memory load reduces the magnitude of RCE when time parameters are held constant (Astle et al., 2012; Shepherdson et al., 2018; van Moorselaar et al., 2015). Reducing the number of stimuli in the display leads to an increase in the average CT each memory item (or pair of memory items for color stimulus) receives, thereby lowering the magnitude of RCE observed. In addition, memory items that were once cued in WM remain strengthened even when attention was later directed to another item (Rerko & Oberauer, 2013; Souza et al., 2016). Once cued memory item receives longer CT than the non-cued items and thus is better protected from interference or decay.

Nevertheless, it is important to note that the consolidation account proposed in the present study does not argue against the notion that other mechanisms may still play a role in the RCE. For example, the invalid cue cost observed in Experiment 3's long-CT condition could be attributed to the removal of non-cued items (Souza et al., 2014; Williams et al., 2013). In addition, several studies have shown that memory item benefited from repeated cueing (Rerko & Oberauer, 2013; Souza et al., 2015, 2018), which is not predicted by the current consolidation account. Therefore, it is also plausible that retro-cue also helps to refresh or strengthen the otherwise forgotten memory representations.

While previous WM consolidation research generally assumes that the consolidation process closely follows the encoding of memory items, the present findings reveal that a retro-cue can evoke the WM consolidation process even after the sequential display of several memory items. Moreover, the observation of significant RCE at the first SP renders sensory memory traces to be unlikely the source of memory enhancement here since 700 ms after the encoding process is beyond the duration that sensory memory traces could sustain (Sperling, 1960). Together, these two findings suggest that in addition to the sensory memory traces, the WM consolidation process may also rely on some other form of memory traces that is more durable than sensory memory. Recently, some scholars proposed an intermediate form of visual short-term memory between the rich but transient sensory memory and the sparse yet durable WM, which is termed the fragile visual short-term memory (FM; Pinto et al., 2017; Sligte et al., 2008; Vandenbroucke et al., 2011; see also Matsukura & Hollingworth, 2011). FM is found to last up to 4 s after stimulus offset, and its capacity is estimated to be around 6–16 items (Sligte et al., 2008). Importantly, Pinto et al. (2013) demonstrated that FM was erased only when the same type of stimuli appeared at the exact location as the to-be-recalled memory items. When visual masks or different types of stimuli were presented during the RI, FM was hardly affected and could still be accessed by a retro-cue before the test probe display (Pinto et al., 2013). These findings may also explain why, while sensory memory traces are easily disrupted by new visual inputs (Sligte et al., 2008; Sperling, 1960), previous studies consistently found that the WM consolidation process can proceed beyond a visual mask (Nieuwenstein & Wyble, 2014; Ricker & Hardman, 2017; Ricker & Sandry, 2018).

One question for the FM store to account for the WM consolidation process is why observers could not continue the consolidation process before selecting their response in the no-cue condition. Although the FM could be erased by the probe display in the current design, it should still be intact in the continuous-reproduction tasks in which there is no probe in the target's location. However, significant RCE is still observed in the continuous-reproduction tasks (Pertzov et al., 2017; Souza et al., 2016). Besides probe interference, another critical factor differentiating the retro-cue and no-cue conditions is the time point of the decision-making process. Nieuwenstein and Wyble (2014) examined the impact of the decision-making process on the WM consolidation process by inserting either a color discrimination task or a color detection task into a WM test. The rationale was that the color detection task would impose less demand on the decision-making process than the color discrimination task. They found that when inserted close to the stimuli display, the color discrimination task imposed a significantly stronger attenuation effect on the WM consolidation process than the color detection task. This finding suggests that the decision-making process could disrupt the WM consolidation process and may also explain why observers could not continue the consolidation process before selecting their response. Furthermore, several studies have shown that the response time distribution is roughly a linear function of set size (Astle et al., 2012; Conway & Engle, 1994; Donkin & Nosofsky, 2012) or the precision of memory representations which is directly affected by the set size (Pearson et al., 2014). This is also evidenced by the insignificant response time difference between no-cue short-CT and long-CT conditions at SP1 (1,037.89 ms vs. 1,088.74 ms, p = .21) and SP2 (1,135.23 ms vs. 1,114.45 ms, p = .75) in Experiment 1A. Nevertheless, participants still responded significantly slower in the short-CT condition at SP3 (1,105.12 ms vs. 995.22 ms, p = .048) and SP4 (1,069.47 ms vs. 899.10 ms, p = .002). This could be either due to distractions induced by the sudden onset of the probe display or the continuation of the WM consolidation process for the memory items at SP3 and SP4 in the no-cue short-CT condition. Therefore, more decisive evidence might be needed to exclude this possible confounding factor.

A second question regarding the current findings is raised by the WM state transfer research. Some recent studies revealed that longer gaps between visual stimuli would transfer these memory representations from the clustered and less stable active state to the more robust passive state (Z. Li et al., 2020; Zhang et al., 2022). Therefore, it is possible that memory representations in the current long-CT condition had enough time to be transferred to the robust passive state and thus no longer benefited from the retro-cue. That is, the retro-cue improves WM performance by inducing the transfer of memory representations into a passive state. Here, we argue that this state transfer account could not fully explain the current findings for two reasons. First, the state transfer account could not explain why the manipulation of a single consolidation period affects the item after but not before the gap (Ricker & Hardman, 2017). If a longer gap between stimuli induces the transfer of memory representations into the passive state, it should affect the memory item before but not after the gap. However, this is contradictory to the findings by Ricker and Hardman (2017). Second, there is evidence that the RCE is dissociable from the state of memory representations (Niklaus et al., 2019) and that memory representations already stored in the passive state could still benefit from retro-cue (Dube et al., 2019). Therefore, we propose that the state transfer account could not solely explain the current findings. Nevertheless, considering the time window for memory state transfer largely overlaps with the WM consolidation process for visual stimuli, it is likely that these two processes work in parallel to protect memory representations from decay or interference.

WM consolidation literature generally assumes that the consolidation process for the memory item at SP1 is completed before the processing of subsequent items. This finding is attributed to the attentional blink process initiated by the appearance of the first item (Ricker et al., 2018). This notion was partly confirmed as we did observe insignificant memory performance differences between the no-cue short and long-CT conditions in Experiments 1A and 1B (0.71 vs. 0.73, p = .29). However, at the same time, a significant RCE was detected at SP1 in the short-CT condition in both experiments. According to the consolidation account, this finding reflects that the memory item at SP1 was not fully consolidated. There are two possible explanations for these seemingly contradictory results. First, the RCE observed at SP1 in the short-CT condition could be attributed to mechanisms other than the continuation of the WM consolidation process. Second, the first memory item may not be fully consolidated in the current experiments. Here, we argue that the first explanation is not likely to hold because, otherwise, we should have observed a significant RCE in the long-CT condition too.

If the first item in the sequence is not fully consolidated, then why did previous studies find no sign of memory improvement at SP1 when CT increases? We propose that the different findings in the present study and WM consolidation literature could be attributed to the distinct experimental designs. In the present study, participants only need to report one memory item randomly selected from the four items. Therefore, taking up the processing time of the second item in sequence would only benefit memory performance in 25% of the trials. However, in the sequential whole-report design adopted in previous WM consolidation studies, participants were required to recall all the memory items. This means that participants would always benefit from fully consolidating the first item in the sequence and thus be more inclined to take up the processing time of the following item to fully consolidate the first item in the sequence. This explanation is also evidenced by the different trend of data between the current study and previous WM consolidation studies: While a numerical increase in response accuracy was observed for the item at SP1 in the present study's no-cue condition when the CT increases (Experiment 1A: from 0.73 to 0.75; Experiment 1B: from 0.68 to 0.71), the performance for the item at SP1 degraded steadily with increasing CT in a sequential whole-report paradigm (Ricker & Hardman, 2017; Ricker & Sandry, 2018). It is also interesting to note that some researchers have raised a similar account that the attentional blink phenomenon is possibly a higher-order competition between attentional selection and the WM consolidation process (Raffone et al., 2014).

The present study also taps into attention's role during the WM maintenance period. While some researchers suggest that focused attention is actively engaged during the maintenance period to counteract time-based decay or interference by rehearsing or attentional refreshing the WM representations (Gunseli et al., 2015; Janczyk & Berryhill, 2014; Vergauwe et al., 2014; Vergauwe & Cowan, 2015), others propose that WM maintenance may not require any active rehearsal via attention (Hollingworth & Maxcey-Richard, 2013; Rerko et al., 2014; Rerko & Oberauer, 2013) or that only the fully consolidated items are not actively maintained (Ricker et al., 2016; Ricker & Cowan, 2010; Ricker & Hardman, 2017). The pattern of results in current Experiment 4 was in line with the idea that fully consolidated items do not require the active maintenance process since the memory performance was protected from the invalid retro-cue when CT is sufficient. In addition, the current consolidation account also reconciles the contradictory conclusions reached in studies using a retro-cue paradigm to investigate the role of attention during the maintenance period. While studies adding a secondary task after the retro-cue presence generally observed no sign of attenuation of the RCE (Hollingworth & Maxcey-Richard, 2013; Rerko et al., 2014; Rerko & Oberauer, 2013), studies that manipulated the cue reliability had consistently found a deterioration of memory performance for the cued item when the cue validity was probabilistic (Dube et al., 2019; Gressmann & Janczyk, 2016; Gunseli et al., 2015). It should be noted that for the studies with a secondary task, the task was usually implemented 500–700 ms after the retro-cue presence (Hollingworth & Maxcey-Richard, 2013; Rerko et al., 2014; Rerko & Oberauer, 2013), which was long enough for observers to complete the consolidation process. Therefore, these studies generally observed no adverse effect of the secondary task on the magnitude of the RCE. Interestingly, two studies reported that the RCE was reduced by inserting a secondary task, and they implemented the secondary task either along with or closely following the retro-cue (Janczyk & Berryhill, 2014; Lin et al., 2021). Whereas for studies manipulating the cue validity, directing attention to the irrelevant items interrupted the consolidation process for the target memory representation and thus resulted in worse memory performance in the invalid cue condition. In sum, these two lines of seemingly contradictory research can be reconciled through the current consolidation account of the RCE.

The present work confirms a consolidation account for the RCE. It is demonstrated that the working memory consolidation process determines the effectiveness of retro-cue, and the post-cue time acts as a continuation of the working memory consolidation process. This new account bridges the gap between working memory consolidation and RCE studies and provides a unitary explanation of previous RCE research.

Astle, D. E., Summerfield, J., Griffin, I., & Nobre, A. C. (2012). Orienting attention to locations in mental representations. Attention, Perception, and Psychophysics, 74(1), 146–162. 10.3758/s13414-011-0218-3

Awh, E., Barton, B., & Vogel, E. K. (2007). Visual working memory represents a fixed number of items regardless of complexity. Psychological Science, 18(7), 622–628. 10.1111/j.1467-9280.2007.01949.x

Backer, K. C., Binns, M. A., & Alain, C. (2015). Neural dynamics underlying attentional orienting to auditory representations in short-term memory. Journal of Neuroscience, 35(3), 1307–1318. 10.1523/JNEUROSCI.1487-14.2015

Barrouillet, P., Bernardin, S., & Camos, V. (2004). Time constraints and resource sharing in adults’ working memory spans. Journal of Experimental Psychology: General, 133(1), 83–100. 10.1037/0096-3445.133.1.83

Bayliss, D. M., Bogdanovs, J., & Jarrold, C. (2015). Consolidating working memory: Distinguishing the effects of consolidation, rehearsal and attentional refreshing in a working memory span task. Journal of Memory and Language, 81, 34–50. 10.1016/j.jml.2014.12.004

Berryhill, M. E., Richmond, L. L., Shay, C. S., & Olson, I. R. (2012). Shifting attention among working memory representations: Testing cue type, awareness, and strategic control. Quarterly Journal of Experimental Psychology, 65(3), 426–438. 10.1080/17470218.2011.604786

Borgo, M., Soranzo, A., & Grassi, M. (2012). Psychtoolbox: Sound, keyboard and mouse. In MATLAB for psychologists (pp. 249–273). Springer. 10.1007/978-1-4614-2197-9_10

Brady, T. F., Konkle, T., & Alvarez, G. A. (2009). Compression in visual working memory: Using statistical regularities to form more efficient memory representations. Journal of Experimental Psychology: General, 138(4), 487–502. 10.1037/a0016797

Conway, A. R. A., & Engle, R. W. (1994). Working memory and retrieval: A resource-dependent inhibition model. Journal of Experimental Psychology: General, 123(4), 354–373. 10.1037/0096-3445.123.4.354

Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson’s method. Tutorials in Quantitative Methods for Psychology, 1(1), 42–45. 10.20982/tqmp.01.1.p042

Cowan, N. (1984). On short and long auditory stores. Psychological Bulletin, 96(2), 341–370. 10.1037/0033-2909.96.2.341

Cowan, N. (2010). The magical mystery four: How is working memory capacity limited, and why?Current Directions in Psychological Science, 19(1), 51–57. 10.1177/0963721409359277

Donkin, C., & Nosofsky, R. M. (2012). The structure of short-term memory scanning: An investigation using response time distribution models. Psychonomic Bulletin and Review, 19(3), 363–394. 10.3758/s13423-012-0236-8

Dube, B., Lumsden, A., & Al-Aidroos, N. (2019). Probabilistic retro-cues do not determine state in visual working memory. Psychonomic Bulletin and Review, 26(2), 641–646. 10.3758/s13423-018-1533-7

Gressmann, M., & Janczyk, M. (2016). The (un)clear effects of invalid retro-cues. Frontiers in Psychology, 7(1), Article 244. 10.3389/fpsyg.2016.00244

Griffin, I. C., & Nobre, A. C. (2003). Orienting attention to locations in internal representations. Journal of Cognitive Neuroscience, 15(8), 1176–1194. 10.1162/089892903322598139

Gunseli, E., van Moorselaar, D., Meeter, M., & Olivers, C. N. L. (2015). The reliability of retro-cues determines the fate of noncued visual working memory representations. Psychonomic Bulletin and Review, 22(5), 1334–1341. 10.3758/s13423-014-0796-x

Hao, R., Becker, M. W., Ye, C., Liu, Q., & Liu, T. (2018). The bandwidth of VWM consolidation varies with the stimulus feature: Evidence from event-related potentials. Journal of Experimental Psychology: Human Perception and Performance, 44(5), 767–777. 10.1037/xhp0000488

Hollingworth, A., & Maxcey-Richard, A. M. (2013). Selective maintenance in visual working memory does not require sustained visual attention. Journal of Experimental Psychology: Human Perception and Performance, 39(4), 1047–1058. 10.1037/a0030238

Janczyk, M., & Berryhill, M. E. (2014). Orienting attention in visual working memory requires central capacity: Decreased retro-cue effects under dual-task conditions. Attention, Perception, and Psychophysics, 76(3), 715–724. 10.3758/s13414-013-0615-x

Jolicœur, P., & Dell’Acqua, R. (1998). The demonstration of short-term consolidation. Cognitive Psychology, 36(2), 138–202. 10.1006/cogp.1998.0684

Katus, T., Andersen, S. K., & Müller, M. M. (2014). Common mechanisms of spatial attention in memory and perception: A tactile dual-task study. Cerebral Cortex, 24(3), 707–718. 10.1093/cercor/bhs350

Kleiner, M., Brainard, D., Pelli, D., Ingling, A., Murray, R., & Broussard, C. (2007). What's new in psychtoolbox-3. Perception, 36(14), Article 1. 10.1068/v070821

Landman, R., Spekreijse, H., & Lamme, V. A. F. (2003). Large capacity storage of integrated objects before change blindness. Vision Research, 43(2), 149–164. 10.1016/S0042-6989(02)00402-9

LaRocque, J. J., Eichenbaum, A. S., Starrett, M. J., Rose, N. S., Emrich, S. M., & Postle, B. R. (2015). The short- and long-term fates of memory items retained outside the focus of attention. Memory and Cognition, 43(3), 453–468. 10.3758/s13421-014-0486-y

Lepsien, J., Thornton, I., & Nobre, A. C. (2011). Modulation of working memory maintenance by directed attention. Neuropsychologia, 49(6), 1569–1577. 10.1016/j.neuropsychologia.2011.03.011

Lewandowsky, S., Duncan, M., & Brown, G. D. (2004). Time does not cause forgetting in short-term serial recall. Psychonomic Bulletin and Review, 11(5), 771–790. 10.3758/BF03196705

Li, Q., & Saiki, J. (2015). Different effects of color-based and location-based selection on visual working memory. Attention, Perception, and Psychophysics, 77(2), 450–463. 10.3758/s13414-014-0775-3

Li, Z., Zhang, J., Liang, T., Ye, C., & Liu, Q. (2020). Interval between two sequential arrays determines their storage state in visual working memory. Scientific Reports, 10(1), 1–9. 10.1038/s41598-020-64825-4

Lin, Y. T., Sasin, E., & Fougnie, D. (2021). Selection in working memory is resource-demanding: Concurrent task effects on the retro-cue effect. Attention, Perception, and Psychophysics, 83(4), 1600–1612. 10.3758/s13414-020-02239-0

Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390(6657), 279–281. 10.1038/36846

Makovski, T., & Jiang, Y. V. (2007). Distributing versus focusing attention in visual short-term memory. Psychonomic Bulletin and Review, 14(6), 1072–1078. 10.3758/BF03193093

Makovski, T., Sussman, R., & Jiang, Y. V. (2008). Orienting attention in visual working memory reduces interference from memory probes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(2), 369–380. 10.1037/0278-7393.34.2.369

Mance, I., Becker, M. W., & Liu, T. (2012). Parallel consolidation of simple features into visual short-term memory. Journal of Experimental Psychology: Human Perception and Performance, 38(2), 429–438. 10.1037/a0023925

Matsukura, M., & Hollingworth, A. (2011). Does visual short-term memory have a high-capacity stage?Psychonomic Bulletin and Review, 18(6), 1098–1104. 10.3758/s13423-011-0153-2

Matsukura, M., Luck, S. J., & Vecera, S. P. (2007). Attention effects during visual short-term memory maintenance: Protection or prioritization?Perception and Psychophysics, 69(8), 1422–1434. 10.3758/BF03192957

Miller, J. R., Becker, M. W., & Liu, T. (2014). The bandwidth of consolidation into visual short-term memory depends on the visual feature. Visual Cognition, 22(7), 920–947. 10.1080/13506285.2014.936923

Morey, R. D., & Rouder, J. N. (2022). BayesFactor: Computation of Bayes factors for common designs (Version 0.9.12) [Computer software]. https://CRAN.R-project.org/package=BayesFactor

Nieuwenstein, M., & Wyble, B. (2014). Beyond a mask and against the bottleneck: Retroactive dual-task interference during working memory consolidation of a masked visual target. Journal of Experimental Psychology: General, 143(3), 1409–1427. 10.1037/a0035257

Niklaus, M., Singmann, H., & Oberauer, K. (2019). Two distinct mechanisms of selection in working memory: Additive last-item and retro-cue benefits. Cognition, 183(1), 282–302. 10.1016/j.cognition.2018.11.015

Oberauer, K. (2005). Control of the contents of working memory—A comparison of two paradigms and two age groups. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(4), 714–728. 10.1037/0278-7393.31.4.714

Pashler, H. (1998). Attention and memory. In H.Pashler (Ed.), The psychology of attention (pp. 319–356). MIT Press.

Pearson, B., Raškevicˇius, J., Bays, P. M., Pertzov, Y., & Husain, M. (2014). Working memory retrieval as a decision process. Journal of Vision, 14(2), Article 2. 10.1167/14.2.2

Pertzov, Y., Bays, P. M., Joseph, S., & Husain, M. (2013). Rapid forgetting prevented by retrospective attention cues. Journal of Experimental Psychology: Human Perception and Performance, 39(5), 1224–1231. 10.1037/a0030947

Pertzov, Y., Manohar, S., & Husain, M. (2017). Rapid forgetting results from competition over time between items in visual working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43(4), 528–536. 10.1037/xlm0000328

Pinto, Y., Sligte, I. G., Shapiro, K. L., & Lamme, V. A. (2013). Fragile visual short-term memory is an object-based and location-specific store. Psychonomic Bulletin and Review, 20(4), 732–739. 10.3758/s13423-013-0393-4

Pinto, Y., Vandenbroucke, A., Otten, M., Sligte, I., Seth, A., & Lamme, V. (2017). Conscious visual memory with minimal attention. Journal of Experimental Psychology: General, 146(2), 214–226. 10.1037/xge0000255

Raffone, A., Srinivasan, N., & van Leeuwen, C. (2014). The interplay of attention and consciousness in visual search, attentional blink and working memory consolidation. Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1641), Article 20130215. 10.1098/rstb.2013.0215

R Core Team. (2022). R: A language and environment for statistical computing (Version 4.2.1) [Computer software]. R Foundation for Statistical Computing. http://www.R-project.org/

Rerko, L., & Oberauer, K. (2013). Focused, unfocused, and defocused information in working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39(4), 1075–1096. 10.1037/a0031172

Rerko, L., Souza, A. S., & Oberauer, K. (2014). Retro-cue benefits in working memory without sustained focal attention. Memory and Cognition, 42(5), 712–728. 10.3758/s13421-013-0392-8

Ricker, T. J. (2015). The role of short-term consolidation in memory persistence. AIMS Neuroscience, 2(4), 259–279. 10.3934/Neuroscience.2015.4.259

Ricker, T. J., & Cowan, N. (2010). Loss of visual working memory within seconds: The combined use of refreshable and non-refreshable features. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(6), 1355–1368. 10.1037/a0020356

Ricker, T. J., & Cowan, N. (2014). Differences between presentation methods in working memory procedures: A matter of working memory consolidation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(2), 417–428. 10.1037/a0034301

Ricker, T. J., & Hardman, K. O. (2017). The nature of short-term consolidation in visual working memory. Journal of Experimental Psychology: General, 146(11), 1551–1573. 10.1037/xge0000346

Ricker, T. J., Nieuwenstein, M. R., Bayliss, D. M., & Barrouillet, P. (2018). Working-memory consolidation: Insights from studies on attention and working memory. Annals of the New York Academy of Sciences, 1424(1), 8–18. 10.1111/nyas.13633

Ricker, T. J., & Sandry, J. (2018). The relationship between masking and short-term consolidation during recall from visual working memory. Annals of the New York Academy of Sciences, 1424(1), 91–101. 10.1111/nyas.13641

Ricker, T. J., Vergauwe, E., & Cowan, N. (2016). Decay theory of immediate memory: From brown (1958) to today (2014). Quarterly Journal of Experimental Psychology, 69(10), 1969–1995. 10.1080/17470218.2014.914546

Shepherdson, P., Oberauer, K., & Souza, A. S. (2018). Working memory load and the retro-cue effect: A diffusion model account. Journal of Experimental Psychology: Human Perception and Performance, 44(2), 286–310. 10.1037/xhp0000448

Sligte, I. G., Scholte, H. S., & Lamme, V. A. F. (2008). Are there multiple visual short-term memory stores?PLoS ONE, 3(2), Article e1699. 10.1371/journal.pone.0001699

Souza, A. S., & Oberauer, K. (2016). In search of the focus of attention in working memory: 13 years of the retro-cue effect. Attention, Perception and Psychophysics, 78(7), 1839–1860. 10.3758/s13414-016-1108-5

Souza, A. S., Rerko, L., & Oberauer, K. (2014). Unloading and reloading working memory: Attending to one item frees capacity. Journal of Experimental Psychology: Human Perception and Performance, 40(3), 1237–1256. 10.1037/a0036331

Souza, A. S., Rerko, L., & Oberauer, K. (2015). Refreshing memory traces: Thinking of an item improves retrieval from visual working memory. Annals of the New York Academy of Sciences, 1339(1), 20–31. 10.1111/nyas.12603

Souza, A. S., Rerko, L., & Oberauer, K. (2016). Getting more from visual working memory: Retro-cues enhance retrieval and protect from visual interference. Journal of Experimental Psychology: Human Perception and Performance, 42(6), 890–910. 10.1037/xhp0000192

Souza, A. S., Vergauwe, E., & Oberauer, K. (2018). Where to attend next: guiding refreshing of visual, spatial, and verbal representations in working memory. Annals of the New York Academy of Sciences, 1424(1), 76–90. 10.1111/nyas.13621

Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs: General and Applied, 74(11), 1–29. 10.1037/h0093759

Stevanovski, B., & Jolicœur, P. (2007). Visual short-term memory: Central capacity limitations in short-term consolidation. Visual Cognition, 15(5), 532–563. 10.1080/13506280600871917

Tanoue, R. T., & Berryhill, M. E. (2012). The mental wormhole: Internal attention shifts without regard for distance. Attention, Perception, and Psychophysics, 74(6), 1199–1215. 10.3758/s13414-012-0305-0

Vandenbroucke, A. R., Sligte, I. G., & Lamme, V. A. (2011). Manipulations of attention dissociate fragile visual short-term memory from visual working memory. Neuropsychologia, 49(6), 1559–1568. 10.1016/j.neuropsychologia.2010.12.044

van Moorselaar, D., Battistoni, E., Theeuwes, J., & Olivers, C. N. L. (2015). Rapid influences of cued visual memories on attentional guidance. Annals of the New York Academy of Sciences, 1339(1), 1–10. 10.1111/nyas.12574

Vergauwe, E., & Cowan, N. (2015). Attending to items in working memory: Evidence that refreshing and memory search are closely related. Psychonomic Bulletin and Review, 22(4), 1001–1006. 10.3758/s13423-014-0755-6

Vergauwe, E., Langerock, N., & Barrouillet, P. (2014). Maintaining information in visual working memory: Memory for bindings and memory for features are equally disrupted by increased attentional demands. Canadian Journal of Experimental Psychology, 68(3), 158–162. 10.1037/cep0000025

Vogel, E. K., Woodman, G. F., & Luck, S. J. (2006). The time course of consolidation in visual working memory. Journal of Experimental Psychology. Human Perception and Performance, 32(6), 1436–1451. 10.1037/0096-1523.32.6.1436

Wheeler, M. E., & Treisman, A. M. (2002). Binding in short-term visual memory. Journal of Experimental Psychology: General, 131(1), 48–64. 10.1037/0096-3445.131.1.48

Williams, M., Hong, S. W., Kang, M. S., Carlisle, N. B., & Woodman, G. F. (2013). The benefit of forgetting. Psychonomic Bulletin and Review, 20(2), 348–355. 10.3758/s13423-012-0354-3

Zhang, J., Ye, C., Sun, H.-J., Zhou, J., Liang, T., Li, Y., & Liu, Q. (2022). The passive state: A protective mechanism for information in working memory tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition, 48(9), 1235–1248. 10.1037/xlm0001092

Submitted: December 4, 2021 Revised: December 18, 2022 Accepted: December 27, 2022