Background: Students' academic self-efficacy maximizes likelihood for success and retention, yet prior research suggests that historically underrepresented (minoritized) undergraduate students in higher education and in college-level engineering show lower self-efficacy, which has been linked to histories of systemic exclusion. To address such gaps in student success, this work examines the effect of a new first-year undergraduate engineering design course on students' self-efficacy, as measured by students' assessment of their ability to achieve engineering design goals, and their confidence in their professional skills such as teamwork, communication, and leadership. It draws upon two aligned survey studies that examine this development (a) among the students participating in the course during the academic semester and (b) among both course participants and non-participants in the year following the course. Survey results for all students were considered, with attention to specific demographic subgroups traditionally underrepresented in engineering. Results: Analyses indicate effect of the course on self-efficacy and other examined constructs, such as communication and teamwork, during the course semester and continued effects in engineering design self-efficacy and tinkering self-efficacy in the year following course participation. Results also reveal differences for specific racial/ethnic and gender/sex subgroups in numerous constructs, including suggestion of specific effect for female students. Conclusions: This study's focus on the implication of engineering design education on self-efficacy and other critical professional outcomes, as well as its attention to specific demographic subgroups, adds to research on engineering education and the effect of design-focused coursework using project-based learning. The study indicates an increased potential role for such coursework, as early as the first year of a university trajectory, in fostering student growth and increased representation in the field. Findings on differences by gender/sex and by racial/ethnic groups, including clearer positive effect for female students but more complexity in effect for underrepresented racial/ethnic groups, support added research probing experience and outcomes within and across these groups.
Keywords: Engineering; First-year; Engineering design; Self-efficacy; Underrepresented groups; Design-based learning (DBL); Higher education
Supplementary Information The online version contains supplementary material available at https://doi.org/10.1186/s40594-024-00467-6.
Recruiting and retaining engineering students is especially important in modern times; technology-related careers employ millions in the U.S. alone, and projections suggest forthcoming critical shortages in the technology workforce globally (da Costa, [
Educational researchers have linked these gaps to historical institutional and structural racism and sexism within education, including STEM higher education, resulting in the systematic exclusion of racial and ethnic minorities and women (Graves et al., [
In parallel to these efforts, and not solely to support minoritized students, various strategies have been employed to engage students in their first year of engineering studies (Brannan & Wankat, [
This study describes the impact of the introduction of a first-year design course on engineering students at a highly selective university in the southeastern United States. Developed to offer mastery experiences with engineering prototyping tools, client-based design projects, and technical communication, the course provides students opportunities to build self-efficacy. Specifically, our study evaluated the effectiveness of the course by tracking the importance of core constructs in the development of self-efficacy, including a focus on engineering/academic engagement and professional skills such as communication and teamwork. The current study incorporated a pre- and post-intervention assessment to examine changes in self-efficacy during the first-year course. It additionally included data collected later in students' academic trajectories to determine how design-focused coursework may affect development following course enrollment. This study's focus on the implication of engineering design education on self-efficacy, including its attention to specific demographic subgroups traditionally underrepresented in engineering and its inclusion of data during the semester of participation as well as 1 year following participation, provides a direct contribution to the role of first-year engineering design coursework in student success.
Bandura ([
Self-efficacy in academic context describes a student's beliefs about his or her ability to achieve educational goals (Elias & MacDonald, [
Strategies to increase student self-efficacy may include intentional creation of "mastery experiences", which are opportunities for students to experience success in overcoming challenging tasks (Bandura, [
Research has examined differences in self-efficacy based on student characteristics, such as sex and gender, race, and ethnicity.[
Professional skills and competencies, such as teamwork, communication, and leadership, are posited to benefit learners and performance as practicing engineers. Anwar and Menekse ([
Within education, pedagogy including design-thinking and client-based work have capacity to contribute to both self-efficacy and professional competency development. Design thinking refers to cognitive processes used to develop solutions to particular contexts, with a focus on understanding end users and ideating to address needs (Wrigley & Straker, [
Project-based learning and design-thinking have been linked conceptually and operationally (Lin et al., [
Based on the aforementioned results linking self-efficacy and professional skills with academic success (Anwar & Menekse, [
This effort expands current scholarship on first-year undergraduate engineering education through its study of an innovative offering of a first-year engineering design course focusing on prototyping and iterative refinement of a client-based project (Reid et al., [
Below, we describe the focal program/intervention, the specific research questions, and the study methods.
The focal engineering school provided an opportunity to examine the effect of a design-focused and project-based course on student engineering attitudes and professional competencies. In particular, the engineering school developed a first-year course, Introduction to Engineering Design and Communication (EGR 101), which could serve as a model to test the effect of design-focused, project-based engineering education on self-efficacy and professional skills. This course was implemented in a School of Engineering in a highly selective R1 university in the U.S. South. EGR 101 began with a pilot student group in Fall 2017, with students (~ 50) in the pilot course selected at random by the Associate Dean. Students received credit for the course, so there was no disincentive to participate. The timing of the course and the availability of other required courses (e.g., math) were such that the cohort was not biased toward or against students with particular types (or absence) of AP credits. From 2018 onward, all incoming engineering students (~ 350) were enrolled in EGR 101.
In EGR 101, students learn an engineering design process (Daniels et al., [
Each section of the course had two to three faculty instructors who mentored teams through the design and prototyping process and graded all team assignments. Upper-class undergraduate engineering students served as teaching assistants (TA), with some TAs embedded in the class and others available in the makerspace classroom during the evenings and weekends to support prototyping. Starting the first year of required enrollment for all students (2018–2019), given the number of students enrolled, students were divided into seven separate sections of 40–75 students each.
While first-year engineering courses are common, few focus so intently on students learning an engineering design process through a client-based project (Beier et al., [
Table 1 Sample projects
Client Project goal Ellerbe Creek Watershed Association Develop a system to catch trash that is flowing into Ellerbe Creek Physician practicing in Tanzania Design a low-cost colostomy bag that can be made of materials readily available in sub-Saharan Africa Duke University Emergency Department Design a physical model that can support physician training to repair shoulder dislocation Duke University clinical research lab Devise a method and related hardware to predict how long a media bag will last on a bioreactor North Carolina Zoo Develop an enrichment device for sea lions Duke Gardens Create an interactive display that demonstrates a Venus flytrap plant
Projects tackled by student teams in EGR 101. Clients from within and outside the university present problems, which are translated to a project goal for teams to tackle
Student teams completed their client-based projects following an engineering design process (Fig. 1). Similar to other design heuristics taught in universities and used in industry, this model focuses on seven key steps: (
Graph: Fig. 1Engineering design process
Prototyping and physically constructing a design solution was a central feature of EGR 101. The classroom design space included various tools and equipment for prototyping, such as 3D printers, laser cutters, power tools, hand tools, sewing machines, and soldering stations, as well as many low- and medium-fidelity materials such as wood, PVC, fasteners, glue, tape, circuit components, tubing, and cardboard. To build prototyping skills, students completed two tools mastery projects concurrent with the Design Analysis Stage (Fig. 1) at the beginning of the semester. Choices included computer-aided design and 3D printing, circuits and microcontrollers, laser cutting and bending, woodworking, and machine shop (mill and lathe). Each tools mastery project had introductory and explanatory material (often via video) to support student learning and was supported by TAs with expertise in prototyping. Because physical prototyping and technical skill development were key learning outcomes, student teams spent more than half of the semester in the prototyping and testing phases of their client-based projects (Fig. 1).
In addition to technical skills, teamwork, project planning, and technical communication were critical for the successful completion of the client-based design project. Like the steps in the design process, these topics were supported using pre-class videos and active in-class support. Student teams documented the results of their journey through the engineering design process through a series of technical memos, oral presentations, and a poster. Support for this emphasis on communication came from the university's writing program and embedded writing consultants that met with teams during class. Near-peer TAs supported teams to form cohesive, high-performing, self-directed teams.
Through our work, we sought to answer the following questions:
(1a) Does a first-year engineering design course using project-based learning pedagogy affect participants' engineering attitudes (primarily self-efficacy) and professional skills during the course semester? (1b) How might the effects of the course differ based on student specifically race/ethnicity and gender/sex?
(2a) Does a first-year engineering design course using project-based learning pedagogy affect participants' engineering attitudes (primarily self-efficacy) and professional skills in the year following course enrollment? (2b) How might the effects of the course differ based on student specifically race/ethnicity and gender/sex?
We examine these effects through two related survey studies implemented at distinct timepoints: within the course semester (addressing Q1a,b) and a year following the course (addressing Q2a,b). Our focus on engineering attitudes primarily addresses self-efficacy in engineering, including ability to succeed in engineering overall, in tinkering, and engineering design. These foci were determined based on aims of the engineering design program as well as conceptual research interest.
These questions were addressed through two related studies described below: Study 1, focused on participants within the first-year course semester, and Study 2, focused on course participants and a non-participant comparison group in their sophomore year. Both studies utilize related surveys examining focal constructs in engineering attitudes and professional skills. These studies are addressed jointly in this paper given their strong conceptual and empirical linkages, and because the two together provide more comprehensive response to our research questions.
Two additional points help to contextualize the empirical designs of Study 1 and Study 2 and the reasons they are described separately. First, during Spring 2018, the engineering school determined to require EGR 101 for all entering engineering students in subsequent years. With this, we were able to examine change within course participants during their semester of EGR 101 of engagement (Study 1), but we were not able to also examine a comparison group (without EGR 101 participation) during their first year. Study 2 thus provided two key elements. First, it provided a longer-term lens on participants. Moreover, it permitted focus on the difference between a comparison group (those enrolling in 2017–2018, the year before the course was required) and a treatment group (those enrolled in 2018–2019, the first year the course was required).
Second, though Studies 1 and 2 are conceptually connected, individual student datapoints could not be directly linked between the two studies. In Study 1, the IRB permitted identifiers that allowed for linking of pre-course and post-course surveys. That, however, did not allow for specific student identification; identifiers were participant-created and were based on students' telephone numbers and dates of birth. In Study 2, the study team received permission to use university-associated student IDs. Due to the change in IRB permitted identifiers between the two studies, we were not able to determine the number of students who completed the survey in both Study 1 and Study 2, nor were we able to directly link individuals' data between the two studies.
The campus IRB approved all research processes and informed consent was obtained from all participants. Participants included primarily students who matriculated into the School of Engineering when they entered the university; though, in few cases, participant students transferred into the School of Engineering after matriculation.
To address the research questions in the period during which the course was offered, we developed and utilized a pre-course (Time 1, or T1; start of semester) and post-course (Time 2, or T2; end of semester) survey assessment. Development of the survey was informed by external research and prior work among this article's study team. First, the research team developed a program logic model that included identification of hypothesized proximal outcome constructs. Second, the team conducted initial exploratory assessment with open-ended qualitative data collection from Fall 2017 pilot course participants, drawing from an exploratory sequential mixed-methods design framework (Creswell & Creswell, [
We additionally defined secondary outcome constructs in a Professional Skills domain, including areas such as teamwork, communication, and leadership.[
Table 2 Outcome construct measures
Construct Measurement source # items in scale; scale measure Example item α* 0.71–0.78 General engineering self-efficacy Mamaril et al. ( 6 items; 7-point scale where 7 = high self-efficacy (6) I can master the content in the engineering-related courses I am taking this semester 0.90–0.95 Tinkering self-efficacy Mamaril et al. ( 10 items; 7-point scale where 7 = high self-efficacy I am comfortable learning new tools 0.93–0.95 Engineering design self-efficacy Carberry et al. ( 7 items; 7-point scale where 7 = high self-efficacy Generating diverse ideas to solve a design problem 0.87–0.93 Engineering academic engagement n/a, original instrument 4 items; 7-point scale where 7 = high academic engagement I am extremely interested in engineering 0.88–0.93 0.81–0.87 Teamwork skills Carter et al. ( 4 items; 6-point scale where 6 = expert skill level Working with others to accomplish group goals 0.77–0.90 Communication skills Carter et al. ( 6 items; 6-point scale where 6 = expert skill level Communicating effectively with nontechnical audiences 0.80–0.89 Leadership skills Carter et al. ( 4 items; 6-point scale where 6 = expert skill level Motivating people to do the work that needs to be done 0.84–0.90
*The Cronbach α indicated here reflects results from our study samples; the range is provided to reflect alpha scores at each timepoint The survey also addressed selected constructs within Creativity and Personal Development that are not the focus of this paper. Cronbach's α ranges reflect results across this manuscript studies' timepoints of data collection, including Study 1 and 2
The target sample included all students participating in the course. Data collection occurred in 2018–2019 and 2019–2020, which were the first two years during which the course was implemented for all students. Table 3 provides an overview of the timing of data collection relative to first-year course implementation. Data were collected as an electronic survey using Qualtrics.
Table 3 Overview of data collection implementation, Study 1
Year EGR 101 Course implementation Study 1 implementation: first-year (semester of EGR 101 participation) focus 2017–2018 Course implemented for a portion of entering engineering students Survey study design developed 2018–2019 Course implemented for all entering engineering students Data collection with first-years enrolled in EGR101 in 2018–19; 2019–2020 Course implemented for all entering engineering students Data collection with first-years enrolled in EGR101 in 2019–20;
In total, 343 respondents completed data collection at both T1 and T2 (of 690 total students in classes for a response rate of 50%). Researchers linked T1 and T2 data at the individual level for paired analysis. Table 4 shows sample characteristics of participating students in Study 1. For the purposes of analysis, researchers grouped students who identified as African American or Black, and/or Hispanic/Latino into a single group (AABHL, akin the how URM is used as underrepresented minority). While the experiences of individuals within this group are undoubtedly unique, they have been historically underrepresented within engineering fields. In addition, combining these students into a single group permitted analyses based on race/ethnicity that otherwise would not have been feasible, given the relatively small number of AABHL individuals participating in the study.[
Table 4 Sample characteristics, Study 1
% Gender/sex*a Male 202 58.9 Female 139 40.5 Did not report 2 0.6 Race and ethnicity* White, Caucasian 162 47.2 Asian, Pacific Islander (API) 105 30.6 African-American, Black or Hispanic, Latino (AABHL) 29 8.5 Multiracial, other 45 13.1 Did not report 2 0.6 Course participation Took EGR 101 343 100 Did not take EGR 101 0 0
For analysis, we pooled the students into one sample to increase the sample size overall and for individual demographic groups. For each outcome construct, we created a construct measure. This was based on the mean of the individual variables included in each construct's multi-item measure. We assessed scale reliability for each scale using Cronbach's a (α ≥ 0.7 for all scales at each time point; see Table 2 for specific values).[
To address the research questions considering effect a year following course engagement, a survey was adapted from Study 1. It included all Engineering Attitudes measures as well as the Professional Skills teamwork measure, based on outcomes of core conceptual interest to course designers and based on evidence of effect in Study 1. Additional demographic information was collected and linked to surveys from academic administrative records. The design of Study 2 included sophomore students who participated in EGR 101 and students who did not by distributing the survey to sophomores at two timepoints. In Spring 2019, the survey was distributed to all sophomores enrolled in the engineering school. This survey captured the students who participated in the 2017 pilot the year prior (~ 15% of students entering the engineering undergraduate school in 2017–2018) as well as the majority who had not participated in EGR 101 in that year. In Spring 2020, the survey was distributed to all sophomores enrolled in the engineering school, all of whom all students participated in EGR 101 in their first year. See Table 5 for an overview of data collection timing relative to first-year course implementation.
Table 5 Overview of data collection implementation, Study 2
Year EGR 101 course implementation Study 2 implementation: sophomore year focus 2017–18 Course implemented for a portion of entering engineering students Study design initiated (via Study 1) 2018–19 Course implemented for all entering engineering students Study design formalized. Data collection implemented with sophomores enrolled in EGR 101 in prior year (treatment group); 2019–20 Course implemented for all entering engineering students Data collection implemented with sophomores enrolled in EGR 101 in prior year (treatment group);
With this data across these 2 years, we categorized respondents for analysis into two groups: (
All surveys were administered electronically through Qualtrics. In total, 295 sophomore students completed the survey representing a 53% response rate. Table 6 shows sample characteristics of participating students in Study 2. As with Study 1, for purposes of analysis, researchers again grouped students who identified as African American or Black, and/or Hispanic/Latino into a single group (AABHL).
Table 6 Sample characteristics, Study 2
% Gender/sex* Male 155 52.5 Female 109 36.9 Did not report 31 10.5 Race and ethnicity* White, Caucasian 117 39.7 Asian, Pacific Islander (API) 70 23.7 African-American, Black or Hispanic, Latino (AABHL) 60 20.3 Other 2 0.7 Did not report 46 15.6 Course participation Took EGR 101 184 62.4 Did not take EGR 101 111 37.6
Demographics for students in Study 2 (N = 295). For Study 2, gender/sex and race/ethnicity data are based on university administrative data, and multiracial identification was not an option. For Study 2, "other" includes race/ethnicity categories that allow respondents to be potentially identifiable based on small sample size if named.
For analysis of Study 2 data, we utilized a process parallel to Study 1 to construct outcome measures. Outcome variables were analyzed in SAS using descriptive statistics for all respondents, by condition (participated in EGR 101 or not), by gender/sex, and by racial/ethnic identity. As with Study 1, we assessed scale reliability for each scale using Cronbach's a (α ≥ 0.7 for all scales at each time point; see Table 2 for specific values).[
We assessed the degree to which EGR 101 participation affects participants' reported development on 7 outcome constructs during the course semester. Results integrate paired t-tests, effect sizes, and linear regression models. For clarity in writing, text generally provides specific statistical test results only for the overall group. Where results pertaining to subgroups are discussed, tables should be considered for added information on statistical results.
Findings indicated that course participants experienced gains in six of the 7 assessed outcome constructs (Fig. 2). Paired t-tests and effect sizes revealed that students improved, at statistically significant levels and with large effect sizes, in engineering design-self efficacy (t = 12.52, p ≤ 0.01, d = 0.72), communication skills (t = 12.90, p ≤ 0.01, d = 0.74), and teamwork skills (t = 10.89, p ≤ 0.01, d = 0.62) between the beginning and end of the course. Results show that students also reported growth, at statistically significant levels and with medium effect sizes, in their tinkering self-efficacy (t = 9.22, p ≤ 0.01, d = 0.53), general engineering self-efficacy (t = 5.89, p ≤ 0.01, d = 0.34), and leadership skills (t = 7.47, p ≤ 0.01, d = 0.43).
Graph: Fig. 2Survey results reported by EGR 101 students at beginning and end of the semester. Reported values are mean ± standard deviation. *** indicates p ≤ 0.001 using paired t -test. Cohen's d effect sizes are noted as L for large effect size (> 0.60) and M for medium effect size (0.30–0.60)
Results do not support evidence of statistically significant gains for engineering academic engagement. However, mean scores suggest that the average student reported relatively high academic engagement on the pre-survey (M = 5.57, SD = 1.06), which persisted through the end of the semester (post survey: M = 5.56, SD = 1.27), suggesting ceiling effect for these measures.
Disaggregating the larger sample by race/ethnicity, we found differences between student demographics in magnitude of gains across constructs (Table 7). White/Caucasian and Multiracial/Other students showed medium or large effect size for all constructs except engineering academic engagement (no group saw medium or high effect size in this), but API students saw large effect size only in engineering design self-efficacy, and AABHL students saw medium effect size for only tinkering self-efficacy and engineering design self-efficacy. Within the Professional Skills constructs, White/Caucasian and Multiracial/Other students show medium effect size for leadership skills while API and AABLH students show small effect size.
Table 7 Pre- and post-mean differences and effect size categories, by race/ethnicity
White, Caucasian Asian, Pacific Islander AABHL Multiracial, other 149 91 22 42 General Engineering Self-efficacy 0.35M 0.25 0.11 0.44M Tinkering Self-efficacy 0.60L 0.27 0.61M 1.03L Engineering Design Self-efficacy 0.83L 0.67L 0.43M 0.69L Engineering Academic Engagement − 0.05 − 0.04 − 0.03 0.20 Teamwork Skills 0.64L 0.38M 0.64L 0.55L Communication Skills 0.64L 0.59L 0.43M 0.55L Leadership Skills 0.51M 0.25 0.20 0.43M
Cell values indicate the difference between pre- and post-scores, where positive values indicate greater post-scores relative to pre-scores. Cohen's d effect sizes are noted as
In multiple linear regression models controlling for initial assessment scores and race/ethnicity as well as year of course enrollment (Table 8), API students show near-significantly lower improvement compared to White/Caucasian students in tinkering self-efficacy and in engineering design self-efficacy; Multiracial/Other students show significant or near-significantly higher improvement compared to White/Caucasian students in tinkering self-efficacy. In addition, we observed near-significant or significant interactions based on pre-score for Multiracial/Other students in tinkering self-efficacy, and for AABHL students in tinkering self-efficacy, with lesser growth among those with higher pre-scores. In multiple linear regression models examining Professional Skills constructs (Table 9), the only significant result by race/ethnicity is evident in teamwork skills, where API students evidence significantly lower improvement when compared to White/Caucasian students.
Table 8 Engineering attitudes regression results, race/ethnicity (β, p)
General Engin. self-efficacy Tinkering self-efficacy Eng. design self-efficacy Eng. academic engagement Baseline score 0.49*** 0.50*** 0.23*** 0.65*** Enrolled 2019–20 (2018–19 as ref.) 0.12 0.14 0.03 0.03 Race/ethnicity (White/Cauc. as ref.) API − 0.68 − 0.91 − 1.05 0.82 AABHL − 0.36 1.33 1.38 0.42 Multiracial and/or Other 1.00 1.52* 0.26 0.56 Interactions API*Baseline 0.09 0.14 0.18 − 0.15 AABHL*Baseline 0.02 − 0.30 − 0.28 − 0.08 Multiracial/Other *Baseline − 0.18 − 0.26* − 0.06 − 0.04 Intercept 2.94*** 2.91*** 4.56*** 1.88*** 0.34 0.33 0.12 0.27 18.74*** 18.54*** 5.06*** 13.34*** 8.00 8.00 8.00 8.00 304.00 304.00 304.00 304.00
t < 0.10; * p < 0.05; ** p < 0.01; *** p < 0.001
Table 9 Professional skills regression results, race/ethnicity (β, p)
Teamwork skills Communic. skills Leadership skills Baseline score 0.39*** 0.45*** 0.36*** Enrolled 2019–20 (2018–19 as ref.) 0.26** 0.13 0.12 Race/ethnicity (White/Cauc. as ref.) API − 0.19* − 0.22 − 0.76 AABHL − 0.04 − 0.72 − 0.89 Multiracial and/or Other − 0.03 0.26 0.07 Interactions API*Baseline − 0.05 0.05 0.12 AABHL*Baseline 0.11 0.17 0.17 Multiracial/Other *Baseline − 0.06 − 0.08 − 0.04 Intercept 2.61*** 2.50*** 2.81*** 0.18 0.22 0.18 8.32*** 10.66*** 8.20*** 8.00 8.00 8.00 306.00 306.00 306.00
t < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001
Within Engineering Attitudes constructs (Table 10), female respondents show higher effect size than male respondents in general engineering self-efficacy (medium for female respondents, small for male respondents) and tinkering self-efficacy (large for female respondents, medium for male respondents). Within Professional Skills constructs, female respondents showed large effect size for teamwork while male respondents showed medium effect size.
Table 10 Pre- and post-mean differences and effect size categories, by gender
Female Male 128 183 General engineering self-efficacy 0.47M 0.20 Tinkering self-efficacy 0.67L 0.49M Engineering design self-efficacy 0.87L 0.64L Engineering academic engagement − 0.02 0 Teamwork skills 0.56L 0.54M Communication skills 0.59L 0.60L Leadership skills 0.35M 0.43M
Cell values indicate the difference between pre- and post-scores, where positive values indicate greater post-scores relative to pre-scores. Cohen's d effect sizes are noted as
Multiple linear regression models examining Engineering Attitudes constructs and Professional Skills constructs, controlling for initial assessment scores and gender as well as year of course enrollment, do not detect significant differences between male and female respondents or significant interactions by pre/baseline score, though beta results for Engineering Attitudes constructs generally indicate lower scores for male respondents compared to female respondents (Tables 11, 12).
Table 11 Engineering attitudes regression results, gender (β, p)
General Engin. self-efficacy Tinkering self-efficacy Eng. design self-efficacy Eng. academic engagement Baseline score 0.45*** 0.44*** 0.21** 0.58*** Enrolled 2019–20 (2018–19 as ref.) 0.15 0.19 0.07 0.05 Male (female as ref.) − 0.67 − 0.04 − 0.61 − 0.22 Male*Baseline interactions 0.12 0.03 0.09 0.04 Intercept 3.09*** 3.08*** 4.69*** 2.28*** 0.32 0.29 0.11 0.26 34.99*** 30.22*** 8.88*** 25.67*** 4.00 4.00 4.00 4.00 304.00 304.00 304.00 304.00
t < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001
Table 12 Professional skills regressions, gender (β, p)
Teamwork skills Communic. skills Leadership skills Baseline score 0.41*** 0.46*** 0.41*** Enrolled 2019–20 (2018–19 as ref.) 0.28** 0.14 0.15 Male (female as ref.) 0.21 − 0.05 − 0.06 Male*Baseline interactions − 0.07 − 0.01 0.00 Intercept 2.49*** 2.46*** 2.52*** R-square 0.17 0.22 0.15 15.52*** 21.19*** 13.25*** 4.00 4.00 4.00 306.00 306.00 306.00
t < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001
Finally, we considered race/ethnicity and gender jointly. When examining effect size (Table 13), results show male White/Caucasian respondents improving with large effect size while female counterparts show medium effect size across multiple constructs. For all other racial/ethnic groups, female respondents show greater effect size than male respondents; this is starkest for AABHL participants, where female respondents show large effect size, but male respondents show no change across multiple constructs. Results further indicated that the gains experienced by students of different races/ethnicities differ in some cases by gender. This is most notable with API and AABHL respondents, with more medium-to-large effect size gains for female respondents in these groups compared to their male counterparts. For instance, female AABHL respondents saw gains in their engineering design self-efficacy with large effect sizes, a finding that was masked when we examined the combined mean scores of male and female students. In fact, male AABHL respondents were the only group of students that did not experience significant gains in engineering design self-efficacy.
Table 13 Pre- and post-mean differences and effect size categories, by race/ethnicity and gender
White, Cauc. female White, Cauc. male API female API male AABHL female AABHL male Multiracial, Other female Multiracial, Other male N 64 89 34 59 9 14 21 21 General engineering self-efficacy 0.46M 0.27M 0.36M 0.18 0.35M − 0.06 0.71L 0.17 Tinkering self-efficacy 0.62L 0.58 L 0.45M 0.16 0.91L 0.40 1.01L 1.05L Engineering design self-efficacy 0.88L 0.79L 0.96L 0.50M 0.98L 0.04 0.66L 0.73M Engineering acad. engagement − 0.18 0.04 − 0.05 − 0.04 0.44M − 0.37 0.27M 0.12 Teamwork skills 0.57L 0.70 L 0.49M 0.31M 0.83 L 0.52L 0.55 L 0.56M Communic. skills 0.55L 0.70 L 0.82 L 0.46M 0.48M 0.40L 0.38M 0.72L Leadership skills 0.43M 0.56L 0.43M 0.15 0.03 0.30M 0.12 0.74L
Cell values indicate the difference between pre- and post-scores, where positive values indicate greater post-scores relative to pre-scores. Cohen's d effect sizes are noted as
Effect sizes for the Professional Skills constructs showed male AABHL and Multiracial/Other respondents evidencing greater effect size than female respondent counterparts in communication and leadership skills; we find the opposite gender trend for API students. In addition, male White/Caucasian respondents show higher leadership skills effect size than do their female counterparts. Only Multiracial/Other students show a gender difference in teamwork effect size category, with female respondents showing larger effect size than male respondents.
Multiple linear regression models further reflect benefit for female respondents within certain racial/ethnic groups. In models examining Engineering Attitudes constructs incorporating gender by race/ethnicity interactions (Table 14), we see significant gender by AABHL interactions in engineering design self-efficacy; the interaction is near-significant for engineering academic engagement. Pairwise comparisons of least squares means show female AABHL respondents reporting greater gains than male respondent counterparts. Results also show near-significant interactions in engineering academic engagement for Multiracial/Other students, with female respondents reporting higher scores than male respondents. In models examining Professional Skills constructs (Table 15), we see a significant interaction for API by gender in communication skills, with female API respondents reporting higher scores than male respondent counterparts.
Table 14 Engineering attitude regression results, race/ethnicity*gender interactions (β, p)
General engin. self-efficacy Tinkering self-efficacy Eng. design self-efficacy Eng. academic engagement Baseline score 0.50*** 0.47*** 0.27*** 0.59*** Enrolled 2019–20 (2018–19 as ref.) 0.11 0.13 0.03 0.00 Male (female as ref.) 0.05 0.27 − 0.03 0.32 API − 0.19 0.05 − 0.02 0.21 AABHL 0.04 0.21 0.35 0.55 Multiracial and/or Other 0.25 0.38 − 0.08 0.71 Male*API − 0.03 − 0.36 − 0.27 − 0.36 Male*AABHL − 0.48 − 0.49 − 0.82* − 1.00 Male*Multiracial/Other − 0.34 − 0.09 0.03 − 0.72 Intercept 2.84*** 2.87*** 4.41*** 2.07*** R-square 0.34 0.32 0.13 0.28 16.53*** 15.15*** 5.01*** 12.61*** 9.00 9.00 9.00 9.00 304.00 304.00 304.00 304.00
t < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001
Table 15 Professional skills regression results, race/ethnicity*gender interactions (β, p)
Teamwork skills Communic. skills Leadership skills Baseline Score 0.37*** 0.47*** 0.41*** Enrolled 2019–20 (2018–19 as ref.) 0.25** 0.14 0.14 Male (female as ref.) 0.03 0.06 − 0.01 Race/ethnicity (White/Cauc. as ref.) API − 0.12 0.19 − 0.15 AABHL 0.20 0.17 − 0.14 Multiracial and/or Other − 0.03 − 0.08 − 0.26 Interactions Male*API − 0.11 − 0.38* − 0.27 Male*AABHL − 0.39 − 0.40 − 0.18 Male*Multiracial/Other 0.02 0.12 0.35 Intercept 2.66*** 2.39*** 2.63*** 0.19 0.24 0.19 7.51*** 10.19*** 7.72*** 9.00 9.00 9.00 306.00 306.00 306.00
t < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001
As with Study 1, results for Study 2 draw from independent t-tests, effect sizes, and linear regression models. For clarity in writing, text below generally provides specific statistical test results only for the overall group; where results pertaining to subgroups are discussed, tables can be used for added information on statistical results.
Independent t-tests and effect sizes (Fig. 3) shows that students in the treatment condition reported greater engineering design self-efficacy (t = 5.05, p ≤ 0.001; d = 0.64) and tinkering self-efficacy (t = 2.24, p = 0.03, d = 0.27) compared to students that did not take the course. The effect of EGR 101 participation for outcome variables remained significant in regression models controlling for gender and race/ethnicity demographic characteristics (engineering design self-efficacy: b = 0.70, p ≤ 0.001; tinkering self-efficacy b = 0.40, p ≤ 0.01).[
Graph: Fig. 3Survey results reported by students who did and did not take EGR 101. The survey was taken at the close of their sophomore year. Reported values are mean ± standard deviation. p value shown as t (< 0.1), * (< 0.05), ** (< 0.01), and *** (< 0.001). Cohen's d effect sizes are noted as high for > 0.60 and medium for 0.30–0.60
Study 2 analyses also examined differences by race/ethnicity, focused on students who are White/Caucasian, API, and AABHL (Table 16).[
Table 16 Mean differences and effect size categories comparing those enrolled/not enrolled in EGR 101, by race/ethnicity
White, Caucasian Asian, Pacific Islander AABHL N (T—Treatment; C—Comparison) 82 T/35 C 41T/29 C 35T/25 C General engineering self-efficacy 0.05 − 0.04 − 0.03 Tinkering self-efficacy 0.28 0.46M 0.37M Engineering design self-efficacy 0.71L 0.81 L 0.53M Engineering academic engagement 0.11 0.11 − 0.25 Teamwork skills − 0.30 0.48M 0.20
Cell values indicate the difference between group/condition scores, where positive values indicate greater treatment/EGR 101 scores relative to comparison group scores. Cohen's d effect sizes are noted as
In multiple linear regression models examining Engineering Attitudes constructs and controlling for initial assessment scores and race/ethnicity (Table 17), AABHL students show near-significantly higher results compared to White/Caucasian students in engineering academic engagement and lower results in teamwork. API students show significantly lower reported teamwork skills results compared to White/Caucasian students. However, and notably, results also show significant or near-significant teamwork interaction terms for AABHL by course enrollment and API students by course enrollment. For each, estimated marginal means show the treatment group of these racial/ethnic groups has significantly higher mean scores on teamwork than for those in the comparison group. Multiracial/Other students show significant differences from White/Caucasian students, but the small sample size (n = 2) should be considered in interpretation.
Table 17 Regression results, focus on race/ethnicity (β, p)
General engin. Self-efficacy Tinkering self-efficacy Eng. design Self-efficacy Eng. academic engagement Teamwork skills Treatment condition (non-enrolled as ref.) 0.05 0.28 0.71*** 0.11 − 0.30 Race/ethnicity (White/Cauc. as ref.) API 0.08 0.01 0.07 − 0.02 − 0.55 AABHL − 0.22 0.05 0.20 0.53 − 0.40 Multiracial/Other − 0.54 2.01** 2.05** 0.99 0.40 Interactions Treatment*API − 0.09 0.18 0.05 0.00 0.78 Treatment*AABHL − 0.09 0.09 − 0.14 − 0.36 0.50 Intercept 5.12*** 4.89*** 4.88*** 5.26*** 4.48 0.02 0.05 0.15 0.02 0.04 0.74 2.11 ( 7.08*** 0.87 1.88 ( 6.00 6.00 6.00 6.00 6.00 249.00 249.00 249.00 249.00 249.00
t < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001
Table 18 shows results by sex for mean differences and effect size. We find that the treatment condition reports higher scores than the comparison for both female and male respondents in engineering design self-efficacy, with an especially large effect size for female students (male: d = 0.51; female: d = 0.97). For tinkering self-efficacy, female students also show higher effect size, although the difference is actually slight (male: d = 0.28, female: d = 0.32) when compared to male respondents.
Table 18 Mean differences and effect size categories comparing those enrolled/not enrolled in EGR 101, by sex
Male Female N (T—Treatment; C—Comparison) 93T/62 C 75T/34 C General engineering self-efficacy 0.16 0.01 Tinkering self-efficacy 0.29 0.33M Engineering design Self-efficacy 0.48M 0.83L Engineering academic engagement 0.03 0.08 Teamwork skills 0.05 0
Positive values indicate greater treatment/EGR 101 scores compared to comparison group scores. Cohen's d effect sizes are noted as
In multiple linear regression models (Table 19), male respondents show higher tinkering and engineering design self-efficacy scores compared to female students. Of note to our study, regression results did not show sex by condition interactions to be significant for any of the outcome variables examined.
Table 19 Regression results, focus on sex (β, p)
General engin. self-efficacy Tinkering self-efficacy Eng. design self-efficacy Eng. academic engagement Teamwork skills Treatment condition (non-enrolled as ref.) 0.01 0.33 0.83*** 0.08 0.00 Male (female as ref.) 0.31 0.54* 0.37* 0.24 − 0.14 Treatment*male 0.15 − 0.04 − 0.34 − 0.11 0.05 Intercept 4.85*** 4.64*** 4.78*** 5.24*** 4.32*** 0.04 0.07 0.11 0.01 0.00 3.89** 6.46*** 11.18*** 0.44 0.41 3.00 3.00 3.00 3.00 3.00 264.00 264.00 264.00 264.00 264.00
t < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001
This paper quantitatively explores the effect of a first-year undergraduate engineering design course on a suite of student-reported outcomes, with a primary focus on Engineering Attitudes (self-efficacy and engagement) as well as Professional Skills, through two related studies. As noted previously, greater self-efficacy has been linked to persistence and achievement of goals within an academic context (Honicke & Broadbent, [
Overall, results provide evidence that a design-focused and project-based engineering course, implemented in a first undergraduate year, positively affects engineering attitudes (focused on self-efficacy) and professional skills development. When examining overall student gains during EGR 101, students in Study 1 reported significant gain from pre- to post-course in a majority of constructs assessed, including in constructs such as engineering design and tinkering self-efficacy, teamwork, communication, and leadership skills—that were most closely related to and targeted by the course content and structure. Furthermore, there was a clear ongoing effect on students' engineering design self-efficacy: per Study 2, students who participated in the course demonstrated greater engineering design self-efficacy at the end of their sophomore year compared to those who did not participate. This provides evidence of efficacy for the course's primary intention of self-efficacy development being sustained more than a year beyond course engagement, expanding upon prior evidence of effect within one semester (Siniawski et al., [
In addition, findings regarding differences in gains based on students' race/ethnicity and gender/sex provide an important step in providing a more nuanced understanding of where disparities exist in students' experiences of engineering coursework. This lens can identify whether there is opportunity for the course design or delivery to more equitably support gain for all students; it is particularly salient given how dominant institutions, including education systems, have centered relatively advantaged groups (McGee, [
In addition, and complicating AABHL findings, we observed that this study's baseline assessments, which provide a lens on perspective and ability at course and university entry, showed AABHL students as having the highest baseline scores among the different racial and ethnic groups examined for over half (4 of the 7) of outcome constructs examined, including central constructs of general engineering self-efficacy and engineering design self-efficacy. It is thus possible that their higher baseline scores may relate to lesser growth. It also speaks to complexity of considering self-efficacy change in relation to measures such as retention, particularly because research speaking to persistence in STEM fields and engineering among AABHL and other minoritized students suggests self-efficacy development as a facilitator of retention but also speaks to retention challenges for minoritized groups (Adedokun et al., [
We also found intriguing and complicating trends among API students relative to White students. While API students demonstrated growth during the course across nearly all examined outcome constructs, both effect size and regression results indicate reported lesser improvement than White/Caucasian peers across a large number of constructs measured. This is notable particularly given that API students are often assumed to be well-represented and high-performing in engineering disciplines, so they may not be viewed as benefiting from additional supports. Interestingly, Study 2 provided evidence of greater value for API students in the treatment condition compared to those who did not take the course. This suggests a potential difference in shorter- and longer-term effects, as well as difference when examining results within racial groups versus between racial groups, and merits further study.
The Study 1 results, in many cases, also indicate particular gain for students with multiracial or "other" racial identities. This result can be complicated to interpret, as this group by definition is composed of a heterogenous mix of multiracial or other racial identities. However, this may suggest evidence of the particular experience of multiracial students; this would support other research (Campbell, [
Overall, our focus on race/ethnicity can expand upon Sheu et al. ([
Differences in effect by gender/sex were noteworthy, generally showing greater benefit for female respondents than male respondents. This is a critical finding given gender/sex disparity in engineering (ASEE,), and it indicates potential value of this course model. For instance, during the course semester, we saw greater effect size for female students compared to male students in Engineering Attitudes and in Professional Skills constructs (e.g., tinkering self-efficacy, general engineering self-efficacy, teamwork skills), and we saw greater gains in female API and AABHL respondents in particular compared to their male respondent counterparts. In addition, Study 2 engineering design self-efficacy effect size results showed female students in the treatment group outperforming comparison group peers, though such results were not reflected in regression results significance. This potential gender/sex effect would underscore the value of team-based, design-focused courses for supporting female students in engineering (Coleman et al., [
This study intentionally does not directly compare results across Study 1 and Study 2 due to the inability to identify repeated respondents between the two studies; however, we did note that assessment of general engineering self-efficacy is lower for Study 2, during a sophomore year (both treatment and comparison), relative to Study 1's first-year post-course data. We observe evidence of similar findings in other work (Marra et al., [
Though our study adds to understanding of engineering instruction and the effects of early undergraduate engineering design coursework, it has several limitations that may help to inform and augment similar research efforts in the future. First, and unsurprisingly, our analyses were challenged by relatively small sample size among students from underrepresented minority groups, particularly students who identified as African American/Black or Hispanic/Latino. These students' underrepresented minority (URM) status meant that they are by definition smaller in number among engineering students, and this necessitated the grouping of minority students into a single underrepresented minority category (AABHL) for analytical purposes. However, existing literature speaks to difference in social and educational experience and attainment within and between ethnic groups comprising this category (Farley & Alba, [
Additional aspects of study design due to constraints and realities of the context can also be seen as limitations and inform future work. While our study benefitted from the incorporation of multiple timepoints across 2 years of students' academic trajectories, we were unable to link Study 1 and Study 2 data for individual students. This was due to differences in data collection processes between the two years and further complicated by the lack of a comparison group in Study 1. Study 2's comparison group also included only one cohort, while participants in the treatment drew from two different cohorts: a smaller number taking the course the same year as the comparison group and a larger number taking the course the following year, when it became a first-year requirement. While examinations of the treatment group by year do not indicate difference by year, this still presents an added variable to consider. Finally, one cohort of Study 2 respondents completed the sophomore year survey during Spring 2020, during the early stages of the COVID-19 pandemic. While it is possible that the context of general turmoil facing college students at this time may have affected students' perspectives on their confidence and future plans, it may be all the more noteworthy that we continued to see treatment students outperform the comparison group at this time point.
Based on findings from Study 1 and Study 2, current study limitations, and additional relevant areas of work, we envision several opportunities for future research building upon this study; this understands this study as taking an exploratory lens. Overall, we would encourage further explorations of the effect of engineering design courses on self-efficacy and professional skills; our work provides suggestion of effect, but further work could explore this in a different context or with different instructional models. One area of exploration might be which aspects of EGR 101 (e.g., success following iterative prototyping, autonomy in choice of project, etc.) contribute most significantly to development of self-efficacy.
Added research, perhaps building on this study's design, could be extended to other institutions that are considering introducing similar first-year design courses. Such research could explore effect of distinct university contexts, shed light on which different aspects of a course (e.g., teamwork, working with a real-world client, mastery of technical skills) are most impactful in related constructs, and help to address issues of smaller sample size for certain subgroups or provide more nuanced variables for of racial and ethnic identities.
In addition, when examining student trajectory over time, an additional lens on retention (i.e., whether students remain within engineering-related majors and ultimately obtain a degree) would be valuable given the known discrepancies in retention of students from varying backgrounds; we are taking specific steps currently. Regarding datapoints examined, ability to link data directly across first-year and sophomore year should also be built into future work, and an additional timepoint even later in an academic trajectory could provide a valuable lens on effect throughout university enrollment. This could be examined in conjunction with retention data to further examine longer-term associations between self-efficacy and persistence in engineering.
Further work should also explore the specific experiences or processes underlying gender/sex and race/ethnicity differences shown in this study. Our study was designed to identify differences, but its design did not permit understanding of specific factors prompting these differences. Added qualitative research, moving towards an explanatory mixed-methods design, could provide added understanding of mechanism underlying patterns addressed in this paper. This is under discussion as an extension of the current study and could include specific experimentation regarding adjustments that may support more equitable gain, such as an iteration including project foci addressing issues directly germane to underrepresented groups. Other recent work has indicated academic hope and STEM belonging are associated with persistence for underrepresented students (Hansen et al., [
Overall, results provide evidence that a design-focused and project-based engineering course, implemented in a first undergraduate year, positively affects engineering attitudes (focused on self-efficacy) and professional skills development during the course and positively affect engineering design self-efficacy even into the year following the course. Results also show difference in effect by gender/sex, with trends toward greater benefit for female participants compared to male counterparts in the course semester and thereafter. There is evidence of additional differences by race and ethnicity, though with more complexity. We see evidence of lesser gains for African American and Latinx respondents during the course as compared to White peers, but data collected a year later show these students demonstrating greater engineering design self-efficacy than their peer African American and Latinx students who did not participate in the course. These results support the value of, and can inform, further curricular efforts to integrate team-based engineering design projects into undergraduate curriculum in students' early years. In addition, the design of the studies discussed here, including the focus on student subgroups and on effect a year following course engagement, can inform ongoing research seeking to understand mechanisms for addressing student self-efficacy. This project's findings on differences by gender/sex and by ethnic and racial groups support added research probing experience and outcomes within and across these groups, and the variation in results across years supports the value of ongoing research examining students at different stages in, and as they progress through, an educational trajectory.
We acknowledge and significantly value the students who participated in the study; their contribution via provision of data was essential to the study's execution. We also acknowledge and value the contributions of Haley Walton in her editorial review of the manuscript to ensure clarity in writing.
JS led study design and oversight and contributed to data analysis, interpretation, and writing. MM led data analysis and contributed to interpretation of findings. MG contributed to study design and led implementation. LS contributed to data analysis. AS contributed to study design and interpretation of findings, particularly given their context for EGR 101, and led charts and table development, and procured the funds. All authors were major contributors in writing the manuscript and have read and approved the final manuscript.
Funding support for this study was provided by the Pratt School of Engineering at Duke University. Authors who designed and implemented the study were based in the School of Engineering (AS) or other entities at Duke University (JS, MM, MG, LS), but others from the funding body (Pratt School of Engineering at Duke University) had no involvement in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Ethical approval was received from the Duke University Institutional Review Board (IRB # 2018-0246). Participants provided electronic written consent via a Qualtrics (web)-based form, with approval for this based on the online survey nature of participant contact for data collection.
N/A (consent was provided via the IRB-approved consent process, and there is no identifiable information included).
The authors declare that they have no financial or non-financial competing interests.
Graph: Additional file 1. Survey Items.
• AABHL
- African American or Black, and/or Hispanic/Latino
• API
- Asian/Pacific Islander
• EGR 101
- Introduction to Engineering Design and Communication
• PACE
- Project to Assess Climate in Engineering
• STEM
- Science, technology, engineering, and math
• URM
- Underrepresented minority
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
By Jessica Sperling; Menna Mburi; Megan Gray; Lorrie Schmid and Ann Saterbak
Reported by Author; Author; Author; Author; Author
Jessica Sperling is Director of Applied Research, Evaluation, and Engagement at the Duke University Social Science Research Institute (SSRI). She is a trained sociologist and has focused on research and evaluation addressing varied arenas within education, including in STEM education efforts to address inequalities in access and persistence. At Duke, she teaches applied research design and methodology at the graduate and undergraduate level and is also associated with the Duke University Clinician and Translational Science Institute. She is the former Board President for the North Carolina Evaluator Network (formerly RTP Evaluators), a state affiliate of the American Evaluation Association.
Menna Mburi is a Ph.D. student in the department of Public Policy at the University of North Carolina. Her research interests include education and child and family policy, with a specific focus on promoting the health, wellbeing, and school readiness outcomes of young children in Black immigrant families. At Duke's Social Science Research Institute (SSRI), she served as a quantitative specialist and was additionally a former secondary educator with a master's degree in Education Policy and Management.
Megan Gray is a former Project Manager & Research Analyst at Duke University's Social Science Research Institute (SSRI). She has experience conducting research and evaluation studies with a primary focus on education and inequality. Prior to joining SSRI, MG worked in the nonprofit sector, where she provided evaluation support and technical assistance to community-based programs serving young children and their families.
Lorrie Schmid is a Lead, Data & Analytics Duke University's Social Science Research Institute (SSRI). LS focuses on quantitative data analytics within education research as well as other domain areas.
Ann Saterbak is Professor of the Practice in Biomedical Engineering and Director of the First-Year Engineering Program. Since joining Duke in June 2017, she launched the new Engineering Design and Communication course. Prior to Duke, she taught at Rice University, where she was on the faculty since 1999. AS is the lead author of the textbook, Bioengineering Fundamentals. For her contribution to education within biomedical engineering, she was elected Fellow in the Biomedical Engineering Society and the American Society of Engineering Education. She is the founding Editor-in-Chief of Biomedical Engineering Education.