key: cord-0056426-3svgi353
authors: Kwon, Hyunkyung; Capraro, Robert M.; Capraro, Mary Margaret
title: When I Believe, I Can: Success STEMs from My Perceptions
date: 2021-02-17
journal: Can
DOI: 10.1007/s42330-020-00132-4
sha: 381b205c259ad201d1e034feb899b1b3eba321b1
doc_id: 56426
cord_uid: 3svgi353

The increasing concern in North America regarding the workforce’s ability to maintain a competitive position in the global economy has led to an increased interest in effective science, technology, engineering, and mathematics (STEM) education. The purpose of this study was to investigate the effect of engaging in STEM project-based learning (PBL) activities on students’ mathematical problem-solving beliefs (PSB) and their perceptions towards SEM subjects and STEM careers. The data were collected using mathematical problem-solving belief and STEM semantics surveys. Participants (n = 68) were secondary students who participated in a STEM summer camp in which STEM PBL activities were utilized. Data were analyzed using descriptive statistics, paired-sample t tests, 95% confidence intervals, Cohen’s d effect sizes, and linear regression. Results showed that students increased their mathematical PSB (p < .001, d = .50) and perceptions towards science (p < .001, d = .68), engineering (p = .002, d = .56), mathematics (p = .015, d = .42), and STEM careers (p < .001, d = .86). Additionally, the linear regression analysis revealed that students’ mathematical PSB predicted their STEM career perceptions positively and statistically significantly (p < .05). These results support providing students with STEM PBL activities to increase their PSB and perceptions towards SEM subjects and STEM careers. Educators should utilize the results to provide opportunities for students to experience STEM PBL activities.

Résumé L'inquiétude croissante en Amérique du Nord concernant la capacité de la main-d'oeuvre à maintenir une position concurrentielle dans l'économie mondiale a conduit à un intérêt accru à l'égard de l'enseignement efficace des sciences, de la technologie, de l'ingénierie et des mathématiques (STEM). Le but de cette étude était d'étudier l'effet de la participation à des activités d'apprentissage par projet STEM sur les croyances des élèves en ce qui a trait à leur capacité de résoudre des problèmes mathématiques, ainsi que leurs perceptions à l'égard des matières SEM et des carrières en STEM. Les données ont été collectées à l'aide d'enquêtes sur les croyances en matière de résolution de problèmes mathématiques et sur la sémantique STEM. Les participants (n = 68) étaient des élèves du secondaire ayant participé à un camp d'été STEM au cours duquel des activités d'apprentissage par projet STEM avaient été utilisées. Les données ont été analysées à l'aide de statistiques descriptives, de tests t à échantillons appariés, d'intervalles de confiance à 95 %, de tailles d'effet d de Cohen et de régression linéaire. Les résultats ont montré que les élèves croyaient davantage à leur capacité de résoudre les problèmes mathématiques (p < 0,001, d = 0,50) et que leurs perceptions s'étaient améliorées à l'égard des sciences (p < 0,001, d = 0,68), de l'ingénierie (p = 0,002, d = 0,56), des mathématiques (p = 0,015, d = 0,42) et des carrières STEM (p < 0,001, d = 0,86. De plus, l'analyse de régression linéaire a révélé que les croyances des élèves en ce qui a trait à leur capacité de résoudre des problèmes mathématiques prédisaient leurs perceptions positives des carrières en STEM de manière statistiquement significative (p < 0,05). Ces résultats indiquent que les étudiants qui prennent part à des activités d'apprentissage par projet STEM croient davantage à leur capacité de résoudre des problèmes mathématiques et que leur perception des matières SEM et des carrières STEM s'améliore. Les éducateurs devraient utiliser ces résultats pour donner aux étudiants la possibilité de faire l'expérience d'activités d'apprentissage par projet STEM.

Keywords Mathematical problem-solving . Project-based learning . Problem-solving beliefs . STEM PBL . STEM career perceptions Unfortunately, North American education in general is not meeting expectations in terms of students achieving science, technology, engineering, and mathematics (STEM) learning goals. Students' skills in STEM areas are declining, and this decline has been long-standing. The National Commission on Excellence in Education in A Nation at Risk (1983) found that the U.S.A.'s role in the global economy is at stake. More recently, the decline has been linked specifically to the quality of educational experiences in K-12 programs (Kao, 2007; Ritz & Fan, 2015) . Additionally, Canada's poor performance on international tests in recent years has raised concerns regarding the country's ability to innovate and produce competitively in a global economy (Alphonso, 2013) . These concerns are manifested in the STEM job market's current competitiveness.

STEM job opportunities are increasing in number annually; however, employers cannot find enough skilled workers to fill required positions (Oberoi, 2016) . The reason is that development and production chains are becoming more reliant on STEM field expertise, and new, high-paying jobs are being created faster than ever in STEM-related careers. Additionally, because of this growing STEM emphasis, even non-STEM jobs that are not appropriate for STEM professionals require complex STEM understandings. This potentially creates scenarios wherein the full design, development, and production of a product cannot take place fully in North America. For example, high-tech glass, semiconductors, and next-generation photovoltaic cells are designed and created in North America but sent abroad for realization. This shows that North America, unfortunately, is not training STEM professionals fast enough to keep pace with increasing domestic demand (U.S. Department of Education, 2008; Torlakson, 2014) .

Partly, this shortage of STEM professionals is caused by the fact that North American students are falling behind in STEM. The failure of North America to adapt to and prepare for the changing global economy is clearly seen in the performance of its students on international assessments. Specifically, students in North America underperformed on international assessments measuring mathematics and science abilities, particularly those requiring higher cognitive demands. Student scores from Canada and the USA have been declining since 2003 on the Programme for International Student Assessment (PISA) tests (PISA, 2019; Chu, 2017) , averaging about − 10 and − 2 points per decade, respectively. The performance of Canadian students on the Trends in International Mathematics and Science Study (TIMSS) mathematics tests in recent years further shows that Canadian students' mathematics capabilities have been falling behind those of students from other top Organisation of Economics and Cooperation Development (OECD) countries, such as South Korea, Japan, Finland, Germany, and France (Gonzales et al., 2007; Provasnik et al., 2015) . U.S. students, on the other hand, have shown a slight improvement of about 5 points per decade on the TIMSS mathematics assessment but are still performing below the top-scoring OECD countries (Provasnik et al., 2015; Slavin & Lake, 2009 ). This may in part be due to the fact that U.S. students struggled to a greater extent with problems that required the application of mathematical concepts to real-life situations (Denson et al., 2015; U.S. Department of Education, 2008 Dorssen et al., 2006; Krishnamurthi et al., 2014) . Failure to score well on STEM assessments, such as the PISA and TIMSS mathematics tests, is indicative of inadequate STEM instruction and predicts the future decision of a student to not pursue STEM studies (LaForce et al., 2017) . Inadequate STEM instruction and preparation is a major factor in the low number of STEM professionals entering the North American workforce, but it is not the only one.

A second but no less important factor is that of STEM subject and career interest. Research has shown the role of students' beliefs and perceptions in promoting academic achievement, course choices, and career decisions (see Harackiewicz et al., 2005; Hidi & Harackiewicz, 2000; Schiefele et al., 1992) . Students who experience an adequate STEM education may develop positive career perceptions or career interest in STEM (Hayden et al., 2011; Oakes, 1990) . Students who had more positive beliefs and perceptions about mathematics were more deeply engaged in learning mathematics and performed significantly higher on standardized mathematics assessments (Boaler et al., 2018) . In contrast, students who had low perceptions of mathematics continued to fall behind in mathematics and lost interest in continuing studying mathematics (Schraw et al., 2001) . This stresses the importance of studying instructional strategies that increase not only students' mathematics achievement but also students' perceptions towards STEM subjects and careers. North America, in its attempt to address its lacking STEM representation, has focused mostly on strategies to increase students' mathematics achievement.

The overall poor mathematics performance in North America has led to an increase and rigor in the number of objectives that students are expected to learn. The first nationwide adoption of mathematics learning standards in the USA was launched by the National Council of Teachers of Mathematics (NCTM) when it first developed the Curriculum and Evaluation Standards for School Mathematics in 1989. Those standards were composed of two sections: content standards and process standards. The content standards comprised five strands: measurement, data analysis and probability, algebra, geometry, and number sense. The process standards comprised five strands: problem solving, reasoning and proof, communication, connections, and representation. The document in total contained only 54 content standards (NCTM, 1989) . In 2009, a state-led effort was undertaken in the USA to develop the Common Core State Standards. Primarily, state leaders from 48 states, two territories, and the District of Columbia collaborated to ensure all students, regardless of state of residence, graduate high school prepared for post-secondary life (Common Core State Standards Initiative, 2010). The Common Core State Standards document now contains approximately 200% more standards (i.e. 311 in total) than the original NCTM document published in 1989, and Common Core State Standards offer a more rigorous and coherent mathematics curriculum (NCTM, 2000) . This fact has not gone unnoticed by researchers (see Polikoff, 2017; Schmidt & Houang, 2012) . The rationale for making standards more rigorous, focused, and coherent and increasing the number of objectives was to improve student content knowledge and to encourage applicability across disciplines. Unfortunately, this policy ultimately created a high cognitive load for students and daunting demands on teachers without marked improvement in STEM literacy (U.S. Department of Education, 2008) or in performance on international comparisons.

One of the more beneficial aspects of the Common Core State Standards is that they emphasize students' deeper understanding of mathematics and problem-solving skills. Many researchers have found problem solving to be very important in developing students' mathematical knowledge and critical thinking skills (English, 1997; Singer & Voica, 2012) , and others have investigated varying instructional strategies that can foster students' problem-solving skills (Rosengrant et al., 2006; Suydam & Weaver, 1977; Van Harpen & Presmeg, 2013) . However, few studies have examined the influence of STEM project-based learning (PBL), an instructional strategy not included in Common Core State Standards, on students' problem solving. It is imperative to study STEM PBL in order to (1) understand how the instructional method may foster students' positive problem-solving beliefs and thus enthusiasm for both mathematics and STEM career aspirations (Kwon et al., 2019; Hayden et al., 2011) and (2) help instructors determine if it is an appropriate alternative or addition to the instructional strategies included in the Common Core State Standards in terms of increasing student perception in STEM. Therefore, researchers in the present study investigated the effect of engaging in STEM PBL activities during a STEM summer camp on students' mathematical problem-solving beliefs and their perceptions towards STEM subjects and STEM careers.

Many careers require mathematics and science skills, and these skills are the primary prerequisites for all other STEM fields. Additionally, the number of jobs that require STEM education are expected to grow 13% from 2017 and 2027 (Education Commission of the States, n.d.). However, the number of secondary students who are interested in STEM-related careers is decreasing (Kim, 2018) . This conundrum of greater job opportunities that are high paying and socially prestigious paired with a decrease in students actually choosing post-secondary STEM disciplines is difficult to untangle. One possible explanation is that students who are capable of pursuing advanced degrees in STEM might be selecting out of these fields because of a lack of quality early experiences dealing with STEM careers.

It is hard for students to pursue post-secondary STEM majors when they lack an understanding of what STEM professionals actually do. In fact, students' lack of early STEM experiences (DeJarnette, 2012; Swift & Watkins, 2004) , the dearth of role models for underrepresented persons in STEM fields (King, 2017) , and the lack of understanding of the daily activities of STEM professionals (Katehi et al., 2009 ) have been identified as potential contributors to the decline in STEM majors across North America. Perhaps the most commonly cited remedy for alleviating the contributing factors responsible for the diminishing numbers of those pursuing a STEM major has been experiences that allow students to recognize their potential for STEM success (Gradwell & Welch, 2003) . It is therefore important for students to have opportunities to build knowledge about STEM careers to understand STEM-related issues in the workforce and in the global community, and to see themselves as STEM capable (LaForce et al., 2017; Olson & Labov, 2014) . However, these opportunities are limited in availability and effectiveness in formal learning environments.

Although STEM education in both formal and informal educational environments seeks to improve interest in STEM careers, formal education is not sufficient to build students' familiarity with STEM professions (Denson et al., 2015; Kitchen et al., 2018; Roberts et al., 2018) . A STEM education can involve deep and meaningful tasks that are clearly linked to underlying mathematics, science, engineering, and technology concepts and foster the learning of the underlying learning objectives; however, many professions in STEM are too complex to adequately explore in a tight curriculum, such as what is often used in formal educational settings, especially when the nature of such professions is not contained within required learning objectives. Informal STEM learning opportunities, on the other hand, are flexible enough to allow for broad and in-depth experiences that support unique and specific learning outcomes. In fact, informal educational settings such as summer camps, nature centres, and museums have been at the forefront of providing STEM education that incorporates STEM profession familiarity for years. This is because an informal learning setting can provide and link presentations from STEM professionals in the field about a topic, instruction by STEM professionals who are addressing similar problems in the real world, and student learning to a specific job and education path (Kwon et al., 2019) . Formal settings typically do not have time in their learning objective-filled day to incorporate such activities. Importantly, research has also shown that informal STEM education can help students' perceptions towards STEM through these and similar added experiences (Dorssen et al., 2006; Krishnamurthi et al., 2014; Tseng et al., 2013) . Thus, informal STEM education opportunities should be provided for all students in order to maximize the opportunities for them to develop positive perceptions towards STEM learning and careers.

Problem-solving beliefs are completely subsumed within mathematics beliefs. It is almost impossible to remove the lurking variable of mathematics beliefs from problem solving (NCTM, 1989 (NCTM, , 2000 . Mathematics is, at its very core, problem solving, and problem solving is so much more than word problems. Mathematics is used to solve problems, and it is not enough for students to just learn mathematical facts and content (Dedane, 2009 ). In fact, many researchers and educators believe that problem solving is one of the most important components in a mathematics curriculum (Dedane, 2009; Guzman, 2018; NCTM, 2000; Stacey, 2005) . Additionally, research has shown that students' beliefs about mathematics affect students' mathematical achievement (Smith, 2014; Suthar et al., 2010) . Therefore, it is important to examine students' mathematical problem-solving beliefs.

To begin, we must know how beliefs affect individuals. Beliefs have been shown to be the best indicators of the decisions an individual makes throughout their daily life (Bandura, 1997; Dewey, 1938; Pajares, 2005; Zimmerman, 2000) . Beliefs also act as an individual's judgment, and they can be inferred from a collective understanding of what a person thinks, does, and claims (Pajares, 1992; Raymond, 1997) . Furthermore, beliefs influence an individual's career goals or choices, the effort they exert to reach said goals, and their endurance to persist when difficulties arise (Bandura, 1997; Pajares, 1992) , and, as a result, beliefs are an important predictor of whether or not a person has the necessary motivation to perform or complete a task (Bandura & Locke, 2003) . In a way, motivation is a key aspect of belief, and this relationship is important to consider when discussing mathematics persistence, interest, and achievement.

One's motivation to persist and succeed on a mathematics task is often referred to as mathematics belief. Mathematics belief, or the intrinsic belief in one's own ability to be able to successfully solve a problem, is positively linked to persistence in mathematics (Michaelides et al., 2019; Wyer & Albarracin, 2005) . Students who have a positive or strong mathematics belief have been shown to overcome mathematics failures and to persist through struggles to solve problems (Chinn, 2012; Hannula, 2016; Schoenfeld, 1989) . They simply believe they can (Carmichael & Taylor, 2005; Kloosterman, 2002) ! The role mathematics beliefs play in mathematics success has been shown in small niche studies (Di Martino & Zan, 2010; Kloosterman, 2002) and large-scale national and international comparisons (Brown et al., 2010; Hodgen et al., 2009; McLeod, 1994) . Therefore, mathematics belief has been identified as an important variable for secondary school students' mathematics success (De Corte et al., 2002; McLeod, 1994) .

Some researchers have suggested that mathematics belief works as a foundation that impacts adolescents' mathematics persistence and mastering of difficult mathematics content through a higher volume of practice with problem solving (Nicolaidou & Philippou, 2003; Pajares & Kranzler, 1995) . This repeated practice is perhaps the mechanism by which students with high mathematical beliefs emerge as strong mathematics capable students. This relationship is why many researchers believe that mathematical problem solving is at the heart of mathematics, as it fosters students' productive mathematical knowledge (Capraro & Han, 2014; English, 1997; NCTM, 1989 NCTM, , 2000 Singer & Voica, 2012; Van Harpen & Presmeg, 2013; Wilson, Fernandez, & Hadaway, 1993) . It is also why the current researchers argue that mathematics problem-solving beliefs directly influence mathematics beliefs and that students who have stronger problem-solving beliefs may have higher mathematics performance and competence.

In addition to affecting academic performance, students' mathematical problem-solving beliefs may also influence their career planning. Students' mathematical problem-solving beliefs are important because people who have strong mathematics skills often fail to pursue mathematics-related jobs because they have low self-efficacy perceptions about their mathematics competence (Zeldin & Pajares, 2000) . Mathematics perceptions differ from mathematics beliefs in that a belief is typically intrinsic, whereas a perception is often extrinsic (Vallerand & Bissonette, 1992) . For example, it is not uncommon for a person to claim a belief as his or her own while also recognizing someone else's influence over his or her own perceptions about a topic or thing (Hannula, 2016) . Secondary school students, especially, are susceptible to having their career trajectories affected by low perceptions and beliefs caused by poor mathematics achievement (Franz-Odendaal, Blotnicky, French, & Joy, 2016; Shapka et al., 2006) . For example, a longitudinal study of high school students' career aspirations in Canada reported that students in ninth grade who had low mathematics achievement had lower levels of career aspirations than those with higher mathematics achievement (Shapka et al., 2006) . Therefore, identifying the perceptions and beliefs of secondary students that influence their mathematics achievement is important.

Students' beliefs in general are especially important when considering personal factors. In fact, researchers in one study found a connection between beliefs and achievement and stated the following: "In the area of student perceptions and expectations, the research indicates that student belief that success in school is possible is one of the most important factors related to school achievement" (Wittrock & American Education Research Association [AERA], 1986, p. 311 ). In addition, middle school students' beliefs and confidence were significantly correlated with their mathematics achievement (Hammouri, 2004) . Finally, students with high levels of mathematics belief performed better on mathematics tests than those who showed high degrees of mathematics anxiety (Hammouri, 2004; Wittrock & AERA, 1986) . Therefore, it is important to investigate the effectiveness of instructional strategies that can promote secondary students' mathematical problem-solving beliefs, which can affect their achievement and their perception towards STEM careers.

STEM education is defined as a set of interdisciplinary instructional and learning practices designed for grades K-12 (Capraro et al., 2013; Aikenhead & Elliott, 2010; Leung, 2020) . One of these is STEM PBL, which conveys STEM topics and concepts through project-based learning activities. STEM PBL differs from standard problem-based learning in one critical way however. In standard problem-based learning, students typically encounter a single problem where the outcome is convergent, that is, students come to nearly identical solutions. In STEM PBL, students approach an ill-defined task with a well-defined outcome (Capraro et al., 2013) . This allows students to bring different prior knowledge to the task and, due to individual or team differences, potentially decide on very different solutions and even encounter different problems than their peers. Furthermore, this student-centred model is unique in that it encourages students to collaborate to solve relevant real-world problems (Capraro et al., 2017) . However, student progress is still typically measured using traditional problem-solving instruments.

There are many benefits to implementing STEM PBL. Engaging students in STEM PBL activities has been shown to improve higher order thinking skills (Capraro et al., 2013) and motivation to learn because it necessitates that students contribute to their own learning (Erdogan & Senemoglu, 2017; Wan Husin et al., 2016) . Inherent within the design of the STEM PBL instructional strategy is the constructivist paradigm wherein students should be active investigators rather than passive recipients during the learning process, which allows them to actively construct knowledge during the learning process (Kwon, 2017) . The constructivist approach to education is evident in the instructional elements of STEM PBL, which include discovery learning, hands-on learning, real-world scenario tasks, and student-centred learning (Steffe, 1995) . Thus, the instructional STEM PBL model is a student-centred and interdisciplinary model that engages students in collaboration to solve relevant real-world problems.

As the demand for problem solvers has increased, so too has the use of the STEM PBL instructional strategy (Kennedy & Odell, 2014) . This is partly because the constructivist component of STEM PBL has been found to positively influence student engagement and academic achievement (Capraro et al., 2017; Aikenhead & Elliott, 2010) . Additionally, engaging students in STEM PBL has the potential to positively impact their learning and enhance their problem-solving skills (Erdogan & Senemoglu, 2017; LaForce et al., 2017) . As such, STEM PBL may be used as an effective instructional strategy to increase students' achievement and problem-solving skills.

Currently, STEM PBL is often implemented in settings where high-stakes testing is not driving instruction. As a result, there has been more research on its effects when used in informal learning settings, such as summer camp programs, than in formal learning settings. Studies have shown that informal STEM PBL allows students to improve their interests, motivation, and learning in STEM (Becker & Park, 2011; National Science Board Commission on Precollege Education in Mathematics, Science, and Technology, 1983; Sanders, 2009) . Studies have also shown that students gain interest towards STEM fields after specifically engaging in STEM PBL during summer camp programs (Capraro et al., 2017; Mohr-Schroeder et al., 2014; Tseng et al., 2013; Yilmaz et al., 2010) . The efficacy of using STEM PBL during summer camp programs has been well studied.

In comparison to many formal learning settings, the duration of a summer camp is often short, a factor that may cause students to experience a relatively high cognitive load. However, one study found that even when experiencing higher cognitive loads in such a setting, students' perceptions towards the STEM fields increased (Capraro et al., 2017) . This is because students were encouraged to apply these subjects to real-life scenarios and were also able to choose the STEM courses that matched their own interests. As such, there was a higher possibility of students developing interests and commitment towards STEM topics (Capraro et al., 2017; Kwon et al., 2019 ). An approach with such desirable outcomes may provide more STEM workers in the future, but there exists little research on how mathematical problem-solving beliefs specifically are influenced by the use of STEM PBL in informal learning settings. Therefore, the researchers investigated a summer camp's implementation of STEM PBL and its effects on secondary school students' mathematical problem-solving beliefs that may in turn impact their perceptions towards STEM careers.

Research Questions 1. What is the effect of informal STEM PBL on influencing problem-solving beliefs and the value secondary school students place on STEM and STEM careers? 2. What is the relationship between secondary students' problem-solving beliefs and their possible career choices?

There were 253 middle and high school students who attended a 1-week or 2-week STEM summer camp during 2019 at a Tier One university. The camp had open enrollment, and students registered for the camp online. The participants for this study had the opportunity to choose if and how their data would be used, as informed consent was gathered from all of the students and their parents before the camp started. Specifically, they could choose one of three options: option 1 was, "I did my best on the survey, please use my data for research"; option 2 was, "I did not complete this survey to the best of my ability, please do not use my data"; and option 3 was, "I did my best on the survey, but I am not comfortable with my data being used for research, please do not use my data." To preserve the integrity of the study, the researchers decided to use only the surveys of participants who were sure about their response and willing to have their data used. As a result, 68 participants were selected for this study. There were 37 male participants and 31 female participants, and their grades ranged from 7th to 12th. Most of the students were from Texas, but three of them were from other states or countries around the world. Participants self-identified as White (67.7%), Latin or Hispanic (14.7%), Asian (11.8%), and African American/Black (3.0%), with the remainder providing no specific ethnicity. This demographic compares well to the larger population of students who participated in the summer camp. The larger population ethnicity distribution was the following: 69% White, 21% Hispanics/Latino, 8.4% Asian, and 1.6% African American/Black, with 1% of the students providing no specific ethnicity.

Two instruments were used for this study. They were administered using a pre-and postsurvey design through Qualtrics. Participants completed the Mathematical Problem-Solving Belief Survey (see Appendix 1), which measured students' mathematical problem-solving beliefs, on their first and last days of the summer camp. The Mathematical Problem-Solving Belief Survey consisted of the Indiana Mathematics Beliefs Scale (Kloosterman & Stage, 1992) and the Fennema-Sherman Usefulness Scale (Fennema & Sherman, 1976) . The Indiana Mathematics Beliefs Scale was developed by Kloosterman and Stage (1992) and was intended to be administered to secondary school students and college-level students and instructors. Kloosterman and Stage (1992) additionally suggested that the six-item version of the Fennema-Sherman Usefulness Scale (Fennema & Sherman, 1976) was an important part in measuring beliefs. Additionally, the STEM Semantics Survey was used to measure students' perceptions towards STEM and STEM careers.

The Mathematical Problem-Solving Belief Survey consisted of 36 Likert scale-type questions that measured participants' beliefs about mathematical problem solving on a 5-point scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree). There were six beliefs as well as six items intended to measure each belief. Among the six items measuring each belief, three of them were negatively worded. The researchers reverse scored the negatively worded items. For example, 1 would become a 5, 2 would become a 4, 3 would stay the same, etc. The internal consistency was a = .79, indicating a suitable reliability (Thompson, 2006) . Moreover, validity was reported for the data by computing inter-scale correlations (Kloosterman & Stage, 1992 ) that showed statistically significant correlations.

The STEM Semantics Survey (Tyler-Wood et al., 2010) was used to measure perceptions towards four constructs: science, engineering, mathematics, and STEM careers (see Appendix 2). The internal consistency was a = .73, indicating a reasonable reliability (Thompson, 2006) . The instrument consisted of 20 items, five items per construct. For each construct, there were five dichotomous adjective pairs. The dichotomous adjective pairs are "boring . . . interesting," "appealing . . . unappealing," "fascinating . . . mundane," "exciting . . . unexciting," and "means nothing . . . means a lot." Students rated these adjective pairs on a 7-point scale, with a higher value indicating a more positive perception for that construct in each case besides two. Three of the adjective pairs needed to be reverse coded before analysis.

Tyler-Wood et al. (2010) tested the validity for the STEM Semantics Survey by running exploratory factor analyses to determine if the variance accounted for a priori factor descriptors and if item loadings reasonably represented the a priori factor descriptors. The results of the analyses showed that all items loaded on the hypothesized factors. In other words, science, engineering, mathematics, and STEM careers were strongly associated with the intended construct, and these results showed credible evidence towards reconfirming the constructs (Tyler-Wood et al., 2010).

A non-randomized quasi-experimental design was used to understand how engaging in STEM PBL, a reform-based instruction, influenced students' problem-solving beliefs and STEM career aspirations. During the summer camp, students experienced two extended activities. Each activity was 90 min long each day. Thus, students who attended camp for 5 days experienced 900 min of STEM PBL instruction.

Students who attended the 2-week camp instead experienced four different 90-min activities for a total of 1800 min of STEM PBL instruction.

STEM PBL courses that were offered during the summer camp were coding, microcontrollers, cryptography, bridge building, and 3D printing. To give an example of the learning activities students engaged in, students in the coding course, after learning the basic and advanced concepts of coding, were tasked with using Python to make a storybook and word guessing game of their own design. All of the courses, however, allowed students to connect STEM knowledge and skills to real-world situations. Moreover, students actively engaged in collaborative, student-centred, and hands-on learning. In addition to these rigorous STEM PBL courses, students participated in panels where professors, counselors, and professional guest speakers in STEM fields came and spoke about their experiences. Student immersion in STEM experiences was further supported by laboratory tours during which students got first-hand experience of what STEM professionals do.

Data were analyzed using SPSS 25. Paired-sample t tests, Cohen's d effect sizes, and 95% confidence intervals were used to answer research question 1. Descriptive statistics were provided to contextualize the data in terms of centre and spread for the variables of students' problem-solving beliefs and perceptions in science, engineering, mathematics, and STEM careers before and after the camp. Paired-sample t tests were used to determine if there was a statistically significant difference between the pre-and postsurvey scores for both problem-solving beliefs and perceptions towards (a) science, (b) engineering, (c) mathematics, and (d) STEM careers. Moreover, Cohen's d effect sizes were calculated to facilitate meta-analytic thinking and to understand the impact of the experience in standardized metric (Capraro, 2004 ). To answer the second research question, scatter plots were used to analyze the data and check for directionality and correlation. Residual plots were then used to examine the goodness of fit. After the assumptions for linear regression were met (Thompson, 2006) , linear regression was used to examine the relationship between students' problem-solving beliefs and students' STEM career interests and to see if problem-solving beliefs can predict students' interests towards STEM careers.

We reported both tests of statistical significance and estimates of effect and precision. Statistical significance testing is limited by assumptions. Assumptions about sample size are used to guard against the inflation of incorrectly failing to reject the null hypothesis when in reality it should be rejected. Many assumptions, however, deal with consequences that are not always present.

In order to provide a complete picture, we provide all three classes of analysis (i.e. hypothesis test, measures of centre and spread, and estimates of effects). Hypothesis testing is perhaps ubiquitous with educational research and provides one-third of the picture of any study (Thompson, 2007) . We also computed and reported confidence intervals and effect sizes. Confidence intervals provide a clear depiction of the spread and centre of the data and range for which means from other similar studies would be contained 95% of the time. The effect size estimates provide insights into the practical importance or the magnitude of change one could expect for each change in standards deviation.

There were three major reasons for why we decided to report both effect sizes and 95% confidence intervals for all hypothesis tests. First, reporting standards of the American Psychological Association (Wilkinson & APA Taskforce, 1999) and AERA recommend their use (Duran et al., 2006) . Second, effect sizes are not a post hoc test but provide information about the magnitude of the influence of the experience without sample size being a factor, thereby mitigating type II error for smaller samples and type I error for large samples (Fritz et al., 2012; Sullivan & Feinn, 2012) . Finally, the two strategies, when employed together, provide information about location, precision, and impact of the experience (Coe, 2002; Thompson, 2007) .

The results were reported by research question, with each preceded by the descriptive statistics for the sample (see Table 1 ). The means of pre-and postsurvey scores for students' problem-solving beliefs (PSB) and perceptions of science, engineering, mathematics, and STEM careers increased after students engaged in the STEM PBL lessons. The mean difference ranged from 2.8 to 4.3. Students showed improvement in their perceptions towards STEM careers the most (X = 4.3), and their perceptions towards mathematics had the lowest improvement (X = 2.8).

To provide a complete picture of how students performed on the pre-and postsurveys, paired-sample t tests, 95% confidence intervals, and effect sizes were used. The five paired-sample t tests included the following: students' problem-solving beliefs and their perceptions towards science, mathematics, engineering, and STEM careers (see Table 2 ). For the five paired-sample t tests, the Bonferroni adjusted alpha level was adjusted to .01. The t test results indicated that students showed a statistically significantly positive increase (p < .01) in their mathematical problem-solving beliefs and in their perceptions towards science, engineering, and STEM careers. However, the change in students' perceptions towards mathematics was not statistically significant.

Calculating 95% confidence intervals provides three pieces of important information (Cumming & Finch, 2005) . First, it allows the reader to see the relative preciseness of the point estimate, in this case the mean. Shorter bars indicate greater precision of the estimate and lower variation. Second, the bars provide a visual about indicating other plausible point estimates for similar studies. That is, other similar studies could obtain means anywhere within the interval about 95% of the time. Finally, a bar's position relative to the others provides an opportunity to make statistical inference, or to know if the bars, given the same scale, show statistical significance (see Table 2 for 95% CIs).

Cohen's d effect size for the difference in problem-solving beliefs was 0.50, which shows practical importance. In addition, Cohen's d effect sizes for the differences in students' perceptions in science, engineering, mathematics, and STEM careers were 0.68, 0.56, 0.42, and 0.86, respectively (see Table 2 ). Differences in students' perceptions towards STEM careers shows the most practical significance. For every point increase in a student's score, it showed a 14.3% difference. Students' perceptions towards STEM careers increased about one standardized deviation (d = 0.86), which is almost a 6-point increase in score, or about an 85.8% increase. Although the students' perceptions towards mathematics did not show statistical significance, the Cohen's d effect size was 0.42, which shows that students' mean scores went up about 4 points and showed about a 57.2% increase.

Next, the researchers wanted to determine the relationship between students' mathematical problemsolving beliefs and their perceptions towards STEM careers. To do this, the Pearson correlation, r, was computed. There was a positive association between the two variables as well as a statistically significant value, r(68) = .34.

To determine if students' mathematical problem-solving beliefs had an influence on students' perceptions towards STEM careers, linear regression analysis was conducted. Mathematical problem-solving beliefs were the predictor of students' perceptions towards STEM careers. When using linear regression, it is important to consider the assumptions that may exist (Thompson, 2006) . In this instance, both normality and homoscedasticity were met. The results of the regression analysis showed a statistically significant regression equation (F (1, 66) = 6.61, p < .05). The analysis revealed that 11.9 % of students' mathematical problem-solving beliefs explained students' perceptions towards STEM careers (see Table 3 ). The researchers did not compute structure coefficients because they only had one predictor variable. The beta weights of the analysis indicated a positive relationship between problem-solving beliefs and STEM career perceptions, and the relationship was statistically significant (p < .05). The regression equation is as follows: STEM Career Perceptions = 18.92 + .119 PSB. The R 2 effect size for the regression is .84. This means that 84% of the variance of the dependent variable is accounted for by the predictor variable. This is a fairly large level of association providing support for the idea that the two variables are associated and move in the same direction.

Two important findings emerged from this research, both of which are important to the research community. One of them, however, has implications for the general population and perhaps even more for parents during a time of pandemic and physical isolation.

The first finding, important for the research and general communities, is that informal STEM PBL activities improve students' STEM perceptions. This finding supports findings from prior research (e.g. Capraro et al., 2017; Aikenhead & Elliott, 2010; Becker & Park, 2011; Mohr-Schroeder et al., 2014; Sanders, 2009; Tseng et al., 2013) . Factors that enhance the effectiveness of STEM PBL lessons may include informal learning settings, hands-on learning experiences, real-life application, collaboration, and motivation. All the STEM PBL activities included the use of engineering, science, and mathematics, and this may have allowed students to have an authentic learning experience where each subject was interconnected through an applied context. These real-life applications combined with the opportunity for students to choose to study subjects that matched with their own interests may have provided a relevance to learning that could have improved their STEM perceptions. As more evidence of the importance of informal experiences that incorporate STEM professionals, meaningful and real-world STEM activities, and links to STEM jobs comes to light, research will be able to move towards the development of the next iteration of which factors still need to be unpacked for increasing STEM success and enticing more students to undertake post-secondary STEM training so they can fill the myriad STEM job vacancies. The important next step is to understand how content knowledge follows these experiences and the viability of the length of time for which the experience may persist. For example, although these students were impressed by their short-term informal STEM learning experiences, we can only hope that this measured change persists through the school year. However, there is too little research that addresses how long the effects of such an intervention persist and the extent to which the effects measured in an informal STEM camp last.

Some of the benefits of STEM PBL can be linked to the tenets underlying its theoretical foundation. From this study, the authentic activities seem to contribute to better perceptions towards science, engineering, mathematics, and STEM careers and to lead to higher mathematical problem-solving beliefs. Although this finding is confounded by teachers, environments, and peers, classroom teachers should take note and seek ways to utilize STEM PBL in their classrooms. The constructivist paradigm experienced through STEM PBL integrally links the learning setting and the course content (Chang, 2005) . Applying course content to real-life issues and culture may increase students' achievement and perceptions towards STEM (Aikenhead & Elliott, 2010) . Decolonizing conventional science and mathematics education and implementing reform-based instruction, such as STEM PBL, can allow students to develop better STEM literacy and perceptions of STEM. Additionally, providing students with open-ended tasks that afford divergent thinking and diverse solutions may lead to increases in perceptions towards STEM subjects and STEM careers. After all, this may be the first time students participated in a class where very differentlooking constructions and completely different presentations are lauded for excellence in a complete absence of emphasis on uniformity. Regardless of the temporal effects, what is not clear is the "sticky" problem. That is, for how long do these experiences remain and for how long do these experiences play a part in students' long-term decision making about STEM and STEM careers? To mitigate this potential problem, we suggest that K-12 teachers work with informal STEM opportunities to provide a continuum of experiences that transition into the traditional formal setting.

The second major finding regarding how informal STEM experiences can have immediate effects on students' perceptions provides a powerful insight that parents and other adults who work with students should understand. The results of this study may inspire more adults, responsible for impressionable students, to seek summer camp and after-school experiences that meet the educational bar expressed in this article. After all, parents are perhaps the single greatest influence on students (Schlee et al., 2009; Toper et al., 2010) . Reaching parents with this information can have long-lasting and profound impacts on the STEM pipeline, such as reversing the trend of female, Black, and Spanish-speaking students' representation in the STEM professions (Garriott et al., 2013; Moakler & Kim, 2014) . It is important to consider, however, that these results do not address the power of virtual informal learning opportunities. It is possible that parents might even be able to find virtual online experiences that meet the bar for improving students' STEM career perceptions while mitigating the loneliness and disconnectedness of the ongoing COVID-19 pandemic isolation.

This study also provided further understanding of beliefs, which are often considered a trait of being. A trait of being is thought to be robust to change in the short term. For this study, we demonstrated that students' problem-solving beliefs could directly influence students' STEM career perceptions. Although this association has never been reported in the literature before, it is directly related to early work on problem solving and problem posing. For that research, over the long term, students have been shown to change their beliefs about both problem solving and problem posing (Arikan & Unal, 2015; Chen et al., 2011; Nicolaidou & Philippou, 2003; Pajares & Kranzler, 1995) . In fact, students have been shown to improve both their problem-solving and problem-posing skills in as little as a couple of weeks (Nicolaou & Philippou, 2007) . By holding problem-solving beliefs constant, we found that there is an important relationship between these and STEM career perceptions. This finding can be most useful to researchers in that future studies can be designed to examine how the state of problem-solving beliefs can be used as mediators or moderators for future research. Additionally, long-term studies can incorporate strategies to improve problem-solving beliefs and examine how such changes might influence STEM career perceptions.

The present study, nonetheless, provided evidence that problem-solving beliefs can benefit from shortterm interventions. The students attending the camp in the present study were at very different levels of mathematics ability and possessed different background experiences. This created a rich environment and one in which each student's cognitive demand was slightly or, quite possibly, dramatically different. Despite the complexities of the program and high cognitive load for students in a short span of time, students' mathematical problem-solving beliefs and perceptions towards science, engineering, mathematics, and STEM careers increased after engaging in STEM PBL activities. This is consistent with previous research that indicated that students gain interest towards STEM fields through summer camp programs (Mohr-Schroeder et al., 2014; Tseng et al., 2013; Yilmaz et al., 2010) . Importantly, students' mathematical problem-solving beliefs during the current study were associated with changes in their STEM career perceptions; specifically, students with better mathematical problem-solving beliefs are expected to have better perception towards STEM careers. Once students have better perceptions towards STEM careers, they are more likely to have career interest towards STEM careers.

This was only a 5-to 10-day-long intervention. Similar interventions in formal education settings can be expected to be longer, particularly during a school year. Such a long-term intervention would be expected to have an even greater positive impact on students' STEM career perceptions. We suggest the use of STEM PBL activities to promote mathematical problem-solving beliefs because they have the potential to increase students' perceptions towards STEM careers. Such an education outcome is vital to the North American economy. In other words, interventions that utilize STEM PBL activities can lead to a positive prediction that there may be a greater number of qualified students entering STEM jobs.

Acknowledgements Thanks to Aggie STEM for providing the data used for this study.

Belief 1: I Can Solve Time-consuming Mathematics Problems. + Math problems that take a long time don't bother me. + I feel I can do math problems that take a long time to complete. + I find I can do hard math problems if I just hang in there.

-If I can't do a math problem in a few minutes, I probably can't do it at all. -If I can't solve a math problem quickly, I quit trying. -I'm not very good at solving math problems that take a while to figure out.

Belief 2: There Are Word Problems That Cannot Be Solved with Simple, Step-By-Step Procedures. + There are word problems that just can't be solved by following a predetermined sequence of steps. + Word problems can be solved without remembering formulas. + Memorizing steps is not that useful for learning to solve word problems.

-Any word problem can be solved if you know the right steps to follow.

-Most word problems can be solved by using the correct step-by-step procedure.

-Learning to do word problems is mostly a matter of memorizing the right steps to follow.

Belief 3: Understanding Concepts Is Important in Mathematics. + Time used to investigate why a solution to a math problem works is time well spent. + A person who doesn't understand why an answer to a math problem is correct hasn't really solved the problem. + In addition to getting a right answer in mathematics, it is important to understand why the answer is correct. -It's not important to understand why a mathematical procedure works as long as it gives a correct answer.

-Getting a right answer in math is more important than understanding why the answer works.

-It doesn't really matter if you understand a math problem if you can get the right answer. + I study mathematics because I know how useful it is. + Knowing mathematics will help me earn a living. + Mathematics is a worthwhile and necessary subject. -Mathematics will not be important to me in my life's work.

-Mathematics is of no relevance to my life.

-Studying mathematics is a waste of time.

To me, SCIENCE is: f a s c i n a t i n g 1 2 3 4 5 6 7 m u n d a n e a p p e a l i n g 1 2 3 4 5 6 7 u n a p p e a l i n g e x c i t i n g 1 2 3 4 5 6 7 u n e x c i t i n g m e a n sn o t h i n g 1 2 3 4 5 6 7 m e a n sal o t boring 1 2 3 4 5 6 7 interesting

To me, MATH is: boring 1 2 3 4 5 6 7 interesting

An emerging decolonizing science education in Canada

Canada's fall in math-education ranking sets off alarm bells. The Globe and Mail

An investigation of eighth grade students' problem posing skills (Turkey sample)

Self-efficacy: The exercise of control

Negative self-efficacy and goal effects revisited

Effects of integrative approaches among science, technology, engineering, and mathematics (STEM) subjects on students' learning: A preliminary meta-analysis

Changing students minds and achievement in mathematics: The impact of a free online student course

Attitudes, gender and attainment: Evidence from a large-scale survey in England

Analysis of student beliefs in a tertiary preparatory mathematics course

Statistical significance, effect size reporting, and confidence intervals: Best reporting strategies

Project-based learning: An integrated science, technology, engineering, and mathematics (STEM) approach

STEM: the education frontier to meet 21st century challenges

Developing a useful and integrative STEM disciplinary language

Impact of constructivist teaching on students' beliefs about teaching and learning in introductory physics

An investigation on Chinese teachers' realistic problem posing and problem solving ability and beliefs

Beliefs, anxiety, and avoiding failure in mathematics

It's the effect size, stupid: What effect size is and why it is important

National Governors Association; Council of Chief State School Officers

Inference by eye: Confidence intervals and how to read pictures of data

Knowing what to believe": The relevance of students' mathematical beliefs for mathematics education

Skills needed for mathematical problem solving

America's children: Providing early exposure to STEM (Science

Benefits of informal learning environments: A focused examination of STEM-based program environments

Experience and education

Me and maths': Towards a definition of attitude grounded on students' narratives

Connecting informal STEM experiences to career choices: Identifying the pathway

Standards for reporting on empirical social science research in AERA publications

Education Commission of the States. (n.d.). STEM Demand

PBL in teacher education: Its effects on achievement and self-regulation. Higher Education Research and Development

The development of fifth-grade children's problem posing abilities

Fennema-Sherman mathematics attitudes scales: Instruments designed to measure attitudes toward the learning of mathematics by females and males

Experiences and perceptions of STEM subjects, careers, and engagement in STEM activities among middle school students in the Maritime provinces

Effect size estimates: Current use, calculations, and interpretation

Parental support and underrepresented students' math/science interests: The mediating role of learning experiences

Highlights from TIMSS 2007: Mathematics and science achievement of U.S. fourth-and eighth-grade students in an international context. U.S. Department of Education, Institute of Education Sciences

Technology education in Canada: A mosaic

Mathematical problem-solving strategies among student teachers

Attitudinal and motivational variables related to mathematics achievement in Jordan: Findings from the Third International Mathematics and Science Study (TIMSS)

ICME-13 Topical surveys: Attitudes, beliefs, motivation and identity in mathematics education -an overview of the field and future directions

Multiple goals, optimal motivation, and the development of interest

Increasing student interest and attitudes in STEM: Professional development and activities to engage and inspire learners

Motivating the academically unmotivated: A critical issue for the 21st century

Lower secondary school students' attitudes to mathematics: Evidence from a large-scale survey in England

Engineering in K-12 education: Understanding the status and improving the prospects

Innovation nation: How America is losing its innovation edge, why it matters, and what we can do to get it back

Engaging students in STEM education

New research shows declining interest in STEM

When teachers get it right: Voices of black girls' informal STEM learning experiences

The impact of college-and university-run high school summer programs on students' end of high school STEM career aspirations

Beliefs about mathematics and mathematics learning in the secondary school: Measurement and implications for motivation

Measuring beliefs about mathematical problem solving

Examining the impact of afterschool STEM programs

Effects of 3D printing and design software on student performance

Mathematics and science self-efficacy and STEM care ers: A path analysis

Problem-based learning (PBL) and student interest in STEM careers: The roles of motivation and ability beliefs

Boundary crossing pedagogy in STEM education

Research on affect and mathematics learning in the JRME: 1970 to the present

The relationship of motivation with achievement in mathematics

College major choice in STEM: Revisiting confidence and demographic factors. The Career Development Quarterly

Developing middle school students' interests in STEM via summer learning experiences: See Blue STEM Camp

Educating Americans for the 21st century: A plan of action for improving mathematics, science, and technology education for all American elementary and secondary students so

Attitudes towards mathematics, self-efficacy and achievement in problem solving

Efficacy beliefs, problem posing, and mathematics achievement

Lost talent: The under-participation of women, minorities, and disabled persons in science. The Rand Corporation

The economic impact of early exposure to STEM education. Committee for Economic Development

STEM learning is everywhere: summary of a convocation on building learning systems

Teachers' beliefs and education research: Cleaning up a messy construct

Gender differences in mathematics self-efficacy beliefs

Role of self-efficacy beliefs and general mental ability in mathematical problem solving

Highlights from TIMSS and TIMSS Advanced 2015: Mathematics and science achievement of U.S. students in grades 4 and 8 and in advanced courses at the end of high school in an international context

Is common core "working

Programme for international student assessment (PISA) results from PISA

Inconsistency between a beginning elementary school teacher's mathematics beliefs and teaching

STEM and technology education: International state-of-the-art

Students' perceptions of STEM learning after participating in a summer informal learning experience

Case study: Students' use of multiple representations in problem solving

STEM, STEM education, STEMmania. The Technology Teacher

Interest as a predictor of academic achievement: A meta-analysis of research

Parents social and resource capital: Predictors of academic achievement during early childhood

Curricular coherence and the common core state standards for mathematics

Explorations of students' mathematical beliefs and behavior

Increasing situational interest in the classroom

Trajectories of career aspirations through adolescence and young adulthood: Early math achievement as a critical filter

A problem-solving conceptual framework and its implications in designing problem-posing tasks

Effective programs in middle and high school mathematics: A best-evidence synthesis

How teacher beliefs about mathematics affect student beliefs about mathematics

The place of problem solving in contemporary mathematics curriculum documents

Using effect size-or why the P value is not enough

Students' beliefs on mathematics and achievement of university students: Logistics regression analysis

Research of problem solving: Implication for elementary school classroom

An engineering primer for outreach to K-4 education

INNOVATE: A blueprint for science, technology, engineering, and mathematics in California public education. Californians Dedicated to Education Foundation

Foundations of behavioral statistics: An insight-based approach

Effect sizes, confidence intervals, and confidence intervals for effect sizes

Parent involvement and student academic performance: A multiple mediational analysis

Attitudes towards science, technology, engineering and mathematics (STEM) in a project-based learning (PjBL) environment

Instruments for assessing interest in stem content and careers

A nation accountable: Twenty-five years after a nation at risk

Science, technology, engineering and math: Education for global leadership

Intrinsic, extrinsic, and amotivational styles as predictors of behaviors: A prospective study

An investigation of relationships between students' mathematical problemposing abilities and their mathematical content knowledge

Fostering students' 21st century skills through project oriented problem based learning (POPBL) in integrated STEM education program

Statistical methods in psychology journals

Mathematical problem solving

Handbook of research on teaching

Belief formation, organization, and change: Cognitive and motivational influences

Hands-on summer camp to attract K-12 students to engineering fields

Against the odds: Self-efficacy beliefs of women in mathematical, scientific, and technological careers

Self-efficacy: An essential motive to learn

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations