Evaluating Assignment Performance to Apply the Transparency in Learning and Teaching Framework in a Community College Physics Course

Zafar Waqar, PhD
zwaqar@ccbcmd.edu

From the Physical Science Department, School of Mathematics and Science, Community College of Baltimore County, Baltimore, Maryland.

ABSTRACT

Transparency in learning and teaching (TILT) is a framework designed to enhance student engagement, learning, and overall success. The efficacy of TILT has been demonstrated through positive data, making it a promising method for higher education institutions to implement and adapt. In addition to increasing traditional students’ mastery of critical analysis and reasoning skills, TILT removes the barriers of figuring out why, what, and how to complete coursework which is of particular benefit to underserved and non-native students. Given the demographics of the student populations at community colleges in the USA, there is an imperative need to explore the practical ways to implement the TILT framework in these settings. Enhancing teaching standards at community colleges has become a prime focus in recent years, due to the changing demographics of enrolled students. In particular, improving students’ learning and retention had become of prime importance. In the current study, assignment performance on physics lab reports were analyzed to select and adapt a target component of the assignment using the TILT methodology.

INTRODUCTION

Mary-Ann Winkelmes, founder of the transparency in learning and teaching (TILT) project, developed her methodology in order to empower students by giving them more time to master essential disciplinary skills, content knowledge, and therefore complete high-quality work (Head & Hostetler, 2015). TILT (https://tilthighered.com/) is a framework educators can employ to make course tasks transparent, giving students clarity about expectations that enable them to focus on targeted learning. Transparency requires simple wording with repetitive alternative words, so all students can understand the task’s objectives and how to complete it. Several studies have investigated the impact of TILT and have reported positive results (Boye et al., 2019; Kang et al., 2016; Magruder et al., 2019; Musselman et al., 2016; Porshnev et al., 2021).

In January 2024, adjunct faculty at the Community College of Baltimore County (CCBC) had the opportunity to engage in a learning community called adjunct teaching, learning, and scholarship (ATLAS) that focused on the TILT framework. During the weeklong session, the cohort read 2 relevant review articles: Introduction to Transparency in Learning and Teaching (Winkelmes, 2023) and Reaching First- Generation and Underrepresented Students through Transparent Assignment Design (Leuzinger & Grallo, 2019). These articles were the inspiration for engagement activities during the session including discussions, floating out-of-the-box ideas, suggestions, and proposed intervention plans from every adjunct faculty member of the group.

For example, the cohort was asked to discuss 3 follow-up questions: 1) What are some things you know from experience that students need? 2) What are some things that you have wanted to do but couldn’t because of time, common practice, etc.? and 3) What are some things that you’ve heard others are doing and you would like to try? Several novel ideas and unique personal experiences were shared by the group, followed by comprehensive discussion on the TILT framework’s key points, purpose, task and criteria.

Another activity involved 4 subgroups of the ATLAS faculty discussing specific topics from the first review (Winkelmes, 2023) including: 1) What ways do you see TILT enhancing your approach to student learning after reading this article? 2) After reading this article, what concerns or challenges do you anticipate when implementing TILT in your courses, and how do you plan to address them? 3) Reflecting on the article, how might you tailor TILT to suit the specific needs and dynamics of your course? 4) Reflecting on the article, how might you tailor TILT to suit the specific needs and dynamics of your student population?

The subgroups were also asked to reflect on the second article (Leuzinger & Grallo, 2019) and consider together: 1) What is the focus of the national survey discussed in the article? 2) What is the significance of transparent assignment design in the context of information literacy instruction in the article outline? 3) What are some key elements of TILT’s Transparent Assignment Template highlighted in the article? 4) What are some future research opportunities suggested by the authors of this article? Participants provided thoughtful and in-depth responses in the discussion, and they also responded to the points made by their colleagues to make meaningful connections.

In the concluding ATLAS session, the final task for each participant was to select an assignment from a course they teach and turn it into a TILTed assignment. The current study is about the experience of selecting an ongoing course assignment and TILTing it in a course taught by the author, Fundamentals of Physics I (PHYS 101). PHYS 101 at CCBC is classified as a general education course, meaning it introduces students to concepts that build a common foundation of knowledge that promotes responsibility, critical thinking, and lifelong independent learning (Community College of Baltimore County, 2024). All general education courses at CCBC are subject to periodic institutional review through a common graded assignment (CGA). The current study looks at data from the CGA administered in PHYS 101 courses in the spring and fall semesters of 2023. The results were used to apply TILT and lean six sigma (another methodology that incorporates factors of TILT) to an original assignment.

METHODS

Assignment Selection

The CGA for PHYS 101 at CCBC includes 4 lab reports. This study selected one, the summary report from the lab on Newton’s 2nd Law of Motion, to be TILTed. This lab report was selected because it has complex instructions spread across different documents; some instructions are included in the lab manual, some are found in a separate rubric for lab report scoring, and yet more are included in the assignment itself. Taken together the instructions are comprehensive, having all related instructions and scoring criteria. However, because all of the information was not available from a single source (ideally included in the assignment itself), it made the assignment unclear for students.

The main reason students struggled with the assignment was not making use of all available resources. Some students addressed only the assignment summary and they did not include the complete formal report, while some students completed detailed reports but omitted the summary. Some students surprisingly mentioned that their instructor assembled the lab apparatus, and they pressed the knob to record data on accelerating force and acceleration in the lab. Afterwards, when plotting data, many students plotted the wrong variables on the x- and y-axes, which made graphs incorrect. The students were supposed to assemble the lab apparatus themselves and collect data, but it was not mentioned in the original assignment. Also, it was the students’ task to identify which variable is independent and which one is dependent, to choose the x-axis for independent variables and the y-axis for dependent variables. Having no mention of these basic facts made students miss correct starting steps. For these reasons this study chose this lab report to reframe using TILT.

Assessing Past Assignment Performance

In order to see what areas the students needed most help with on the assignment, student performance on the CGA was evaluated. The sample included courses taught during spring and fall of 2023, with a total sample of 88 students. The assessment evaluated written and oral communication, critical analysis, technical component, information literature, and scientific reasoning. Microsoft Excel was used to calculate the means, medians, and quartiles of assignment scores.

RESULTS

Past Assignment Performance

Figure 1 shows the percentage grades on the CGA for specific criteria: written and oral communication, critical analysis, technical component, information literacy, and scientific reasoning. The analysis includes the scores of 88 students in PHYS 101 sections at CCBC in spring and fall of 2023. The scores for scientific reasoning are from 0 - 50%, with an average of 33%. The scores for critical analysis range from 25 - 50%, with an average of 37%. The scores for information literacy range from 25 - 75%, with an average of 44%. On the categories of written and oral communication and the technical component, both had scores from 50 - 75%, with average scores 65% and 63% respectively.

Figure 1. Assignment scores on the 5 specific criteria by 88 students in PHYS 101 during spring and fall of 2023.

CCBC’s goal for CGA performance is for students to score on average between 65 - 95% on the components. Only the average scores on written and oral communication from these PHYS 101 sections meet that goal, at the very lowest percentile (65%). The scores for the technical component nearly miss the target range at 63%, and the other criteria are further behind. The average score for information literacy was well below the target, and the task students are graded on in this assignment is providing a correct citation from a credible source. When students are provided with the information they need to satisfy the criteria of this citation task, the majority of students are expected to score 90% or above. Employing TILT to the instructions for this task should improve the performance. The lowest scores among the criteria were from critical analysis and scientific reasoning. These are advanced cognitive functions, and for students to perform their best they need clear, accessible instructions on what is expected from them. To this end, the assignment was TILTed for providing students transparent guidance on these tasks.

The TILTed Assignment

The original unTILTed assignment read: “Analyze your graph from Newton’s Second Law of Motion (refer to the lab activity). There are two important ideas to be extracted from your data. The first is that the resulting acceleration is proportional to the applied force. Does the graph support the first concept, or does it disprove it? How? The second concept is that the proportionality constant between the force and acceleration should be the reciprocal of the object’s mass. Does the graph support the second concept, or disprove it? How?”

Due to not having all relevant instructions included in the assignment itself, some students made the mistake of presenting the data and graph from a different lab, Newton’s 3rd Law of motion. This mistake may have not happened if the mathematical form F = ma of Newtons 2nd Law of motion had been included in the original assignment. Having this mathematical form visible in the instructions may have helped students to re-write this equation in the form of a straight-line equation:

y = mx + c, where x is independent variable, and y is dependent variable while c is y intercept and m is slope of the straight line.

One of the tasks for TILTing this assignment was to have Newton’s 2nd Law of Motion in mathematical form in assignment with mention of modifying it into form of a straight-line equation y = mx + c. Next the mathematical form of Newton’s 2nd Law of Motion (F = Ma) after modifications per independent and dependent variables becomes like the straight line equation as follows: a = (1/M) F [acceleration a = (1/accelerated mass M which is the total mass of the system consisting of glider mass + hanger mass, + added mass either on hanger or on glider) times the accelerating force F]. The accelerating force F is the sum of the mass of the hanger and the added mass on the hanger multiplied with acceleration due to gravity g = 9.8 m/s2. The comparison of this rearranged modified equation with straight line equation y = mx + c, shows that here y is acceleration a, slope m is (1/M) and x is accelerating force F. When the data is entered into an excel sheet with acceleration a on the 1st column (independent variable) and accelerating force F on the 2nd column (dependent variable), the plotted graph will show the slope m (1/M) and y intercept (c) per the data of the experiment. Thus, this equation now clearly shows the slope m = 1/M or M = 1/m. In the TILTed assignment, it is now straightforward to calculate the mass from the graph’s slope and draw a conclusion as required.

Another way the assignment was TILTed was by adding the formula for percent difference into the instructions as follows:

% difference = (Calculated Mass-Actual Mass)/(Actual Mass) X 100

Having this formula available increases students’ confidence that they will correctly complete the assignment. Students calculating a small value of the % difference will prove the 2nd criteria of Newton’s 2nd Law of Motion, which is that acceleration is inversely proportional to the mass. This can be seen from the equation a = (1/M) F.

The assignment was also TILTed by rewriting the instructions to include accessible language. This approach is also supported by the lean six sigma methodology, where alternative, simple words are used to explain complex concepts. This is particularly helpful when a considerable portion of the class has a remedial level of English language vocabulary. This work may be best completed by subject matter experts, in this case faculty with higher degrees in physics. These expert physics instructors will be better suited to utilize discipline-appropriate scientific reasoning and translate it into everyday language for students, while an instructor having no physics background would not be able to perform this job as required. The author has a PhD in physics and was able to enhance the assignment instructions appropriately in this way.

DISCUSSION AND CONCLUSION

Performance by the PHYS 101 students on the components of the CGA were on average low, and most were below the target level. A reason for their low scores on the assignment may be because the instructions were spread across multiple resources, making students confused and more likely to miss key information. With regard to the current study’s chosen assignment, while it was written carefully, the related information was spread across different documents including the lab manual, the assignment rubric, and the assignment itself. Students typically enroll in 3 - 4 courses during a semester, so to manage their workload they want to complete assignments as soon as possible. They are likely to try shortcuts in their pursuit of speed, and when instructions are spread across multiple resources, the students may not thoroughly consult each resource. To avoid this scenario, it is the responsibility of the instructor to have all instructions in one place, ideally the assignment itself. Instructors who are subject matter experts will be the most adept at unifying the course resources for student success. Making the CGA lab report assignment comprehensive and self-sufficient with all related instructions was a main goal of the TILTed assignment in this study.

Another factor which contributes to students’ lower grades is the fact that many students do not correctly understand the assignment. This typically happens due to the use of jargon, challenging language, and unclear wording in the assignment instructions. In other words, assignments which lack transparency can result in failing the students. If we critically analyze this situation, not having used simple easy and alternative words in an assignment is a failure on the part of the assignment, not the students. Student knowledge can’t be accurately assessed when instructions are unclear and blurry. Additionally, community colleges serve a large proportion of immigrant students who speak English as a second language. Crafting assignments using the TILTed approach to use clear terms and include multiple synonyms is key to student success. Instructors who have a full command of the subject matter are best suited to creating accessible TILTed assignments.

Overall, the data from student performance on the assignment strongly support the need to TILT the physics lab report assignment. Using clear language and collating all the necessary instructions into one document will give students a much higher chance of demonstrating their skills of scientific reasoning and critical analysis, which were the lowest of the criteria evaluated in this study. Now that the assignment has been TILTed, it should be used in future sections of PHYS 101 and the assignment performance evaluated to understand the effectiveness of the revisions.

ACKNOWLEDGEMENTS

I would like to thank ATLAS cofacilitators Haleh Azimi and Jewel K. Jackson for providing valuable training and resources. ATLAS helped me immensely, and their insights and guidance were the inspiration for this article.

REFERENCES

  1. Boye, A., & Tapp, S., Couch, J. N., Cox, R. D., Santa, L. G. (2019). Faculty voices and perspectives on transparent assignment design: FAQs for implementation and beyond. In M. A. Winkelmes, A. Boye, & S. Tapp (Eds.), Transparent design in higher education and leadership. (pp. 55–69). Routledge. https://doi.org/10.4324/9781003448396
  2. Head, A., & Hostetler, K. (2015). Mary-Ann Winkelmes: Transparency in teaching and learning (email interview). Project Information Literacy, Smart Talk Interview no. 25https://projectinfolit.org/smart-talk-interviews/transparency-in-teaching-and-learning/
  3. Kang, Y., Kelly, J., Murray, C., & Visbal, A. (2016). Transparency and problem solving: The UHD experience. Peer Review, 18(1/2).
  4. Leuzinger, R., & Grallo, J. (2019). Reaching first-generation and underrepresented students through transparent assignment design. Library Faculty Publications and Presentations, 11. https://digitalcommons.csumb.edu/lib_fac/11/
  5. Magruder, E., Scott, W., Willard, M., Ruiz Mesa, K., & Drew, S. (2019). Transparency to close opportunity gaps in the largest state system. In M. A. Winkelmes, A. Boye, & S. Tapp (Eds.), Transparent design in higher education and leadership. (pp. 55–69). Routledge. https://doi.org/10.4324/9781003448396
  6. Musselman, J., Lock, C., Long, C., Loughran, S., & Saclolo, M. (2016). Design and implementation: Transparency and problem-based learning at St. Edward’s University. Peer Review, 18(1/2).
  7. Porshnev, A., Cera, R., Sinclair, M., & Antonietti, A. (2021). Transparency in higher education: An investigation of university students’ perceptions in Italy and Russia. Ricerche Di Psicologia - Open Access, (3). https://doi.org/10.4324/9781003448396
  8. Community College of Baltimore County. (2024). General education assessment. CCBC. https://www.ccbcmd.edu/About/Accreditation/pages/Learning-Outcomes-Assessment.html
  9. Winkelmes, M. A. (2023). Introduction to transparency in learning and teaching. Perspectives in Learning, 20(1). https://csuepress.columbusstate.edu/pil/vol20/iss1/2