Improving learning outcomes from the undergraduate chemistry laboratory

The paper describes how to improve and evaluate learning outcomes in writing and critical thinking in the laboratory courses in the Bachelor program in Chemistry at the University of Bergen. The suggested changes are based on research that show that these learning outcomes are improved when the students are involved in self-assessment, where they perform peer-review and are encouraged to reflect on their own learning experience. This process needs to be accompanied with systematic and frequent feedback. It is also crucial to establish formal collegial groups where one can maintain systematic discussion regarding teaching in the laboratory courses. Suggested changes, and how they can be evaluated, are described in detail for two of the five mandatory laboratory courses in the program.

In the Bachelor (BSc) program in Chemistry at the University of Bergen (UoB) practical exercises in the lab together with correct and thorough reporting on the obtained results are important and integrated parts of the program. In total, 22 lab reports are submitted by students in the five courses that are mandatory in the BSc program. The program also includes a BSc project in the final semester where the students write a more extensive report based on a short research project. Students work individually or together on the practical part of the lab exercises, while all of the reports are submitted individually. Naturally, substantially resources are being allocated to the practical part. However, equally important is the time allocated for lecturers and teaching assistants to grade the reports and give proper feedback to students. Typically, students are given a detailed feedback where the main focus is the formal structure of the report together with a correct analysis of data and proper reporting of results. The students then update and resubmit the report. This process is often iterated multiple times for each report, leading to considerable use of teaching resources and student workload.
With such an extensive focus on analysing data and reporting results, and the corresponding time and resources allocated, one would assume that students develop good skills in writing and critical thinking. However, informal observations of the writing processes taking place in the BSc project indicate that a majority of students struggle with the task of critically evaluate results and to write a clear and concise report. It seems that the learning outcomes with respect to writing and critical thinking do not correspond to the time and resources allocated.
Furthermore, although intercommunication between lecturers on the different lab courses is taking place, it is not common, and usually does not happen in a structured manner. This leads to a lack of a common standard for lab report writing and other pedagogical approaches. However, through a collegial discussion group the department has recently established common guidelines of how a formal lab report should be structured. The structure follows the Introduction-Method-Results-and-Discussion (IMRaD) format. This paper describes how to improve and evaluate learning outcomes in writing and critical thinking from laboratory exercises in the BSc program in Chemistry. The proposed changes are based on the existing knowledge in education research, and some selected literature are described in more detail below.

LEARNING THEORIES AND FORMER STUDIES
In our department, as in many others, we teach lab report writing by the method labelled as the Formal Lab Report [4]. This method requires a formal report for each weekly experiment. The report is assessed by teaching assistants (TAs). The Formal Lab Report method emphasises the structure of the report. Consequently, students consider lab reports more about reporting and less about evaluation of results and critical thinking.
Klein [5] reviewed the current theories and studies concerning learning processes connected to writing. It was concluded that it is important to teach students "write-to-learn", but this has to take place in a process where receiving feedback is an essential part.
According to Biggs and Tang [6], learning outcomes are considerably improved when students are involved in self-assessment where they perform peer-review and in addition are encouraged to reflect on their own learning experience. This is an essential part of formative assessment, which can help the teachers to get feedback from the students on the teaching/learning process during the course, and that potentially leads to positive backwash [6].
Several other research papers have investigated different approaches for improving learning outcomes with respect to writing and critical thinking. Below, a brief description of some selected papers are given.
Deiner et al. [7] established a scaffold method for teaching report writing in chemistry. For each chapter of the lab report, the students were presented with a worksheet with important reflective questions that should be addressed, and each report in the lab course focused on a specific chapter. In total this constituted a scaffold for lab report writing. The goal was to transfer students from a random, nonstructured and non-enthusiastic approach to a more goal-oriented and organised writing process. It was found that there was an improvement in the students' ability to write a structured report and to be more aware of the communicative purpose of the report.
Vargas and Hanstedt [4] introduced the portfolio method, a combination of formal lab reports, portfolios and contemplative essays. Briefly, the students submit a formal lab report each week. The instructor then gives feedback on and grades the results and discussion section, while providing feedback only on the introduction and theory sections. At midterm the students submit portfolios consisting of two revised formal lab reports and a contemplative essay. In the latter students discuss the revisions they made to the scientific content and the writing, as well as what they learned from the revisions. The process was repeated in the second half of the semester based on the reports from the second part of the term.
McCreary et al. [8] introduced a concept called 'Workshop Chemistry', where the regular lecturer is replaced by a team of undergraduate TAs. Before and after the lab exercises the students, together with the TA discuss, reflect, and also present to each other, the relevant information about the exercise. The Workshop Chemistry groups were compared to groups of students undergoing traditional teaching, and it was found that the students from the Workshop Chemistry groups performed at a higher level when it came to critical thinking, data analysis and writing.
Holstein et al. [9] found that receiving feedback during the writing process, either by fellow students or a TA, was a key factor in improving student writing. It was, however, emphasised that the students needed to be given instructions as to how to conduct peer reviews. Similarly, Amores [10] and Diab [11] found that students best learn how to edit their own writing through editing each other´s written work. This may be related to what Zamel [12] describes as a tendency for lecturers to make comments on writing based on a single standard without considering the effects that these constraints can have on writing performance. A common factor in these studies is that the learning outcomes are improved when the students take a more active part in the process of reporting lab results, but that this process needs to be accompanied with a frequent feedback and follow-up from the lecturer and/or other teaching staff.
Addressing the challenges related to assessing learning outcomes in the undergraduate chemistry laboratory, Galloway and Bretz [3] developed an assessment tool to measure students' expectations and experiences related to cognitive and affective parts of the learning process in the lab course. The tool was called Meaningful Learning in the Laboratory Instrument (MLLI), which consisted of a questionnaire given to the students both prior to the course (expectations) and after the course had ended (experiences). It included statements of the type "When performing experiments in my chemistry lab course this semester I expect … to learn critical thinking skills". The last part can be substituted to address different aspects of learning outcomes. The students answer on a scale from 'completely agree' to 'completely disagree'. Responses collected at the two different points in time were compared and correlated.
Handal [13] suggested to use your colleagues as 'critical friends'. Handal emphasized the importance of discussing teaching with other colleagues but points out how difficult it can be to establish good mechanisms and proper communication channels for such discussions. It was suggested to establish formal collegial groups in form of reflective teams, where it is possible to maintain a certain level of systematic consultation regarding teaching.

CHANGING THE CURRENST PRACTISE
We believe that the main drawbacks with the current series of laboratory courses are: (1) too much focus on the formal part of report writing; (2) a lack of a gradual learning scale for critical thinking and writing of reports; and (3) poor inter-communication between the different laboratory courses.
Our suggestion for necessary changes is based on research that show that learning outcomes are improved when the students is involved in self-assessment and/or perform peer review when planning and reporting lab results [4,6,8,9,11] (addressing drawback 1). However, this process needs to be accompanied with a systematic and frequent feedback from the teachers [8,11] (addressing drawback 2). Furthermore, it is crucial to establish formal collegial groups [13] in form of reflective teams of teachers, where it is possible to maintain a certain level of systematic consultation regarding teaching in the laboratory courses (addressing drawback 3).
Even though studies recommend reducing the focus on the formal lab report, it is important for the students to be able to write such reports, but in a manner that enables more reflection when it comes to communication and the purpose of the different parts of the report. We therefore suggest keeping the main focus on the formal lab report in the first semester, but with an approach that can lead to an improvement in writing and critical thinking.
It is not feasible to make immediate changes in all the lab courses or on all levels. In the following, more detailed plans are presented for a course in the beginning of the program (KJEM110, 1 st semester) and a course in the middle of the study (KJEM210, 4 th semester).
In the two first reports in KJEM110 the Introduction and Methods chapters are pre-written, and the students need to write Results, Discussion and Conclusion in full. The third lab report is written in full by the students. These three reports are assessed (approved/not approved) by the TAs. Lab report four and five are also written in full but are now graded (A-F scale) by the TAs.
Based on the suggestions and findings by Deiner et al. [7], we suggest introducing a version of the scaffold method for teaching report writing. Through a total of five reports the students will gradually be presented with a set of questions for each of the parts in IMRaD, constituting the total scaffold. In the first two reports students will see examples of how the scaffold is being applied in Introduction and Methods, while in the three final reports they will apply the scaffold themselves. Each set in the scaffold will contain questions that encourages students to think critically about the purpose of each chapter, reflect over the importance of the different parts of the chapter, and how the individual parts are related to the experiments performed in the lab. To evaluate the effect of the scaffold method, the students will randomly be divided in two groups. One group will follow the current practice, while the other will follow the scaffold method. A student evaluation will be conducted in order to investigate potential differences and improvements in learning outcomes.
The current practise in KJEM210 includes a total of three lab reports. The first lab report is based on a numerical exercise that the students perform individually. The second and third lab report involves reporting experimental data obtained in the lab, according to the IMRaD-structure established by the collegial discussion group. This practise can be improved by implementing a more iterative, collaborative and reflective approach to report writing and critical thinking. We base our suggestions on the theories and studies described above [4,6,8,9,11]. The idea is to keep the first report as a formal report, which the students write and submit individually, and which is assessed by the TA. With regards to the second and third lab reports, which are based on experiments performed in the lab, a revised approach to lab report writing will be adapted.
In the lab experiments the students work in pairs. After each of the lab experiments, the students first submit an individual lab report. These reports are subject to peer review by students that are not lab partners. Prior to this, the students will have been given instructions and in-class assignments to familiarise themselves with the process of peer-review. After the peer review process, the students meet with their laboratory partner to discuss their individual reports and the feedback they have been given. With two individual reports, including feedback from students outside of the group, the lab partners then arrive at a final collaborative lab report which is submitted and assessed by the TA. In addition, a reflective text, where the students describe the process of arriving at the final report, should be submitted individually. At the end of the semester, a student evaluation will be conducted in order to contrast and compare learning outcomes and experiences from the two strategies for lab report writing, and if necessary, adjust the strategy accordingly.
To evaluate the fulfilment of the learning outcomes on a longer time scale we plan to include a modified version of the MLLI tool [3] into the two above-mentioned lab courses. Students learning experiences through the course can then be evaluated and necessary modifications can me made.
Above, changes in the current practice were given for two of the five mandatory lab courses in the program. Experiences and data from the evaluations of these changes should be used as a fundament to evaluate the current practice in the other courses. To facilitate such a process, the teaching staff at each lab course need to communicate and coordinate in a structured manner. University teaching tends to be more "personal", and less subject to peer-review, than research. While sharing the research is part of the university culture, this is not the case when it comes to sharing our teaching methods and philosophy. However, the ongoing restructuring of teaching from the auditorium to digital platforms due to the covid-19 outbreak has resulted in more discussions of teaching practices and ideas among the teaching staff at our department. This is a promising start which we will try to capitalize on to establish different collegial groups for teachers at the department. As a first step we aim at establishing a weekly or monthly "teacher´s coffee" which focus on a specific topic related to the teaching practice at the department. Later on, specialized groups, dedicated to certain sub-topics, will be established.
This gradual and consistent introduction of learning outcomes related to writing and critical thinking will help students to achieve enhanced and deeper learning. The proposed changes in the two lab courses will be carefully evaluated. In a second, follow-up paper we plan to present data from these evaluations.