Because of the pandemic, the blended learning model with a mix of online and on campus provision was adopted as the main teaching and learning approach for the first term in Sept 2020 in the UK HE sector (UOG, 2020). Although there are some surveys (e.g., Lapworth (2020)) which investigate the impact of blended learning on student satisfaction at the sector and university levels, there is no evidence at the module level for specific programmes. I use the results of the Module Evaluation Surveys (EvaSys) for one MSc programme suite at one UK University to provide some latest evidence on the impact of blended learning on student satisfaction at the programme level, and I find that although overall student satisfaction is high, it is necessary to improve the EvaSys measures so that they can more accurately reflect student satisfaction of the blended learning model.
Existing evidence indicates that students, in general, are satisfied with blended learning. For example, Lapworth (2020) finds that 51 per cent of students were satisfied with the quality of their teaching during the pandemic. However, the evidence on the impact of different elements of blended learning on student satisfaction is mixed. After I looked at the results of EvaSys, I find that students, overall, are satisfied with module deliveries, and especially happy with pre-recorded lectures. However, they are less satisfied with the way that modules have been adapted to blended learning and the online learning platforms. I find that this divergence could be because that the overall student satisfaction measure in the EvaSys may be dominated by several elements of student learning experience. Therefore, I propose to use the equal-weighted average student satisfaction score (ewaverage) because it could capture the impact of different factors on student satisfaction better.
This blog will take you through my data and methods that I used to derive my results. I will share with my ideas on the findings which I think are very interesting and important to improve student satisfaction by using Evasys results and improving the Evasys itself.
1 Basic information about the research of the blog
I use a sample of 10 EvaSys of 2020/21 Term 1 and 10 EvaSys of 2020/21 Term 2 for one MSc programme suite at one UK University. I choose this sample because in general, MSc programmes have more international students and blended learning model could have more impact on the learning experience of these students. I examine all elements related to blended learning models and some other main elements in the EvaSys. I compare the average scores of these elements of EvaSys, and their correlations with the overall student satisfaction score. I also use regression models to examine the relative weights of these elements of EvaSys in explaining the overall average student satisfaction score.
2 Overall satisfaction is high
I find that overall student satisfaction is high because the average for Term 1 is 4.13 and Term 2 is 4.29 respectively. I find that there are several factors which could have possibly enhanced student satisfaction. For example, students are much happier in Term 2 about learning outcomes and learning resources, and blended learning. In terms of facilities, students are very happy with lecture recordings with the average score for lecture increasing from 4.27 in Term 1 to 4.33 in Term 2. Moreover, students are happy with Teams and Moodle with the average scores being above 4 in both Terms.
I also find that overall student satisfaction has high correlations with seven factors and their correlation coefficients are over 70%. However, it does not have high correlations with other elements.
In addition, I notice that there are high correlations between elements in the Evasys. The correlations between module organization, student experience, subject taught by staff, learning outcomes and blended learning are very high. I think it may be better to use some other factors which are not highly correlated in the EvaSys so that the EvaSys can reflect more diversified student experiences.
3 But what is the specific impact of blended learning?
I am curious about the specific impact of blended learning on student satisfaction. So, I tried regression models to find the answer. I use overall student satisfaction (average) as the dependent variable and all factors as the independent variable. I find that only assessments criteria, learning resources and lecture recordings report statistically significant coefficients at the 10% significance level. All other factors are not significant and some report negative coefficients which are difficult to explain. Therefore, I argue that it is difficult to evaluate the impact of blended learning on student satisfaction and the main reason is the high correlation between some factors as discussed above.
My results suggest that many variables do not have predictive power for overall student satisfaction although they have positive correlations with overall student satisfaction. It is possible that average does not reflect the diversified student experiences in different aspects. To improve the measure for overall student satisfaction, I suggest use ewaverage. In general, ewaverage has much higher correlation with most elements than average, and its correlation with average is 90.35%. Moreover, when ewaverage is used as the measure for overall student satisfaction in the regression model which include all factors, the results are much better.
I find that it is also important to minimize the correlation between different factors because this can improve the effectiveness of using ewaverage as the measure for student satisfaction. For example, student experience and module organization have very high correlations with subject taught by staff and several other factors. After excluding both module organization and student experience from the independent factors, ewaverage can be properly explained by the regression model. Moreover, when overall student satisfaction is regressed on student experience, the coefficient is 0.97 (p-value is 0), however, when ewaverage is regressed on student experience, the coefficient is 0.68 (p-value is 0). This indicates that student experience dominates overall student satisfaction which suggests that ewaverage is better than overall student satisfaction in capturing the other factors of student satisfactions.
4 To correct the weaknesses of Evasys
I find that overall student satisfaction is at high levels for both terms of the 2020/21 academic year and blended learning did not reduce overall student satisfaction. But it is difficult to identify the specific impact of blended learning. I argue that this is because of the weaknesses of the current Evasys.
I think we can improve the current Evasys in three aspects. First, the overall student satisfaction measure used in the Evasys may not reflect all aspects of student experience, and it could be replaced by ewaverage. Second, the questions in the Evasys can be improved to reduce the high correlations between answers to these questions so that more diversified aspects of student satisfaction can be reflected in the Evasys. Third, some questions in the Evasys, such as module organization and student experience, can be improved to reduce their high correlations with overall student satisfaction if overall student satisfaction is used as the key measure for final performance reporting.
I argue that using ewaverage is better than using the overall student satisfaction measure for both lecturers and students to evaluate student learning experience. First, ewaverage summarizes all the factors that reflect different aspects of student learning experience. Second, ewaverage can be improved if lecturers improve all the factors in the Evasys because each factor has equal weight in computing the score. However, although overall student satisfaction is impacted by all the factors in the Evasys, it can be significantly affected by some factors and may also be affected by some random factors which are not identified in the Evasys.
My findings are based on one sample from one UK university due to availability of data. But I think they propose new ways for the HE sector to evaluate the student satisfaction surveys more effectively. In addition, I think my methodology can be applied to the NSS which applies similar overall student satisfaction measure, and more meaningful results could be found on both programme and university levels, which will be more relevant for improving the student satisfaction in the HE sector.
Blog Author
Dr Guoxiang Song
Senior Lecturer
Department of Accounting and Finance
Greenwich Business School
sg63@gre.ac.uk
References
Lapworth, S. (2020). What students are telling us about learning during lockdown. Available at: What students are telling us about learning during lockdown – Office for Students. (Assessed 30/05/2021).
UOG. (2020). Initial guidance on adapting programmes for blended learning for term 1 of 2020/21. Available at: Guidance & Resources: Adjusting to Blended Learning Environment | Articles | University of Greenwich (Assessed 30/05/2021).