ICMS takes Hybrid Learning Insights to HERDSA Conference

Data collected from a survey of ICMS students enrolled in synchronous hybrid learning classes over a two year period revealed that satisfaction with the implementation of blended or mixed-mode teaching increased over the period under review.

This is according to research conducted by an ICMS academic team lead by ICMS DVC (Learning and Teaching) Dr Heidi Le Sueur.

The findings, and ongoing research, were presented at the HERDSA Conference in Brisbane in July, 2023. HERDSA brings together academics and professionals from Australia’s tertiary sector for engaging discussions on a wide range of topics related to teaching, learning and the student experience.

Presenting: Synchronous Hybrid learning: Analysis of students’ experience and their educational outcomes

Context

Over the past few decades, the increased interest in, and demand for, synchronous hybrid learning, which combines in-person and online learning experiences, led to an increase in the implementation of so called blended, or mixed-mode teaching (Spanjers et al., 2015).

“The restrictive learning and teaching conditions of the Covid-19 pandemic accelerated the development and rollout of online or hybrid learning (a combination of online and on-campus) synchronous learning across the Higher Education sector, but ICMS ensured a solid framework was in place”, explains Dr Heidi le Sueur.

In the two years since ICMS has been delivering hybrid classes, over 500 surveys were submitted from students – between Term 2 in 2021 to Term 1 in 2023 – which gave the ICMS academic team, as well as the learning, teaching and innovation team, a measure of how the students experienced hybrid learning.

Improvements could be made in response to the feedback on an ongoing basis as it was received each term, and the hybrid mode will continue to be a mode of delivery at ICMS going forward.

Practice

To ensure the successful implementation of synchronous hybrid learning at ICMS and to address both the concerns and opportunities, several pedagogical strategies, practices and improvements were put in place based on Beatty’s (2019) four core values: Learner Choice, Equivalence, Reusability, and Accessibility.

These included regular hybrid teaching training for academic staff, the introduction of online engagement and collaboration tools, installation of appropriate classroom technology, and tailored teacher support during terms in response to student feedback obtained from mid-term evaluations.

“The role of the co-pilot to support the main lecturer in the hybrid class delivery has offered a high-touch model for our students, enhancing the learning experience”, states Head of Learning, Teaching and Innovation, Jonathan Hvaal.

Methods of evaluative data collection and analysis

The synchronous hybrid delivery mode needs more empirical investigation to complement existing qualitative studies on the impact of synchronous hybrid learning on student experiences, engagement, and outcomes (Raes et al. 2015).

To address this gap, a 16-item online survey was administered over two years (Term 2 in 2021 to Term 1 in 2023), at each trimester term to students enrolled in 166 hybrid classes delivered at the institution. In total 515 surveys were completed by students over the period who had enrolled in hybrid classes. The average class size was 22 students. Student educational outcomes with regards to grade distribution and fail rates were monitored and analysed at the end of each term over this period, and evaluated in this study.

Evidence of outcomes and effectiveness

The findings of the survey results showed an increase in student satisfaction with synchronous hybrid learning over time as the number of classes increased across two years, suggesting the positive impact of the several improvements that were initiated over the two years.

Findings further indicate the significant opportunities the delivery mode provides to students, based on the majority of respondents agreeing or strongly agreeing with the importance of learner choice for the mode of learning and that they felt strongly engaged and included in the learning experience.

Student Feedback

Despite concerns raised by regulatory bodies such as TEQSA (2022) about the impact of mixed mode teaching on student engagement and support, these modes have been found by ICMS students to offer numerous benefits such as increased flexibility for students and several organisational and pedagogical benefits, confirming findings from other researchers (Raes et al., 2020).

In the following areas summarised across the two years, it was found:

• Learner Choice: Students agreed (84%) that it was important for them to be able to choose whether they learned online or face to face.
• Reusability and Learner Engagement: 76% found class activities to have been engaging in content and in the use of different online educational technologies. 77% felt included and engaged in the learning and teaching happening in the physical classroom, albeit interacting from an online platform.
• Equivalency: 82% felt the teaching was equal in terms of attention given to virtual students and face to face students.
• Accessibility: 86% were certain that if they missed a class, they would be able to access the recording and engage in the activities later.

While ICMS received affirmation for hybrid learning, there were also critical comments from students. Students found that group work between students in-class and online was challenging with technology creating a barrier.

It was furthermore queried whether there was an even balance of learning and inclusion between the modes of attendance, and it was suggested to find more ways to interact with students attending online.

Educational Outcomes: hybrid classes compared with on-campus and online classes

To analyse educational outcomes, final subject grades across all grade bands (from high distinctions to fail grades) were analysed and compared between three modes of delivery: hybrid, on-campus and online.

“The large scale and longitudinal data provided the team with trend-data for analysis for each trimester over these two years,” confirmed Dr Stuart Wiggins (PVC Employability and Registrar).

Over the two years of study, which included six teaching terms, students that enrolled in classes on campus achieved on average higher combined percentages in higher distinction and distinction grades than their counterparts in hybrid classes.

Further, the fail rates recorded in hybrid classes were slightly elevated compared with on-campus or online delivered classes in four out of the six terms.

What do these findings mean for ICMS as an institution and what is next?

As an immediate response, the delivery mode for classes will be reviewed with a planned increase in on-campus classes offered to students. A selection of subjects will also continue to be available exclusively online.

The hybrid delivery mode will continue to be offered to subjects which are offered once per term/year to cater for both the online and on-campus students based on demand. This typically applies to specialisation subjects and 2nd or 3rd year subjects.

The learner engagement during hybrid class delivery is believed to be a critical aspect for the success of students and therefore teaching staff will deploy inclusive active learning practices using the REAL framework, creating a sense of belonging and stimulate learner motivation.

Dr Le Sueur recognises that, “the findings from this study about student satisfaction and educational outcomes are not conclusive and the institution will continue to monitor and analyse the various qualities of the hybrid delivery mode to inform the ICMS delivery strategy.”

“I thank all our Faculty, Program Managers, Deans and the IT operations team, who have believed in the hybrid mode of delivery at the institution and actively supported ongoing improvements to meet students’ expectations,” concludes Le Sueur.

Article on active learning at ICMS: https://www.icms.edu.au/news/academic/active-learning-under-the-spotlight-at-icms-faculty-day/

Hybrid teaching guidelines: https://policies.icms.edu.au/hybrid-learning-and-teaching-guidelines/

References:

Beatty, B. (2019). Hybrid-flexible course design. London, UK: EdTech Books.

Bower, M., Dalgarno, B., Kennedy, G. E., Lee, M. J. W., & Kenney, J. (2015). Design and implementation factors in blended synchronous learning environments: Outcomes from a cross-case analysis. Computers & Education, 86, 1–17.

Eggers, J. H., Oostdam, R., & Voogt, J. (2021). Self-regulation strategies in blended learning environments in higher education: A systematic review. Australasian Journal of Educational Technology, 37(6), 175-192.

Raes, A., Detienne, L., Windey, I., & Depaepe, F. (2020). A systematic literature review on synchronous hybrid learning: gaps identified. Learning Environments Research, 23, 269-290.

Spanjers, I., Könings, K., Leppink, J., Verstegen, D., de Jong, N., Czabanowska, K., & van Merriënboer,

74. (2015). The promised land of blended learning: Quizzes as a moderator. Educational Research Review, 15, 59–74.

Tertiary Education Quality and Standards Agency (TEQSA), (2022). Teaching and learning.  https://www.teqsa.gov.au/guides-resources/online-learning-good-practice/teaching-and-learning

Tertiary Education Quality and Standards Agency (TEQSA), (2022). Modes of Delivery report. https://www.education.gov.au/higher-education-standards-panel-hesp/resources/modes-delivery-report