Helen Coker, University of the Highlands and Islands
The development of online modules, when informed by feedback and reflection, can lead to better student engagement. In this case study, the online space of a teacher training course was developed through utilisation of a critically enquiring approach. Data, feedback gathered and theoretical literature were drawn on. Changes made to the structure of the space led to an increase in student engagement in the following iteration.
A critically enquiring approach involves the use of evidence gathered from practice, theoretical reflections and professional dialogue (Sinnema, 2011). This facilitates enquiry into learning and teaching, and critical reflection on the factors which enable and constrain student participation. While the online space cannot be observed in the same way as a physical environment – one cannot literally see what students are doing, observations can be made using the data collected as students participate. In this case study, data were gathered from online modules within the University’s Virtual Learning Environment (VLE). Data from online spaces are being increasingly used for business, economics, healthcare and government (Bollier, 2010). Learning analytics have made use of tools utilised in the business sector and applied them to the improvement of learning and teaching (Elias, 2011). This has been found to benefit both learners and teachers (Drachsler & Greller, 2012). Data gathered are processed and analysed; the results are then applied to the learning environment (Elias, 2011). This information from online spaces can provide tutors with participation rates and student contributions to different areas of their modules.
The course reports used in this study provided observations which were used to inform future developments. The data were approached by taking an ethnographic stance. It provided observations of student activity within the online space: vignettes of participation. These were critically reflected on to inform developments.
This case study presents the development of the VLE for a blended learning, taught, full-time course, delivered to a distributed cohort of postgraduate students. Collaborative activities and discussion were an important aspect of the VLE through which the theoretical aspects of the course were delivered.
The context was a two-year teacher training course delivered through joint provision between local authorities and the University. During the first year, students were based in the University attending placements in local schools. This was the year in which the VLE was used. Following this, the second year consisted of classroom teaching supported by the local authority. At the end of the two years, full registration as a teacher was attained through meeting the General Teaching Council for Scotland (GTCS) teaching standards. The initial delivery of the course took place in the academic year 2013/14. Twenty students were split across four geographically disparate Academic Partners (APs). The blended structure included a face-to-face residential in August 2013. The students came back together at a closing residential in June 2014. For the rest of this year, students worked online in the college VLE and through video conferencing (VC) as well as taking part in local sessions and placements.
The initial VLE design was theoretically informed. Social, cultural and pedagogic factors influence the use of technology in education (Creanor, 2013). Being based in the Education department, the pedagogic approach was an important consideration. Meaning was seen to be negotiated (Bruner, 1996; Wenger, 1998) through social interactions in a cultural landscape (Cole, 1998). The online space provided a context for learning but did not determine it (Oliver, 2013). The technology itself was viewed as a mediating artefact (Conole, 2013), enabling and constraining students’ participation. This theoretical philosophy was congruous with the current philosophy of the school-based curriculum in Scotland at the time. The VLE spaces were designed to enable students to actively engage with content, reflect on their learning and interact with each other; they modelled the pedagogy they taught.
The course was designed to enable students to enter the community of practice (Wenger, 1998) of teaching. Learning to become (Thomas & Seely-Brown, 2009) a teacher was seen to require more than just the acquisition of information. The course aimed to enable students to enter the professional community of practice; to develop their identity as a teacher. This required access to particular social capital – expertise and resources (Penuel, Riel, Krause, & Frank, 2009). Students needed to use the knowledge available to them in ways particular to the community of practice they were entering. Participation was important. Emotion and cognition are inextricably linked (Immordino-Yang & Damasio, 2007). The social, emotional and cognitive aspects of learning were considered. Students were encouraged to actively participate and collaborate; to think (and feel) like teachers. VLE sessions were therefore designed around a template which emphasised participation (Sfard, 1998). Information was presented through a range of media, and peer collaboration was a regular part of each session. To encourage students to reflect on their learning and act on that reflection (Freire, 1996) the GTCS (General Teaching Council for Scotland) teaching standards were embedded across all VLE sessions and linked directly to the students’ e-portfolios.
Two platforms were utilised: Mahara – an e-Portfolio platform and Blackboard – the University’s VLE.
Figure 1 Design of the online space 2013/14
The methodological approach applied to this case study was interpretative (Cohen, Manion, & Morrison, 2013) and sought to observe and understand students’ participation rather than measure it. As opposed to finding a hypothesis, and testing it, the case study aimed to document the way in which students participated in the VC and the way in which the VLE mediated that participation. This enabled the module team to take a critically enquiring approach (Sinnema, 2011). Feedback from VLE sessions, students and staff added further layers of evidence, which enabled critical reflection and informed developments. Research and the experience of colleagues were also drawn on. In this respect, the methodology could be described as critical (Freire, 1996), as a central motive was change.
Essentially, the approach to the data was ethnographic: the data collection and analysis were conceived of as digital fieldwork – observation of the participation and interaction of people in the online space. Fieldwork has been described as a central tenet of the ethnographic approach (Hammersley & Atkinson, 2007). Traditionally, the role of the researcher was to immerse oneself in a culture by being there physically. Online ethnographers immerse themselves in virtual worlds, not by being there physically but by being there virtually (Mills & Morton, 2013). The data drawn upon for this case study enabled observation of action within the online space, and the researcher’s role as a tutor on the course added an auto-ethnographic aspect (Maanen, 1988).
Ethnographic studies have used GPS data to observe people’s movements in physical spaces (Christensen, Kraftl, Horton, & Hadfield-Hill, 2014). Further layers of evidence in the form of participant observation and interviews develop a rich picture of people’s use of physical spaces. This case study took a similar approach, using data from the VLE to observe students’ online movements and interactions. Layers, in the form of session feedback, collaborative student feedback and tutors experience, were used to create a detailed observation of the online space.
Data were collected from VLE relating to the students’ participation within sessions and on discussion boards. The data collected were from sessions held between 26 August 2013 and 4 April 2014. Data were not collected during VC sessions or from Mahara. Data were compiled and collated from course reports (a function of the VLE). They were presented in a series of graphs which visualised the participation of students in the online space. The data formed part of the researcher’s doctoral studies, for which ethical consent had been gained. All data were stored securely, and any output anonymised the names of students and tutors involved.
Feedback was gathered during the course from short surveys at the end of each session. A formal review session in December gathered overall individual and group feedback from students. External examiner reports, the experience of tutors and course results were all drawn on during the critical enquiry. Any reference made to these in external outputs was anonymised and the documentation was stored securely. The evidence collected was only used by the course team to develop the course; data drawn from the online space through Course Reports had no influence on the students’ final results.
The course reports generated a number of data sets. These were explored to observe students’ patterns of participation: the time spent online and places students visited within the online space.
Data extracted showing the days of the week students accessed Blackboard were collated.
Figure 2 Access by days of the week
The data suggested that students spent more time in the VLE on Mondays and Tuesdays. This was when their timetabled sessions normally took place. However, access on all other days and at the weekend was common. While some of the access on other days could be accounted for by VC sessions (accessing resources and giving feedback), it could also suggest that students were not always following the timetable of Monday and Tuesday VLE engagement. They may well have been utilising the potential of the flexible working which online resources offered. The information gathered thus suggested that the design of the platform, which utilised a session-based approach, was not necessarily reflected in the students’ participation.
Data relating to the hours of the day in which students logged on added another layer to the data analysis.
Figure 3 Access by hours in the day
The data in the chart depicted in Figure 3 showed that although students were most active during the day, they logged into Blackboard in the evening as well. This further supported the inference that students were not sticking to the session-based approach presented by the platform. The quantitative data had begun to build a picture of the patterns of participation of students. However, as these data averaged out the participation of all students, they were not representative of any particular individual.
Students were then sorted in groups as they were located across the network. The four Academic Partners (AP) each had a different number of students: AP 1 had the largest group with nine students, AP 2 had four students, AP 3 had five students and AP 4 two students. The amount of time students spent online was compared with the number of content area and discussion board hits.
Figure 4 Time online sorted by AP
Figure 5 Discussion board hits sorted by AP
Figure 6 Content area hits sorted by AP
Contrasting patterns of participation were found across the APs. The biggest difference was between AP 1 and AP 4, suggesting the possibility that the number of students in an AP may contribute to the level of participation in the VLE. In AP 4, where there were only two students, the time spent online, content area and discussion board hits were all high. In AP 1, where there were nine students, time online, content area and discussion board hits were noticeably lower across all nine students. Taking the community of practice approach (Wenger, 1998), this could be reflective of the access to a face-to-face community for students in AP 1: Students in AP 1 spent VC sessions and workshops in a group of nine students, whereas in AP 4 face-to-face opportunities were with only one other student. For students in AP 4, access to the academic community – access to their peers – was through the online platforms, both VC and VLE. However, this could also have been a reflection of the small numbers, and the two students in AP 4 could both have engaged with the VLE for other reasons. This was less likely for the AP 1 group, where all nine students were below average for the course in terms of participation in the VLE. This suggested that the social situation of students may have influenced their use of the VLE.
Patterns of participation across the year appeared to be similar for all students in relation to their overall participation. The analysis above suggested that the social (AP) context in which students were based influenced their participation online. However, the fact that patterns of participation across the year were comparative for all students suggested that the VLE itself also played a role in mediating participation.
Figure 7 Time online for all students
All students were more and less active at similar times during the year. This suggested that the Blackboard sessions may have been more and less successful, in terms of how they engaged students, at different times. The periods of high and low engagement did not appear to correlate with holidays or assignments – external factors which could have been expected to influence participation.
The course discussion boards provided a means for all students to interact with each other. In the year the case study took place, discussion boards were used as a student space; tutors fed into VC sessions and interacted with students in face-to-face contexts and through feedback on assignments and the ePortfolio. Observation of the discussion board posts and hits, and content area hits, highlighted a correlation between participation and achievement of students. High achieving students (as identified by course tutors) spent a more equal amount of time and had a more equal amount of hits in the content area and on the discussion board. While the frequency of hits did seem to be a factor, it was the relationship between the two which correlated most noticeably with achievement.
Figure 8 Content area and discussion board hits sorted by student
Figure 9 Relationship (DB/CA)*100) between discussion board and content area hits sorted by student
Student 7 was identified as working at Masters Level (the course sat at SCQF – Scottish Credit and Qualifications framework – level 10; Masters courses in Scotland sit at SCQF level 11). Students 5, 12 and 19 submitted strong assignments. Student 1 resubmitted the first written assignment. Students 4, 9, 11 and 17 resubmitted the second written assignment. The relationship between content area and discussion board hits, and the high hit ratio for able students on discussion boards, suggested that particular patterns of participation related to course success. The data were not able to show whether this reflected more able students’ work patterns or enabled success. The successful students were identified by their work on the course. This did not correlate directly with previous qualifications. This might imply that the pattern of participation enabled success rather than reflected able students. However, the data set was small and the relationship between what enables and what reflects ability is likely to be a complicated one. If the pattern was reflected in a much larger data set, the implication could be more thoroughly investigated.
The correlation between able students and greater movement around the VLE and within the discussion boards suggested that those students may have had greater access to the social capital available in the VLE (Penuel et al. 2009). The high hit rates of able students on discussion boards (which didn’t relate directly to the number of posts) suggested that they were reading more posts; in doing so, students were able to access the expertise of their peers. They were more likely to have become aware of resources used by others, thus being more likely to access them in turn. Their pattern of engagement was more dialogic. Rather than simply posting on the board and leaving, movement around the board suggested that high achieving students were reading as well as writing, actively participating in dialogic practices (Cunliffe, 2012).
A comparison of the discussion boards suggested that some were more successful than others in engaging participation and interaction: data relating to the number of participants on each discussion board were used to compare discussion boards. The largest participation was seventeen students and the lowest zero students. Boards with no students were removed from the data.
Figure 10 Number of students participating in discussion boards from November 2013 to April 2014 (sorted by participants – high to low)
Each discussion board had an associated task which was located within a particular VLE session. The top five and bottom five discussion boards were compared in regard to tasks set. All the tasks asked students to reflect on the content of the session and relate this to either their own experience or their own practice. Out of the top five boards, three asked students specifically to comment on others posts. The top board stated that comments would be reviewed before the ‘collaborate’ session (an interactive seminar session in Blackboard, used on one occasion as a trial) where they would be discussed. Of the bottom five boards, none of them asked students specifically to comment on others’ posts or stated that the posts would be used. Three boards from the bottom five asked students to post their experiences or reflections. One asked them to use the board to “share and explore”. Students were encouraged to “borrow” from others and “share” but were not asked specifically to comment on other posts. One board in the bottom five asked students to “have a look” at others’ posts and discuss differences.
The data provided a layer of observation; it enabled vignettes of participation to be observed. It showed the course team what students were doing in the online space.
The social nature of learning within the online space of the course was observed: students participated with the knowledge they had access to through the VLE on the social plane (Vygotsky 1987; Rogoff, 1995) as they negotiated the meaning (Wenger, 1998) presented by others. Those who made most use of this opportunity had access to the most social capital (Penuel et al., 2009) and were able to use this to develop the depth of their understanding (as demonstrated in written assignments). This can be seen through the relationship between content area and discussion board hits for higher achieving students. The patterns of participation of high achieving students reflected a dialogic approach (Cunliffe, 2012). They spent a more equal proportion of time between the two spaces: the content area enabled engagement with the content of the course, while the discussion boards enabled interaction with other students. By engaging with both areas, students had most access to the developing understanding of others and the social nature of learning was more salient. Learning can be described as a social practice in which understanding is negotiated on the intermental plane (Haste, 1999) before it is internalised onto the intramental plane (Haste, 1999). The process of understanding something involves negotiation of meaning (Wenger, 1998) and contextualisation in relation to what is already known (Greenfield, 2011). For students, the content area introduced them to new concepts and knowledge. This was negotiated individually through reading, watching or listening as well as activities set for them to work through. They related new concepts and knowledge to what they already knew. The discussion boards provided a space for interaction where students could see part of this process in other students. Those students who made use of the discussion boards were more exposed to the mutuality of others, mutuality referring to the awareness of others in the process of negotiating meaning (Wenger, 1998): in the process of participating with someone else, one experiences the other’s ability to negotiate meaning or construct understanding. The awareness of someone else in the process makes participation more explicit and makes salient the human aspect, the social experience, of meaning making. In reading another student’s interpretation, or opinion, mutuality was visible. Students compared their own understanding with that of others. Those who read the most posts, of others, would have had the most opportunity to compare, and deepen, their own understanding. The correlation between students spending time on discussion boards, potentially reading as well as posting, and high achievement in written assignments suggests that this may have happened in the online space.
The online space acted as a mediating artefact in shaping students’ experience of, and interaction with, the world – explicitly and implicitly (Vygotsky, 1987). In the comparison between the discussion boards, we can see how the tasks set differed in their success (in relation to student participation on the boards). Looking at the language used to set these tasks, it is possible to infer that the pedagogy of tutors may have implicitly mediated participation. The tasks set for the boards which were most utilised explicitly valued the collaborative nature and potential for two-way dialogue of the discussion board. Through using the asynchronous discussion board activity to inform a subsequent synchronous discussion-based activity, the discussion board with the highest number of hits was the most explicit in valuing collaboration. The tasks set for discussion boards with low postings used language which may have implicitly devalued interaction and collaboration simply by not emphasising it.
Students’ engagement with the VLE shaped the VLE itself just as the VLE shaped students’ participation: “The mountain only guides and the river only carves but in the process the carving becomes the guiding and the guiding becomes the carving” (Wenger, 1998, p. 71). For students who did not spend much time in the online space, their experience of the course content, of new skills and knowledge was limited. Those who utilised the online space had most access to knowledge and most opportunities for knowledge building. The “landscape of practice” (Wenger, 2009a) experienced by the students altered depending on their movement around the space. This might account for why some boards were not used at all, as arriving at an empty space students simply moved on.
Teaching can be seen as a community of practice (Wenger, 2009b); a professional community into which we are introducing our students. Being a community of practice, it will change over time through the participation of its members. The process is one of change and resistance, as seen in New Zealand; a report from the New Zealand Ministry of Education (Sinnema, 2011) documented the resistance to change found in New Zealand teachers when a new curriculum was introduced. Teachers were found to assimilate aspects of the new curriculum into their existing practice rather than make a real change. The recent curriculum changes in Scotland have been found to be negotiated in similar ways (Priestly & Sinnema, 2014); in both cases, the new curriculum models are being used and the community of practice is evolving. The course introduced students to the teaching ‘community of practice’ in Scotland. Part of this process happened in their school placements, but it is likely that the VLE played a role.
Social presence research (Cui, Lockee, & Meng, 2012; Kehrwald, 2008; Slagter van Tyron & Bishop, 2009, 2012) has suggested it is of benefit for students to be aware of one another in the online space. Being aware of the other participants as people is more likely to support the development of a community. Movement around the discussion boards may well have contributed to students’ awareness of others. The two students based in AP 4 had formed relationships and found their place in the wider cohort (demonstrated through interactions at the final face-to-face residential in June); both of these students were active on discussion boards. As the course was blended and students saw each other regularly on VC and at the residential at the beginning of the programme, the VLE was potentially not as central as it would have been on a fully online course, but it is likely that it played a role in the development of the student community. Larkin (2009) described writing as a culturally situated metacognitive practice. When writing on discussion boards, students were writing in a particular way, using the ‘voices’ (Wertsch, 1991) of the community of practice in which the course was situated and into which the students were moving: postgraduate study and primary teaching. In writing comments on discussion boards, students were required to reflect on their thinking (metacognition) and their use of the academic voice (written language) in order to communicate. This was different to the verbal communication of VCs, as writing can be seen as a more reflective practice (Larkin, 2009). In reading the thoughts of their peers, students were able to situate their own understanding in the community of their peer group. Students were able to compare their own understanding with that of others and put it in the context of what they already knew (Greenfield, 2011). This meant that students were able to gain feedback on their own thinking and to further develop their identity as a primary teacher by aligning it with their peers. Those students who spent more time in the VLE engaged with a wider variety of ways to develop their identity, to understand the community of practice they were entering and to understand the values and beliefs which shaped it. This is reflected by the students who spent more time in the VLE doing well on placement and in assignments, written and oral. They utilised the additional space provided to support the development of their professional identity. Following up on the students’ progress as they move into the professional community may shed light on the extent to which this was enabled.
This case study was used as part of a review which informed the development of the structure of the online space for the programme. Taking a critically enquiring approach to module development, the course team used evidence gathered over the 2013/14 iteration of the course to inform the design and delivery of subsequent iterations. Feedback was collected from students through regular short surveys embedded into VLE sessions and two structured VC evaluative sessions. Feedback from staff also informed the review as did the work produced by students for their e-portfolio, their academic assignments and their placement practice.
The staff used this evidence to engage with the needs of the students. Changes were made during the year based on the first formal feedback session and ongoing feedback. At the end of the year, when all the feedback was collated, it was used to reflect on the student experience. Through engagement with this evidence, with research and with literature as well as drawing on the experience of colleagues, the VLE structure was developed.
Reflecting on the pattern of participation of the students in regard to their time online, the session-based nature of the VLEs was reflected on. At the beginning of the next year, the same structure was used, as it was felt that it helped scaffold students’ engagement with the VLE. As the year went on, however, the timetable became less session-based; VLE sessions were presented in day or two-day chunks so that students could plan their own working patterns.
The differences in engagement across the four cohorts and the links between achievement and patterns of participation led to the re-organisation of the Blackboard modules. In order to open up additional spaces to enable dialogue, seminar sessions were introduced in Blackboard. Blackboard itself was reorganised, making use of one module space rather than four. Being taught holistically, this enabled a thematic organisation of content which was much more congruous with the underlying approach. Students were required to meet the GTCS standards for provisional registration at the end of the university based year. These had been embedded throughout the course through the e-portfolio. Thematic organisation of content meant that they could also be used to structure the space. The standards were taken to represent the narrative template of teaching in Scotland (Bakhtin, 1981; Philpott, 2014). By organising the structure of the space, they were able to explicitly shape as well as implicitly permeate the structure, the narrative template having been present throughout in the values and beliefs of the professional community which the VLE introduced. This enabled the ‘voice’ of the professional community to be more explicit to the students (Wertsch, 1991).
Figure 11 Online space 2014/15
A single discussion board was used each week. As participation appeared to correlate with academic achievement, students’ use of the discussion boards was supported in a number of ways. The tasks for the boards, particularly those in the first few weeks of the year, were reviewed and developed: students were asked explicitly to read and comment on the posts of others. Discussion boards were also moderated by tutors and referred to directly in VC sessions. Participation in discussion boards increased: In the first two-week teaching block, 100% of the students contributed to the discussion boards.
Engagement with the online space was found to relate to academic achievement in regard to essay submissions. The use of one Blackboard space, rather than four, aimed to ease access and reduce constraints on participation. The space was organised so that students could locate previous content more easily, thus enabling autonomy and independent learning.
Seminar sessions were introduced with cross-AP groupings. This enabled all the students on the course to work in small discussion groups. Seminars were compulsory and were placed within the online space, thus requiring students to utilise it. Collaborative activities were introduced; each college block a collaborative task was set where students would work together in their seminar groups on a shared project before sharing it with the cohort. This activity aimed to scaffold the development of critically enquiring practitioners who would develop practice through both engagement with theory and new initiatives, and professional dialogue with others in their community of practice.
Feedback from tutors at the end of semester one suggested that more depth of learning had been enabled through the development of the VLE.
Helen Coker is based in Inverness College, University of the Highlands and Islands. She is Deputy Programme Leader: Masters of Education. She lectures in Education and is currently studying for a PhD exploring collaboration in online environments.