Rick Hayman, Northumbria University, UK
The merits of timely, honest, respectful, insightful, understandable, personalised, memorable and balanced assessment feedback are well documented as powerful and transformational influencers upon the continued academic, professional and personal development of Higher Education (HE) students (Brown, 2015; Cann 2014; Davis & Ryder 2012; Hattie & Timperley 2007; Lizzio & Wilson 2008; Nichol 2009). Effective feedback directly addresses assessment marking criteria, indicates how well students are understanding and engaging with new material, recommends how future performance may be acted upon and improved , provides constructive criticism when necessary, plus strengthens students capacities to self-regulate their own work (Brown, 2015; Dixon, 2015; Middleton, 2011; Sambell, 2016). Previous studies have reported how HE students value, have high expectations and eagerly await summative grades and supporting comments (Brown, 2015; Higgins, Hartley, & Skelton, 2012). Yet, feedback without action in unproductive (Sambell, 2016), and it has become increasingly apparent how traditional forms of written only strategies may be failing to sufficiently engage, motivate and empower high numbers of modern day century HE learners to improve future work (Dixon 2015; Gibbs & Simpson 2004; Rotheram 2009).
To illustrate, examples from the extant literature demonstrate how they have struggled to apply received written feedback to future work and take the next steps, describing it as being overly complex, challenging to interpret and understand plus generally dispiriting, unhelpful, alienating and unsupportive (Brown, 2015; Cann, 2014; Fawcett & Oldfield, 2016; Glover & Brown, 2006; Ryan & Henderson, 2017; Scott et al., 2013; Värlander, 2008; Walker, 2009). To counter these issues, wide-ranging efforts have recently been made to persuade HE colleagues to consider alternative ways of producing, communicating and delivering increasingly engaging, constructive and valuable summative feedback which justifies grades but also respects feelings, provides personalised and authentic learning experiences, promotes self-esteem and perceived self-efficacy, plus nurtures a positive connection between themselves and their students (Bond 2009; Chalmers, MacCallum, Mowat, & Fulton, 2014; Hepplestone, Holden, Irwin, Parkin, & Thorpe, 2011; Laughton, 2013; Macgregor, Spiers, & Taylor, 2011; Rotheram 2009; Rowe, Fitness, & Wood, 2014; Warner & Miller, 2015).
Employing digital technology to create and cascade audio feedback whereby helpful praise plus areas for future development are carefully discussed and conveyed using a positive, enthusiastic and caring tone of voice, has been extensively trialed over the past decade (e.g. Chalmers et al., 2014; Fawcet & Oldfield, 2016; Laughton, 2013; Lunt & Curran, 2010; Merry & Orsmond, 2008). Across wide-ranging subject disciplines within HE, student cohorts have warmly embraced receiving audio feedback as a useful strategy for enhancing their motivation to learn, self-awareness, critical-reflection, capability to work autonomously and confidence to approach lecturers (Cann 2014; Dixon 2015; Jackson, 2012; Knauf, 2016; Lunt & Curran, 2010; Merry & Orsmond, 2008; Middleton, 2011; Rowe, 2011). It must also be acknowledged how several negative reactions towards audio feedback populate the extant literature base. For example, Munroe and Hollingworth (2014) revealed recording and editing mp3 files as barriers to creating audio feedback, whilst King, McGugan and Bunyan (2008) struggled to locate quiet locations to actually record the audio. From a student perspective, Rodway-Dyer, Knight and Dunne (2011) found first-year geography undergraduates were more likely to perceive audio feedback negatively than their second and third year peers due to its overly severe perception and Bilbro, Iluzada and Clark (2013) reported there are students who prefer their feedback in written only format.
The need for this study arose because current understanding of audio feedback effectiveness, as a tool for enhancing student engagement, morale and achievement, is based predominantly upon retrospective recall (e.g. data collected up to 6 months after audio feedback was delivered to students), employing full-time undergraduate student cohorts and descriptive quantitative data collection tools (e.g., Fawcett & Oldfield, 2016; Lunt & Oldfield, 2010; Martini & DiBattista, 2014) which may restrict the potential to fully unearth, explore and explain the meanings of obtained findings. To break new ground and redress this balance within the literature, distinctive aspects of this unique study included utilising semi-structured interviews to go beyond simplistically evaluating if participants considered audio to be ‘superior or ‘inferior’ than traditional written feedback. Therefore, the primary aim of this qualitative study was to capture work-based postgraduate sport students’ expectations, attitudes and experiences of receiving audio feedback for all summative assessments within two separate 30 credit work-based learning modules. Specifically, the three objectives of the study were to:
1. Explore participants’ initial emotional response to receiving audio feedback.
2. Unearth the general impact of using audio feedback upon satisfaction and engagement levels.
3. Establish the value of audio feedback in supporting participants to feed-forward, plus consequent impact upon professional development and work-based practice.
Employer engagement is central within all strategic plans at Northumbria University. The institution has pioneered work-based learning (WBL) provision for many years across the HE sector and has a work-based learning framework (WBLF) which enables departments to develop flexible and responsive WBL programmes and modules to meet the needs of learners within the workplace. Nationally, the Department of Sport, Exercise and Rehabilitation had been at the forefront of academic developments within work-based sport-industry provision over the past decade.
Each academic year, students attend three, two-day intensive teaching blocks, comprising seven hours delivery per day; thus, 14 hours contact per month in late September, mid November and mid March respectively. Using a blended learning approach, the part-time programme is delivered using a combination of face-to-face block-delivery including seminars, tutorials and discussion groups, plus additional on-line interaction support, by means of skype and blackboard collaborate.
Part-time students undertake two coaching-specific modules in year one (initial 12 months of programme) and two work-based learning modules in year two (months 13-24 of programme). The dissertation module is completed in year three (months 25-30 of programme). Assessment one promoted personal and professional skills development and was examined through a critically reflective 5,000-word work-based portfolio. A key learning outcome was to raise awareness of the workplace as a ripe learning environment. Assessment two developed techniques required to plan, organise, lead, and critically evaluate a 4,500-word work-based project in a coaching organisation, thus granting opportunity to enhance individual effectiveness, employability and core coaching knowledge
Eight (male = 6 and female = 2) part-time, second-year, work-based postgraduate sports coaching students (aged 24 to 37 years) agreed to participate within the study. Once institutional ethical-clearance was granted, face-to-face debriefs reinforcing study aims, objectives and procedures to be followed were completed. Pre data collection, all were assigned pseudonyms to protect anonymity, informed they could withdraw at any time and provided written informed consent. Individual semi-structured interviews were undertaken with all participants at convenient times and locations throughout May and June 2017.
When undertaking qualitative research, it is important for the interviewer to build rapport and trust with interviewees (Lincoln & Gubba, 1985). The author previously taught several of the sample as undergraduates, was an ex-postgraduate sport student himself and possessed over 10 years experience of creating and disseminating audio feedback for previous cohorts. This strengthened rapport and bonding, helping to make them feel comfortable and at ease to answer honestly and insightfully. Each interview started with an informal discussion revolving around why they decided to embark upon a postgraduate sport coaching program and their progress to-date. Following this rapport-building exercise, data was collected using semi-structured interviews capturing how the participants viewed the audio feedback provided.
Interviews commenced with open-ended questions which initially probed students emotional responses evoked by audio feedback (e.g. “what was it like to receive for the first time”), moved onto exploring the impact on engagement (e.g. “what was it like receiving audio instead of written feedback”) and concluded by focusing on its effectiveness in supporting feeding forward, professional and work-based practice (e.g. “to what extent have you used audio feedback to support your future learning or that of others”). To elicit further richness to responses, supplementary probes were posed ad-hoc including “why did you feel this way”, “what was the reason for this” and “how does this compare to previous experiences”.
Duration was dependent on how much each participant had to disclose and ranged in length between 23 and 46 minutes. Each interview was transcribed verbatim, scrutinised multiple-times over a seven-day period to detect significant meanings and subjected to similar thematic-analysis guidelines published by Braun and Clarke (2006). Notes reflecting interesting and pertinent participant comments were placed within margins to unearth and capture the essence of the data. Initial associations and connections based on similarities and patterns between emergent themes were made, resulting in the development of three main-categories. Interview extracts representing each theme were selected. The final analysis stage involved developing written-accounts from these themes. Three weeks post-interview, each participant undertook a short member-checking telephone conversation with the author. This reduced ambiguity, enhanced accuracy and validity of responses and enabled participants to add things they may have forgot to initially mention (Lincoln & Gubba, 1985).
Each participant received circa eight-minutes of individualised audio feedback in mid-March 2017 which was specific to assignment one (4,500-word portfolio) and again in early May 2017, which was specific to assignment two (5,000-word work-based project). A digital audio-device, with inbuilt universal serial bus port enabling mp3 format recording, was used to create all files. This format is widely accessible and playable on all available modern-day technological devices. All audio files were created, internally moderated, and emailed to participants’ university email accounts, ensuring confidentiality and privacy, within 14 days of each assessment submission deadline. As a general rule, it took approximately 30 minutes to produce each audio recording at the desired level of quality for both assessments. This included time taken to read the assignment, identify key points and take home messages to be included and save the file in mp3 format.
All audio feedback, spanning both assessments, was produced and recorded adopting the following six-stage format:
1. Participant greeted in welcoming and pleasant manner with process to follow explained
2. Clarified which assessment the audio feedback related too
3. Ensured audio was developmentally focused, supportive and aligned with assessment criteria
4. Commented logically and insightfully on all assessment sections, emphasising key areas of strength plus future development (even if the work was exceptional)
5. Reiterated key points to feed-forward, provided grade and offered additional support (e.g. opportunity for informal face-to-face follow up meeting)
6. Provided final summative grade then concluded in a friendly and motivating manner
Each participant had no previous experience of ever before receiving summative assessment based audio feedback throughout their academic studies. All found the mp3 files simple to access with content generally perceived as comprehendible and clearly articulated. They particularly appreciated the understandable terminology and supportive language employed. For both assessments, every participant listened to their individual audio file at least twice within 72 hours of its delivery. This was mainly by phone, but occasionally via laptop. Participants’ understanding of audio feedback principles, how it was produced and the potential learning-gains it may support was limited. The following quotes clearly emphasise this point:
I was impressed by the approach and interested in what devices were used to create it (Matt)
I made an educated guess to what I thought audio feedback would entail but I can honestly say that I had never heard about this way of giving back feedback before (Paul)
The thought of receiving audio feedback was initially met by all with a general sense of curiousness, vulnerability, apprehension and nervousness, thus provoking several negative emotional responses including “worry”, “insecurity” and “uncertainty”. They had acclimatised and become accustomed to receiving more traditional written modes of summative feedback. Half wished for their grade to have been provided immediately once the audio commenced, thinking it would help alleviate the heightened discomfort, uneasiness and sense of trepidation they were encountering.
When compared with normal protocol (e.g. waiting for written assessment feedback to be returned), three participants in particular encountered elevated feelings of anxiousness, concern and restlessness in the days leading up to the audio feedback being released. This was particularly prevalent in the preceding 24 hours. In the following quote, Eve explained a discouraging situation of feeling slightly uncomfortable and overwhelmed:
this was the first time I had ever received audio, and I will be honest with you, in that I found it a bit odd and could not really get my head around it. Before it came through, I was thinking how is this going to help me as I have always just received written feedback. To start with, I was nervous in what was going to be said and did not really know how to take it
Similarly, Carol discussed experiencing a heightened feeling of unease immediately before she listened to her audio feedback. She said:
when I received the audio feedback, I went away to a corner of the staffroom at work, so I could hear it intimately using my head phones in private on my own. I was initially fearful about receiving audio feedback for the first time in that I was thinking, oh my goodness, is what is said going to be good or bad. It was scary and unnerving at first but I really listened intently and actually found it quite intriguing
It is important to realise how a minority lacked a general sense of confidence and belief to fully immerse with the feedback. To begin with, a couple of participants were especially unwilling and reserved to engage, only listening to short snippets at a time. The following quotes highlight the essence of such experiences:
to be completely honest, I was unwilling at first to pay full attention to the feedback as I just wanted my mark, but after listening to the first few minutes, I realised how not doing so could be a wasted opportunity because it was concise and understandable (John)
I sat there staring at the email feeling quite petrified as to what the feedback would say and how it would make me feel (Paul)
For every participant, their early reservations and apprehensiveness speedily eroded as they all started to engage proactively and enthusiastically with the feedback provided. For all, a positive picture began to emerge, with audio becoming perceived as “sincere”, “authentic”, “poignant”, “caring”, “helpful”, “invigorating” and “heartfelt”. The developing “buy-in” was reflected by several participants who discussed a heightened sense of feeling “valued”, “encouraged”, “appreciated” and “connected” as a result of engaging with the audio feedback.
Broadly speaking, the enthusiasm, expression, nuance and tone of voice employed to create the audio appeared to be a powerful catalyst for nurturing consequent engagement, emotional closeness and self-efficacy. This was clearly the case for Adam who said:
the audio feedback was not condescending or patronising in any way. I am big on relationships and I got a real feel you were going the extra miles to try and help me as a learner. I must say the audio feedback for me, goes beyond, and is empathetic, uncomplicated and critical when needed but all in a nice manner. I felt much better once I had listened through a couple of times after my initial jitters and fears of failure
Orally receiving critical feedback using an enthusiastic but calming delivery style came across to participants as more tactful and considerate. Furthermore, all gauged an accurate initial vibe for quality of work and subsequent summative grade based on the emotive, personable and poignant nature of language provided. For example, Steve stated:
I could tell by the tone of voice if the work was high quality or not which actually really helps and puts me at ease
Participants appreciated the thought, time and effort taken to produce the feedback, plus its potential to positively impact upon their future motivation to learn and individual growth. David emphasises this point in the following passage:
hearing the emotion and positive nature of comments gave me so much self-confidence to continue with my studies. I liked hearing the emotion in the feedback and this helped me digest the key points made in a far better way than any written feedback I have received previously has
Several participants described their feedback as being concise, informal, balanced and sincere. They liked how the approach encouraged them, more so than written feedback, to take greater responsibility for their learning. For example, John confirmed this by commenting:
for me, the audio helped trigger inner-thought and reflection and was far more complimentary than some of the directionless and disjointed written feedback I have received over the years
Over half believed audio feedback would lend itself well to other modes of summative assessment, particularly those comprising personalised elements, such as individualised action-plans and reflective-blogs. To illustrate this point, Carol said:
the feedback really clicked for me as it was so relatable to what I did well and not so well. I also really valued the feedback because of the authentic feel and vibe of the assessments. I am not so sure it would work so well for me if I received it for an essay or report. I think probably not if I am being honest
Once initial anxieties and concerns were overcome, each participant progressively engaged with the audio content. The following anecdote by Adam reinforces this attitudinal change:
I will be truthful and confess I have never read all the written feedback provided on other assessments at both undergraduate and postgraduate levels, mainly because I do not always understand the language, nor can I read it. In this case, I liked this way of feeding back and I must stress it is not just because of its uniqueness or the novelty factor. For me, I just found it easier to take the comments on board, especially the content discussing how I could look to improve upon next time. If this feedback was written, I would most likely have taken the huff but the way things were well explained in an encouraging way put me at ease
In comparison to written feedback, participants valued the greater depth, meaning and volume of detail provided. They especially welcomed the insightful nature and non-generic dialogue which supported areas for future development and feeding forwards. The general consensus deemed feedback duration of eight-minutes maximum to be sufficient, with anything longer running the risk of “disengaging” and “losing” participants. To illustrate, Ellie explained:
I would not want any more than seven minutes or so of feedback. I feel if it went on longer, then people may start to get a bit nervous and possibly switch off. I think too much could actually be counter-productive and ultimately lead to people disengaging
For the majority, the experience became a more personable, satisfying, inquisitive and pleasurable way to receive feedback and for consequent learning to emerge. For example, David and Steve discussed similar perceived engagement benefits:
receiving the feedback in oral format over email is more engaging for me as I can immediately listen to it on my phone and return back to it at any time. It is there for keeps whereas I just place assessments with written feedback provided into my files (David)
the key thing for me is that it just engaged me more. Like, with written feedback, I would just read it once or sometimes twice, but with the audio I felt much more inclined to listen to it several times also listen to it when working on preparation for other assignments in other modules. It has motivated and helped me to reflect on my previous work. It just comes across as less generic and descriptive and more tailored to you at a personal level (Steve)
The following statement demonstrates how appreciative Adam was for the well-intentioned and personable feedback he received. He explained:
the opportunity to receive feedback through an audio method makes me more inclined to listen to the content, pause, rewind, listen again and make my own additional notes around comments mentioned. Like, the most recent audio feedback I received stated how my referencing within the text was slightly incorrect so I decided to write a few notes of my own down about what I needed to do differently then went through my assessment and highlighted all the different times where I made the particular mistake. I found that to be a powerfully motivating activity and am now much more confident I can, and will apply, the new learning into future assessments
For some, the process of having the audio easily accessible helped navigate challenges faced with simultaneously managing modern-day student-life with competing work and family-life demands. To explain further, Steve stated:
with the way the world is changing, and the hectic way modern day life is, you always have your phone with you so it makes the feedback more accessible to engage with, potentially helping make us develop more sense of ownership and independence
David talked openly about the convenience and ease at which he was able to access and engage with his audio feedback:
for somebody who is on the move around the country a lot, audio feedback is more accessible. A big thing also for me is having quick and easy 24 hour access to my feedback. As I always have my phone with me, I actually listened to the first piece of audio feedback again yesterday morning and I just accessed it via email on my phone. Accessibility is just much better. I would never do that with written feedback. It also helps that general sense of connection with your studies and lecturer
Carol valued the heightened accessibility, empathetic tone and considerate vibe of audio feedback. She stated:
I recently listened to the feedback when sat in my car whilst waiting to pick up a family member. I just placed the feedback on a drop-box so I can access anytime and anywhere. Everything was simple to understand and clear, well-paced and expressed nicely. It all linked, making it easier to retain, and help me to understand what I did well and not so well
Five participants thought placing summative grades towards the end of the audio file helped optimise engagement and consequent feeding forward and learning. To illustrate, Ellie said:
I think it is very appropriate and clever how the final assessment grade is placed towards the end. If you started by saying, hi, feedback for this assessment will follow but the grade awarded is 60% then I would probably not listen to what follows. But, by placing the mark the way you do, you have me on a string where I am constantly thinking what grade I have received based on the comments you provide. It is like you are a poker player and I am trying to guess what card you are playing
Participants valued the bespoke nature of audio feedback, explaining how it felt more like a two-way process than traditional written comments, directly promoting critical reflection, feeding forward and sense of belonging and connection with academic staff. This opposed a sense of passivity, disconnection and alienation which was unearthed as a limitation of previous written-assessment feedback. To illustrate, Steve stated:
I must stress, I do not generally mind written feedback, but having audio for me means I am spending less time emailing lecturers to query certain points. I can just listen to the feedback and I realise, oh yes, I get what that means and what is being alluding to there. It has started to make me more independent and confident as a learner as well as developing a strong relationship and friendly vibe between myself and the programme leader.
Half suggested scheduling informal, follow up, face-to-face conversations between themselves and academic staff as being worthwhile, indicating it may allow for further targeted discussion and prompting towards learning independently and implementing effective feeding forward strategies. John said:
I would like to be able to come and meet up with my lecturers having had time to reflect on the information provided and maybe even completed a feed-forward task which encouraged me to think about how best to apply my learning. I think this would take the process to the next level
In their various roles as lecturers, coaches, educators and developers, two participants had already experimented with the approach by mid June 2017, whilst three others had encouraged work-place colleagues to consider doing so in the future. Providing feedback of this nature had not previously crossed the mind of Adam, yet he was in the process of providing grassroots football coaches with audio feedback snippets for short in-situ practical sessions which formed part of their formal national governing body coaching award criteria. He explained:
a lot of day-to-day activities in my current role involves providing coach education support and I am now presenting my learners with short snippets of audio feedback that is recorded on my phone and sent remotely to them later that day. The candidates tell me they find this useful plus it helps me when I have to write up formal written reports as I can listen to what I recorded about a particular session I observed and it refreshes my mind on areas that were good and those that required some work
As part of her further education lecturer role, Carol discussed having trialed in providing audio feedback to year one foundation degree sports students. She explained:
I was interested to go through the process with my own students but I was shocked in that I had to do many repeats and different versions. I now have first-hand experience in that it really does take time and it makes you realise how much effort your lecturer places into creating the feedback for you. I thought it would be quicker providing audio to written feedback.
In recent times, audio feedback has grown in popularity across diverse educational contexts and disciplines (Bond, 2009; Carruthers et al., 2014; Dixon, 2015; Lau, 2008; McGarvey & Haxton, 2011; Munroe & Hollingworth, 2014). This project extended the existing research base by carefully considering the voice of eight part-time postgraduate sport coaching students as to their expectations about, opinions towards and experiences of receiving assessment feedback in audio format. The evidence from this study suggests receiving audio feedback has the potential to be a sensitive, unsettling and overwhelming experience for students. Initially, all had limited understanding of the approach and receiving feedback in this manner was met with a general sense of nervousness, apprehension and uncertainty (Lunt & Curran, 2010). Historically, participants had received traditional written forms of summative feedback and were much better acclimatised and familiarised with this process.
Replicating the findings of Gleaves and Walker (2012), Jackson (2012) and Olesova and Richardson (2011), participants initial concerns and worries towards the audio approach soon started to diminish, with all progressively engaging with the feedback proactively and enthusiastically. Consistent with the extant literature, valuable audio feedback was portrayed as being detailed, personable, concise and devoid of repetitive or complex academic language (Dixon, 2015; Ice, Curtis, Phillips, & Wells, 2007; Knauf, 2016; Lunt & Curran 2010; Olesova & Richardson 2011; Rotherham, 2009; Van der Kleij, Adie, & Cimming, 2017; Varlander, 2008; Walsh, Larsen, & Parry, 2009). This clearly emphasises the potential for audio to inspire, motivate and nurture both the learning and day-to-day practice of future work-based postgraduate student cohorts. In line with Hepplestone et al., (2011) and Sipple (2007), ensuring audio is comprehensible, relatable and personable were particularly key findings, whilst potential strategies to promote feeding forward and learning more independently were also unearthed. Whilst the findings of Lunt and Curran (2010) suggest how providing audio feedback has the potential to save staff time, this was not the case for one participant who consequently trialed the approach with their further education sports students, encountering several barriers and operational problems along the way (e.g., having to start recording again after making a mistake). Overall, collective views and findings were generally positively framed and provide valuable contributions to academics, educational-developers, support-staff and senior-management teams, seeking guidance and support to inform future audio feedback design, delivery and practice.
The limited sport-specific audio feedback research is surprising considering the large student cohorts recruited annually to academic programmes within the discipline, and provides avenues for further investigation. For example, mixed-methods studies exploring experiences of both undergraduate sport student recipients plus academic staff who utilise the approach, would prove valuable and timely contributions to the contemporary literature base. Based on the study findings, a collection of ‘take-home’ recommendations and guidelines are provided in table 1 which may help support greater student engagement with audio feedback in the future.
Table 1: Recommendations and guidelines
|
Academic staff, with limited experience or understanding of the approach, should be appropriately supported and provided with appropriate re-tooling and/or professional development opportunities should they wish to create and distribute audio feedback to their students. This applies to all stages of the career-spectrum and not just newly appointed or inexperienced colleagues.
Prior to receiving, it is important that students are suitably informed about the very concept of audio feedback, plus the potential emotional impact it may place upon them. Therefore, encourage them to focus on comments rather than grades. Helping them to understand that everything you say is focused towards supporting them to positively enhance their academic, personal and professional skills forward is crucial.
Be up-front and honest with colleagues with limited experience of producing and disseminating audio feedback. Initially, the process can prove time-demanding, impact heavily on academic workload and challenging to administer, coordinate and manage. This is especially the case with large module cohorts and academic module delivery team. It can also take time to produce for those with limited experience and requires appropriate resource (e.g. equipment to create and edit mp3 files as well as locating quiet places to record).
The early stages of employing audio feedback can be a particularly sensitive time for student buy-in and engagement. Therefore, it is important to offer them additional academic and personal guidance to ensure they have accurate expectations of and feel suitably confident, connected, prepared and supported.
Create communities of practice with colleagues who use audio feedback, or intend on doing so in the future. This may encourage sharing of ideas and best practice, how to improve further and the merits of learning from mistakes.
|
Dr Rick Hayman is a Senior Lecturer within the Department of Sport, Exercise and Rehabilitation at Northumbria University, Newcastle and Senior Fellow of the Higher Education Academy. He has over ten years experience in providing summative audio feedback to undergraduate and postgraduate sport student cohorts.
The author thanks the eight participants who consented to be interviewed for their generosity, insight and time.