Susan Carter1, Barbara Kensington-Miller1,
and Matthew Courtney2
The University of
Auckland, New Zealand
The University of Melbourne,
Australia
In the current globalised environment, doctoral supervision is increasingly complex, multi-layered and challenging (see Nerad & Evans, 2014). The European Union Bologna Process third cycle opened an interesting can of worms in its drive to make doctoral education more fluid across national boundaries. Bologna Process decisions affect education beyond the European Higher Education area (for example, in Australasia – see Bishop, 2006; Sewell & Poutasi, 2008), raising challenges and opportunities (Carter, Fazey, Geraldo, & Trevitt, 2010) that inflect postgraduate supervision. Supervision must negotiate the consequences of globalisation. At our institution (a large research-focused university in New Zealand), for example, just over half of doctoral registrations in 2015 were international; most of these will not have English as their first language. Greater numbers of international doctoral students create new dimensions to supervision: cross-cultural social integration (Ali & Kohun, 2007; Chan & Dimmock, 2008) and the linguistic challenges of writing in English as an additional language (Paltridge & Starfield, 2007). With more conversations about the doctorate across the world, there are emerging new options for doctorate processes at our institution: practice-based (Aitchison 2015; Winter, Griffiths, & Green 2000), with performance (Ravelli, Partridge, Starfield, & Tuckwell, 2013) and by publication (Guerin 2016) rather than through the one magnum opus. A new global mobility energises discussion of doctoral pedagogy, with new kinds of support for doctoral students in addition to supervision: generic support from student learning centres or departments (Carter & Laurs, 2014; Golde & Walker, 2006), and peer writing group support (Aitchison & Guerin, 2014). Even so, supervisors are under more pressure to produce outputs – faster and in greater numbers – and the squeeze is unlikely to ease off any time soon. The background to supervision has turned it into a knottier academic practice. This article informs on the challenges experienced by supervisors for readers who want to calibrate their supervision realities. Are your experiences standard, or do they lie outside the norm?
Our project was driven initially by desire to design a suite of university-wide seminars for supervisors addressing some of the main pressure points of supervision and creating a space where practice could be openly discussed in a safe refuge from the professional front (Grant & Barrow, 2013). We expected that it would be more effective to tailor our seminars to discipline purpose and wanted a better understanding of where supervisors perceived the challenges to lie in their experiences with their various research epistemologies, ontologies, methods and tribal rituals (Becher & Trowler, 2001). The suite of eight seminars was predicated on a sound understanding of pressure points from the doctoral student perspective gleaned from over 1,000 hours of consultation by the lead author in a generic doctoral support role.
That longitudinal experience suggested that difficulties in supervision broadly related to three particular areas, namely, communication, overall project management and thesis writing, the areas that the questionnaire explored.
This paper makes the case that support for supervisors is more important in the current environment than even five years ago. Not only is there pressure for more supervision to timely completions, arguably there is also an ‘uberisation’ of the academic work environment (Carrigan, 2016). Universities flushed rosily in a renaissance of neoliberalism (Harris, 2006) might be viewed as an industry in which “terms such as audit, accountability, performance-based learning… and value-added learning have become familiar and contentious” (Rothblatt, 2008, para. 2) “We have been led to a democratization of job insecurity” (Brown, Hesketh, & Williams, 2004, p. 27). There is more focus on supervision, as interest in the fiscal potential of the knowledge economy plays out (Walker & Nixon, 2004). Erosion of job security makes it important to academics that they are effective teachers, and research supervisors. Often it is early career academics who step up for difficult supervision work (or are shunted into it), and experience it as deeply challenging (Carter, 2016) at a vulnerable stage of their career. Nonetheless, our experience suggests that supervisors tend not to flock to support and that something of a sea change in awareness is needed.
This study’s objective was to better understand the complex dimensions of supervision in different discipline contests, and to establish whether or not discipline-centric supervisory workshops might be a more useful way forward than generic ones. We expected this would be the case, particularly around writing, given the obvious epistemological differences. We entered this project expecting to come out with STEM (Science, Technology, Engineering and Mathematics) and HASS (Humanities, Arts, and Social Science) workshops tailored according to our findings. We came away with a stronger understanding of supervisory challenges across disciplines, and findings indicating cross-disciplinary similarities rather than differences. Overall, we found that supervisors need support, and that centrally provided rather than faculty-centric academic development workshops are appropriate: the problems are the same across the board and critical mass in a workshop where talk is open and safe assists in teasing them out.
We used a two stage multi-method study design, conducting a survey of accredited supervisors at our institution [n=226], followed by a series of interviews with experienced supervisors [n11]. Data cleaning for missing values (Dempster, Laird, & Rubin, 1977) resulted in a final dataset of 216 cases.
Theoretical sampling identified expert informants to ensure that we captured perceptions from those with extensive experience: senior research colleagues assisted us to identify faculty members with supervision excellence typically recognised by high supervision completion rates, research leadership roles within their faculties, teaching award achievement and recognition from students and co-supervising colleagues. We conducted 11 semi-structured interviews averaging an hour long. We focused on areas outside of our own experience in Education, Humanities and Mathematics, looking at Science (Physics, Psychology, Computer Science), Engineering (Chemical, Mechanical), Faculty of Medical and Health Sciences (FMHS: Surgery, Obstetrics and Gynaecology, Medicine), Dance Studies (Performance), Arts (History), Law, and Education (elearning).
The interview process firstly asked expert informants to review our planned suite of supervision seminars and to give feedback on whether they would meet their particular faculty needs, and whether important aspects were missing. We then investigated dominant epistemic and methodological research approaches, and explored what factors made supervision difficult. Finally we asked for insights into perceived blind spots: what might we not know about research supervision as people outside the particular discipline.
With ethics committee approval, an anonymous questionnaire using convenience sampling was emailed to a census of 1,247 doctoral supervisors registered on a list of accredited supervisors at the start of semester 1 (March), 2014. The response rates for each of the eight faculty groups are presented in Table 1.
Table 1: Participant Supervisors by STEM, non-STEM, and Faculty Cohorts
Note. STEM = Science, Technology, Engineering, and Mathematics; Creative Arts= National Institute of Creative Arts and Industries, who are primarily practice and performance based.
The participants were each concurrently supervising on average close to nine doctoral students (M = 8.72, SD = 7.91).
In terms of experience, 81 participants (37.5%), had more than 13 years’ experience in the role; 28 (13.0%) had between 10 and 13 years; 30 (13.9%) had between 7 and 10 years; 45 (20.8%) had between 4 and 7 years; and 27 (12.5%) had between 1 and 3 years. The smallest proportion had five participants (2.3%) who had less than one year. The depth of supervisor experience behind the data gathered adds to its reliability.
In terms of position, all were tenured academics: 57 (26.4%) were professors; 59 (27.3%) were associate professors; 82 (38.0%) were senior lecturers; 14 (6.5%) were lecturers; and, 4 (1.9%) identified themselves as “other”. Therefore, overall, there is a marked bias towards those with more experience.
The lead author developed the on-line questionnaire in consultation with colleagues in senior research positions, with 22 questions covering aspects of communication, thesis writing, and overall project management during doctoral supervision. The questions were presented in a five point rating scale from ‘Never’ through to ‘Always’.
Table 2: Results for Kruskal-Wallis Tests of Effect of Respondent Faculty Grouping on Variables of Interest
The initial dataset included 226 cases. SPSS 22 (IBM, 2015), and Kruskall Wallis tests were used to measure faculty dominance for each of the 22 questions. Where faculty dominance existed, all possible faculty-to-faculty comparisons were explored using the post-hoc Krushall Wallis procedure (Bonferroni adjustments were made). Effect sizes were also calculated to determine the magnitude of the difference between faculties using the Frequency Sheet tool calculator (Wilson, 2001). To support readers not familiar with statistics we replicate a simple explanation of statistical significance (p) and effect size (d).
First, statistical significance: from a general perspective, classical statisticians encode evidence on a 0.00 to 1.00 scale, where smaller values mean that there is a stronger probability that the findings are not a fluke and are therefore reliable. A p (probability) value of 0.05 is where the reliability line is conventionally drawn. Therefore, based on the p column in Table 2, there is evidence of significant single-faculty dominance in seven of the 22 questions where p is less than 0.05 (p < .05).
Second, effect size (d): Cohen’s d is a standard measure of the magnitude of the difference between two means, which can be used to compare faculties. Hattie (2009) suggests that Cohen’s d values of 0.20 to 0.39 are small, 0.40 to 0.59 are medium, and 0.60 and above are large. One finding in this study was that Faculty of Education staff had greater resultant passion from supervision than Business staff with d = 1.00 (p = .019). The magnitude of this difference is represented visually in Figure 1 where the overall Business’ supervisors’ responses (darker shade) are dominated to the right by the overall Education staff responses (lighter shade). The overlap shows the difference.
Figure 1. Cohen’s d value of 1.00 showing Education and Business difference in supervision passion
Here we present and discuss the quantitative findings first and then the qualitative findings that enrich the figures. The most important overall finding from our quantitative data was that there were no faculty differences in what supervisors found difficult: the challenges to doctoral supervision are the same across disciplines. Commonly education discourse assumes that generic ‘study skills’ seminars are never as strong as those tailored for specific courses (e.g., Wingate, 2006). We approached this project expecting to find that the supervision support we provided would be improved by designing to the specific needs of supervision within different epistemological frameworks. That this is not the case supports the proposition that a generic suite of seminars offered to all is a constructive way to support supervisors. The design tailoring that gives value is the merit of addressing different pressures in supervision not discipline difference: all who choose to attend a workshop on a specific topic, e.g., supervising across cultures, will be facing the same issues and will benefit from attending.
However, quantitative data showed that what differed between faculties were the experience of both supervisory work satisfaction and improved social relationships: not the challenges but the perceived benefits. At least one faculty group dominated another for 7 of the 22 variables that showed the positive benefits from supervision. These are represented in bold in Table 2. Each question in Table 2 had a 5 scale rate from ‘Never’ through to ‘Always’.
We note that statistical analysis ignores complexity such as gender and culture, limitations that may explain for the faculties differences detailed here. Further research could unpack this and would be needed to fully interpret causes of difference. In this instance, that there is faculty difference in supervisory satisfaction and pleasure may intrigue readers, prompting future investigation.
Data showed five instances of faculty difference relating to positive aspects of supervision. Results of the post hoc Kruskal-Wallis pairwise comparisons reveal the specific instances of faculty dominance for each of the seven question items asking about supervisors’ positive experience: Education supervisors found meetings energising (Q5) to a greater degree than in the Business (d = 1.04, p = .008).
Enhanced interconnectivity in academia also showed faculty differences. Satisfaction in the strong working relationship developed with a doctoral student during supervision (Q6) was more intensely experienced in Science than in Business (d = 0.72, p = .019), with Education also dominating Business on Q6 (d = 1.21, p < .001).
Those from Science (d = 0.67, p < .033), Medical and Health Sciences (d = 0.82, p < .013), and Education (d = 0.97, p < .020) strengthened their professional networks through supervision (Q7) more strongly than those from the Faculty of Arts. In addition, Medical and Health Sciences dominated the Faculty of Business (d = 0.84, p < .050) regarding the strengthening of professional networks.
Medical and Health Sciences academics enjoyed helping their students develop their writing (Q14) more than those in the Faculty of Business (d = 0.99, p = .034). When it came to benefits from co-publication (Q17), Education (d = 1.77, p = .020), Science (d = 2.57, p = .020), Medical and Health Sciences (d = 2.97, p = .020), and Engineering (d = 4.38, p = .020) dominated responses from Creative Arts.
There were faculty differences in the passion resulting from supervisory work itself. In this case, the Faculty of Education had greater resultant passion (Q19) than Business (d = 1.00, p = .019) and those from Arts (d = 0.90, p = .022). Education academics likewise experienced supervision as the best aspect of academic work (Q22) more often than those in the Business (d = 1.07, p = .020) and those from the Arts (d = 0.98, p = .028).
Results here raise questions rather than answers. Are some faculties really more enervated by the practice of supervision? Why is that? What lies at the heart of supervision pleasure, satisfaction and passion?
Several key themes regarding challenges emerged from the interview data: time-restriction, and practical considerations such as funding, materials, and teamwork. You will recall that because we had experience ourselves in supervising reflective, narrative or enquiry based research, we approached experts in disciplines where we lacked experience and whose students’ research is performed, empirical, or practice-based.
The most salient point made was the challenges caused by tightening time constraints. Although their students completed and submitted on time, participants felt potential for development was sometimes lost. For example, students had restricted development as researchers exploring the realms of possibilities. The dance supervisor declared,
The trivial is important. Discovery is a better tool than completion. Curiosity.
The engineer recounted how much satisfaction he had found in the past when time was more flexible and he could show doctoral students the ropes as academics and researchers. Then he said,
…but things have changed, now with a three year time frame, I can’t…[pause] How can they learn how to be researchers, teachers, and gain an academic set of skills? … I don’t want to supervise research students like undergraduates, where I tell them what to do and they do it and come back and I tell them what to do next…
And the gynaecologist also mourned:
It’s something of a tragedy that we have lost those rich discussions, in science you don’t have time, it has to be output…
A surgeon supervising students doing lab work explained the missed opportunity as a loss to research progression:
When you are nearly there when do you stop—especially if you think you could do another month and then you are making a quantum—it’s really hard to draw a line in the sand. Once you stop, the lab often can’t pick it up either, that is so hard, it’s too hard to them. You lose the whole thing…
For him, the research itself rather than the individual student’s development was what was sacrificed. This limitation compromised his own ability to maintain a laboratory that was reliant on external funding. Similarly, a physics supervisor warned:
Be aware of the lab book. A lot of projects last longer than the students.
Although the doctorate has one set of criteria and regulations, in practice-based, empirical and performance-based research there was sometimes tension as the research-team dimension, materials, or responsibility to external funders demanded compliance. For most students working in labs, supervisors commonly “dictate the topic as [this is] more productive” and some students felt this as limiting. The medical supervisor believed the best model was when “the culture of supervision is protocol driven” arguing that this was the most productive and would provide the best way forward for both student and research project. He added that, “it’s their PhD, but it’s also our PhD… we put out 15-25 papers every year between 5-6 people…” Dependence on students’ output for one’s own success becomes more critical in such cases. In general, many PhD students involved in STEM subjects, such as chemical engineering, are “seen as junior staff on teams” as “you don’t have lone wolves in this discipline, it is all team based”. Here, the supervisor justified compliance with team needs, believing that:
it is easier if they are part of a team as then they can see research is about uncertainty. That helps them to survive and then the uncertainty is actually part of the journey. I tell them it is okay to be doubtful. In engineering there are set rules, very structured ways of thinking towards an endpoint but research is different and if they are doing it right there will always be more questions at the end. This is the level of maturity they need and I tell them it will be an emotional rollercoaster.
Funding practicalities were a real constraint for researchers working in laboratories, while the ability to sub-contract work meant doctoral students were not learning as many skills as they had to in the past. The cost of equipment in engineering affects supervision as “sometimes they want to do something but the machine they need costs $100,000….one thing that always comes up is material costs – The cost of all these things affects the learning students do, but it’s hard for a supervisor too.” The same supervisor was unhappy that practice around technology has changed: instead of learning how to do all aspects of laboratory work themselves, “these days, some students, they drop their material off and pay someone to get the results, then they say ‘don’t ask me, the technician did it for me’”. He felt that supervisors could not always feel that research was rigorous when doctoral students took what he thought of as shortcuts that nonetheless he could see saved them time as they drove the project to timely completion.
Similarly, the physicist voiced funding concerns and the importance of PhD students working within her framework as “the funding only covers my work, and also if they damage the equipment it is costly to replace”. Laboratory work was often risky for supervisors, particularly for costs and the worry that they could lose precious funding money if students didn’t work out. The usual practice in chemical engineering is for supervisors to have funding organised in advance to attract PhD students to come and study with them. The pressure for enough money to support the research and the student were enormous, including the expectation of fees to be also paid and the “pressure to find more funding for stipends and then they may need extra for equipment. The problem arises if they fall out as then you lose your funding”.
Practice-based research added complexity to the time squeeze dimension. The surgeon was full time – he explained that you can’t be a part-time surgeon – so was using some of his funding to employ a full time position for someone to effectively do supervision work for him. This research assistant supported his research students’ ethics approval and grant applications, thesis writing and even gave holistic psychological support, and with a team-work approach, the surgeon himself was able to oversee “more doctoral students than most supervisors get through”. He was cross about a current cull at our institution of professional support workers, because it intruded on academics’ research capability. The fairly recent rigour of timely completion was felt to compromise the quality of what supervisors were able to do with students when they had to fulfill practice requirements.
Pressure for timely completion was new at our institution, which has systematically tightened regulations over the last ten years or so – and it increased the difficulty of juggling costs, funding and team work in laboratories that are endemic to practice-based supervision e.g. clinicians or surgeons. Another difficulty that was discipline-related to applied and practice-based research was the need for students who might be highly experienced as practitioners to cross the doctoral threshold concepts (Kiley & Wisker, 2009) required of research in academia.
Doctoral students who were excellent practitioners often found it demoralising to be weak academically. The surgeon said that many of his doctoral candidates were highly-skilled, established surgeons entering doctoral work to contribute to problems that they had encountered and that they believed needed research-based solutions. These experts often floundered when reduced to novice status in academia: “we’re pulling people out of surgical training, into academic surrounding: it’s like taking a fish out of water and putting it on dry land.” And the dance supervisor observed that this was often true of academics too, who inspired with their performance, but did not themselves have that much experience or understanding of academic convention. Sometimes, he felt, the result was poor supervision: “good performers are not necessarily good teachers. Often they have a tacit brilliance – it enables awe but not students who can progress.” The same may be sometimes true of creative artists both as supervisors and as students.
Then sometimes with work in applied research, students are very good at application, and not so skilled at writing, and therefore averse to producing it. The computer science supervisor found this perhaps the most frustrating factor in her supervision:
They’re working in software. That software they build isn’t in the thesis – the developed programme is not handed in, examiners don’t even look at it. Students would prefer to write the programme than to write the thesis. It’s really hard to get them to do that. I don’t know what to do when I give advice that they ignore. You can lead a horse to water…
Although the majority of Law students end up practising law, doctorates “are grounded in casework and theoretical models”. In this discipline, as in Arts and Social Science, the doctoral student works in isolation, which is a risk if “the student finds a topic interesting and spends a disproportionate time going off on a tangent”. All the supervisors complained about the writing phase being a difficult stage. The physicist simply stated that “students in physics can’t write” and that:
the problem is usually with the local students not the international ones. It is not always the case that if you have English as your first language that they write well… I always manage to get my students to write as early as possible to show them how important it is…
In contrast, the law supervisor was happy with local students, saying, “domestic students are pretty good, as they’ve come through our courses, and their literacy is reasonable” but that:
It’s quite different with students from overseas, some supervisors rewrite almost every page – I did it with a Masters student. What will commonly happen is that I do corrections as I go through. With a Masters, though, there is time compression. The problem for students is that you are embedded in course requirements, and don’t have time to go off and polish your English.
The education supervisor emphasised the importance of getting to know her students and what sort of feedback works best for them. Her method was to “take a paragraph and rewrite it, then talk the student through it”. The physicist talked about the need for more ownership of the process, saying “at some point it is their PhD so they need to take more initiative and to have discussion when it is not going well”. She believed that it was kinder and better with the feedback process for the student if she was:
more direct, and to make sure it is not ambiguous. For example, if you say ‘that’s not too bad’ this can be interpreted in many ways. You need to say either that is good or that is not good. The feedback needs to be explicit to the student. Don’t say it is fine as what does that mean.
Supervisors shared recognition of the need to be patently clear with communication. The law supervisor also talked about being direct and the “potential for miscommunication so you need to select your vocabulary”. He discussed the different ways language could be interpreted when giving direction with feedback explaining “the difference between ‘you have got to’, ‘you could’, and ‘I think you should’… this is a key factor. You sometimes need to just say ‘you must’”. It was important to be necessarily unambiguous and further explained that if the “supervisor says I want you to mention x and y, as they think that x and y are major, then if the student isn’t interested and it is not there, you begin to doubt the quality”.
They also recognized the benefit of establishing a community of practice (Wenger, 1998) so that support is broadened. Supervisors in both STEM and non-STEM disciplines who had a number of doctoral students talked about having group meetings on a regular basis. The psychologist described how she would “provide a paper and then I ask the students to have turns taking the lead” at these meetings. The chemical engineer also used these meetings for pastoral care, encouraging “the older PhD students to take on the new ones and basically they enculturate each other”. Sometimes mentoring is more formalised, as the Science supervisor explained: “I always have a group of PhDs so I make an informal appointment with one of the senior research fellows, they do the shepherding, and work with the others’ writing.”
Overall, our informants unanimously (and often enthusiastically) endorsed the proposed curriculum, and expressed a perceived need for these workshops. Our evidence indicates that supervisors from STEM and non-STEM disciplines face the same challenges. It seems that challenges are generic, and our statistically valid evidence confirms the benefit of centrally-provided generic supervision workshops as one approach to support for supervisors.
Nonetheless, our statistics showed that the Education, Medical and Health Sciences, and Science faculties at our university have markedly better doctoral supervisory experiences. On the other hand, Business, Arts, and Creative Arts had markedly lower levels of enjoyment with doctoral supervision. Education’s supervisors enjoyed most socio-emotional aspects: improved passion, energising conversations, student-supervisor relationships, research profiles, expanded networks, and more frequent identification of doctoral supervision as an academic highlight. Medical and Health Sciences’ experiences centred around improved enjoyment, research profiles, and expanded networks. The Faculty of Science’s more positive supervisory experience centred on improved relationships, research profiles, and expanded networks.
This study contributes to the understanding of supervision in several ways. First, it teases out what troubles practice-based researchers, and much of this is distinctly different from research practice in the Humanities and Social Science areas of work with which we would expect. Nonetheless, secondly it finds reliable numerical evidence that despite practice difference, supervision challenges differ little across a wider context. Thus it establishes that supervisors benefit from generic workshops with peers from across campus despite obvious discipline differences in practice. Our findings endorse the importance of writing support for doctoral students as espoused elsewhere (e.g., Carter & Laurs, 2014; Guerin, Carter & Aitchison, 2015) in assisting supervisors by providing additional support for their research students. Workshops that provide practical suggestions for supporting research writing would probably be of benefit along with writing retreats for both supervisors and students (Murray 2015a; 2015b; Murray & Newton, 2009).
Time-to-completion is an issue for empirical research: supervisors in those areas might keep time management to the fore of their supervisory discussions and share their time-management practices with early-career colleagues who are entering supervision. We believe that bureaucratic measures to ensure timely completion protect doctoral candidates, and we know that many have families and jobs and cannot afford the luxury of the intellectual idyll, the chance to pursue curiosity. Nonetheless, we empathise with supervisors who feel that their ability to really contribute to the research community is compromised by the drive for timely completions.
Our surprising finding here shows that challenges around writing, communication and overall project management are shared by both STEM and non-STEM disciplines. There will be variations on what ‘overall project management’ entails, but in every methodological approach, the pressure and challenges seem similar. There is scope for more finely nuanced study, which may tease out more difference than we have here. Certainly, too, there is scope for more support for the practice of supervision.
Susan Carter is an academic developer at the Centre for Learning and Research in Higher Education, the University of Auckland, New Zealand. She researches doctoral pedagogy.
Barbara Kensington-Miller is a senior lecturer at the Centre for Learning and Research in Higher Education, the University of Auckland, New Zealand. Her research fosters and promotes teaching and research of early-career academics.
Matthew Courtney is a Research Fellow in the Assessment Research Centre at the Melbourne Graduate School of Education who specialises in statistical analysis of quantitative educational data.