From Theory to Practice: Critical Thinking as a Multifaceted Concept

A pilot study investigating consensus between students’ and tutors’ perceptions in higher education

Beverley Barnaby, Middlesex University, UK

Developing students’ critical thinking skills is a main aim of higher education and many educationalists agree that cultivating such skills is an essential aspect of academic study (Behar-Horenstein & Niu, 2011). The Quality Assurance Agency’s descriptor for a Bachelor’s degree requires universities to develop students’ critical thinking. Such skills enable learners to dig below the surface of subjects and engage in critical dialogue with its main arguments and theories (Cotterell, 2011); but there are also wider social benefits, for example critical thinking is expected to help people to be responsible citizens (Behar-Horenstein & Niu, 2011), and to support better decision-making on the basis of careful and detailed evaluation of evidence, which is important in an increasingly complex society (Renaud & Murray, 2008). However, whilst academics frequently agree about the importance of developing students’ critical thinking skills, there are multiple definitions (Xia-jun, 2012), and theories of critical thinking (Mason, 2007). Paul and Elder (2006) embrace this diversity but Lloyd and Bahr (2010) suggest that there is a need for a precise and consistent definition. Knight (2007) argues that students and tutors need to have a clear and shared understanding of critical thinking for fair and valid assessment of students’ work to be made.

This study draws on experts’ views on how critical thinking is conceptualised both as a set of skills (Facione, 1990; Paul & Elder, 2006) and as a set of dispositions or traits (Ennis, 1998; Paul & Elder, 2006). In relation to skills, Facione reports on the views of 46 academics from a range of disciplines who defined critical thinking as a cognitive practice “using interpretation, analysis, evaluation, inference and explanation of the evidential, conceptual, methodological, criteriological and contextual considerations upon which it is based” (Facione, 1990). Whilst certain cognitive skills are considered essential for critical thinking, the way in which these skills are applied will differentiate a strong critical thinker from a weak critical thinker (Facione, 2011; Paul & Elder, 2006), and this is influenced by an individual’s critical thinking dispositions (Ennis, 1998). These include intellectual integrity, humility, sense of justice, perseverance, fair-mindedness, confidence in reason, courage, empathy and autonomy (Paul & Elder, 2006).

Students’ critical thinking skills need to be developed from one year to the next as tutors are required to design courses that adhere to QAA (2014) level descriptors, which set out the outcomes for students studying in years one, two and three. Skills relating to critical thinking become increasingly higher order as, among other abilities, students are expected to evaluate and interpret data in year one, critically analyse in year two and accurately deploy established techniques of analysis in year three. In the institution where this pilot study was conducted the assessment tasks of each module are designed to assess these critical thinking abilities, and tutors incorporate student activities such as debates, concept mapping, case studies and high level questioning.

Although there appears to be a consensus that critical thinking is a major outcome of higher education and universities are committed to developing students’ ability with regard to this cognitive ability, there does not appear to be a consensus of opinion between academics as to how critical thinking is defined and characterised. It may, therefore, be deduced that this lack of agreement will be found amongst tutors and students in higher education. This study will therefore focus upon the way in which critical thinking is understood within a BA in Early Childhood Studies programme at a university in North London. This study is part of an ongoing research project, covering a long phase of academic progression, from MA to DProf. The results presented are from an initial small-scale investigation and act as a pilot study for a larger project which is currently being conducted. A longer paper will follow with the next phase and further data will be collected and built into a Doctorate.

Methodology and methods

The main questions driving this research attempt to discover whether students studying for a BA in Early Childhood Studies and their tutors have an agreed understanding of what constitutes critical thinking, and how it can be assessed. For the main research instruments I have drawn on two leading authors in this field to compare students’ and tutors’ understanding of critical thinking skills (Facione, 1990) and attributes (Paul & Elder, 2006). It can be argued that the way in which learners and tutors understand the definition of critical thinking will affect how the concept is learnt and taught, for example, Knight (2007) insists both groups need a shared understanding of assessment. Conversely, Hemming (2000) and Wright (2002) suggest that a lack of knowledge regarding critical thinking will inhibit tutors’ ability to foster the development of students’ critical thinking skills.

Although an initial report (Barnaby, 2014) included more qualitative data, this paper draws mainly on quantitative data, relying on semi-structured questionnaires and interviews to collect data. The participants in this study are tutors (all of whom have been teaching in higher education for at least five years) and undergraduate students who are studying for a BA in Early Childhood Studies. Eight students were given a questionnaire: four from year two and four from year three; two from each year group were averaging First or 2:1, and two were achieving a 2:2 (those with lower attainment were not selected). First year students were not included in the study as their understanding of critical thinking may need further development and some drop-out from their course. All second and third year students were invited to participate and those chosen were the first to offer, who met the attainment criteria. Although the participants are known to the researcher, anonymity is ensured as questionnaires are only labeled with the degree classification being achieved.

The questionnaire included three types of questions. First, some open-ended questions elicited participants’ understanding of critical thinking. Second, rating scales questions were used to explore students’ perceived level of critical thinking and tap into the students’ attitudes and perceptions (Cohen, Manion, & Morrison, 2007). Third, rank ordering questions were also used to ask participants to prioritise the skills (Facione, 1990) and qualities (Paul & Elder, 2006) involved with critical thinking. Respondents were asked to only rank their first five priorities, as too many considerations may be overwhelming (Cohen et al., 2007). The frequency of occurrences is counted when analysing the results (Oppenheim, 1992).

Three tutors teaching on the BA in Early Childhood Studies programme took part in a focus group interview. This method was chosen as it offers an opportunity for the tutors to engage in a discussion which could yield a wide range of responses and increase the depth of the interview (Cohen et al., 2007, Oppenheim, 1992). The department’s co-ordinator for learning development was interviewed independently as she specialises in academic writing and the researcher did not want her opinions to impact upon the subject tutors’ personal responses.

Both of the interviews were semi-structured in that they used a schedule of questions based on the questionnaires, but the sequence varied depending on the responses. This allowed more open answers and offered an opportunity for the researcher to probe answers that may be too general (Costley, Elliot, & Gibbs, 2010). Both interviews also included the ranking tasks, which were administered on a face-to-face basis.

As the sample size in this study is quite small, it may be considered difficult to generalise the project’s findings to other situations in higher education as they may be considered peculiar to this selection of people on an Early Childhood Studies programme in North London. However, the main object of this pilot was to assess the extent to which these data collection methods were useful in exploring the issues raised in my discussion of the literature on critical thinking.

Evidence, analysis and discussion

In this section data is analysed and discussed in order to gain an insight into the conceptualisation of critical thinking within a BA in Early Childhood Studies department. In particular, the discussion focuses on whether there is consensus about critical thinking characteristics, between students and tutors.

The students were also asked to define critical thinking in their questionnaires, and the tutors were asked to define critical thinking in the interviews. The results have been coded in relation to Facione’s (1990) list of skills and Paul and Elder’s (2006) list of attributes. Whilst this was a straightforward process using the quantitative data, the free responses were turned into visible data by coding words that directly matched the experts’ lists. A second level coding procedure was used to identify synonyms, and another coder shared and compared the coding to assure a level of inter-rater reliability.

Figure 1: Indicates how students and tutors both mention skills associated with critical thinking by Facione (1990) and Paul and Elder (2006).

Bar chart showing the percentage of students and tutors identifying particular critical thinking skills. Percentages enable the results to be compared easily on the same chart but one should note the data refers to 4 tutors and 8 students.

Whilst one can identify broad agreement in relation to the cluster of skills on the left of Figure 1, tutors also identify self regulation, reasoning and problem solving as necessary for critical thought but these skills are not suggested by any students. This aspect of the results will be explained in more detail, when the data related to the next question is analysed.

Figure 2: Shows that when rating the top 5 skills necessary for critical thinking all of the students and tutors agreed that analysis should be rated within the top 5.

Bar chart showing the percentage of students’ and tutors’ top 5 thinking skills that are necessary for critical thinking.

Although there is a consensus of opinion that analysis is a top requirement of critical thinking, and a general consensus that reflection, reasoning, evaluation, problem solving, argumentation, interpretation and metacognition are rated within the top 5 necessary skills for critical thinking, there is a mismatch between the tutors and students as to the importance of inference, reconstructing, self regulation, synthesis, explanation and assessing. In particular, reconstructing, self regulation and synthesis are rated by at least two of the tutors but not by any of the students. A majority of students (n=5) rated explanation and assessing as being in their top five critical thinking skills but none of the tutors agreed.

Whilst a pilot study with such small numbers means it is difficult to generalise findings; a degree of overlap between staff and students understanding of critical thinking may be expected as tutors play a major part in scaffolding students’ learning. Also, the differences in the way that students and tutors rate critical thinking are interesting and flag up issues to consider for further exploration.

Figure 3: Indicates how students and tutors rate the top five critical thinking attributes (attributes have been taken from Paul & Elder, 2006).

Bar chart showing the percentage of students’ and tutors’ top five critical thinking attributes.

Figure 3 demonstrates that students’ and tutors’ rating of their top five critical thinking attributes appear generally consistent with each other. Differences appear in some areas, for example seven students rate intellectual fair-mindedness as being in their top five attributes for critical thinking but only one of the tutors agrees; two of the tutors rank intellectual humility as being in their top five attributes and no students agreed.

While there is a general consensus regarding the importance of many cognitive skills, there is a discrepancy in the weighting that students and tutors placed upon some of the thinking skills; apart from analysis which all students and tutors saw as being a top critical thinking skill. Although the results from this initial study demonstrate that students and tutors generally agree as to which critical thinking attributes should be rated highly, only one tutor regards intellectual fair-mindedness as a top attribute whereas seven out of the eight students see this as an important attribute.

In this initial pilot phase, research findings suggest that students’ and tutors’ understanding of critical thinking overlap. However, there is a discrepancy in the way in which students and tutors rate the importance of critical thinking characteristics.

All of the students felt that their critical thinking ability had improved since entering higher education (one reported their skills had developed ‘slightly’; four ‘well’ and three ‘significantly’). When asked how they rated their present ability to think critically (‘poor’, ‘fair’, ‘average’, ‘good’ or ‘excellent’), three students said ‘average’ and five ‘good’. Out of the four students presently achieving a First / 2:1, three said their ability to think critically was ‘good’ and one said ‘average’. Out of the four students presently achieving a 2:2, two said their ability to think critically is ‘average’ and two said ‘good’.

When asked ‘how developed they thought that students’ critical thinking skills were when they entered university’, all four tutors replied that they thought their ability was either low or poor. The Learning Co-ordinator said ‘I would say quite low… there is a naivety about academic study.’ Chris agreed ‘I would say quite poor.’ Val added ‘I don’t think… they have developed critical thinking skills’ and Anne concludes ‘Thinking can be black and white’.

Asked if they thought that second and third year students had improved their critical thinking skills since entering university, all four tutors agreed that the students had improved this skill but the answers suggest that that they are not completely sure as to the extent of improvement so that further improvement would be desirable. The Learning Co-ordinator said, ‘I think yes… but to what degree depends on how critically thinking they were in the first place.’ Val suggested ‘I think they… have to some degree but I don’t think it’s as much as I would like to see.’ Anne commented ‘All of them will be developed in some way at some level [.........] at the level we are not looking at grades.’ Finally Chris stated ‘I think they all improve in some way if they have engaged… even to a minimal extent.’