Susan Smith and Rebecca Sellers, Leeds Beckett University, UK
Classification is the taxonomic science in which a system of categories or attributes is established in a logical structure (Travers, 1980). Taxonomies have long been used to define such diverse entities as plants, animals, algorithmic processes and educational objectives. In science education, both conceptual change (Dykstra, 1992) and critical thinking skills (Gilbert, 1992) can be classified. Within general education, taxonomies have mainly focused on evaluation and objectives (Bloom, Englehart, Furst, Hill, & Krathwohl, 1956) and are intended to help teachers, research workers and administrators to discuss and deal with curricular, assessment and evaluation issues (Veal & MaKinster, 1999).
The former university taxonomy of assessment domains (Link 1) was based on Bloom’s original Taxonomy (1956) which identified six developing levels within the cognitive domain from simple recall to the highest order (classified as evaluation). It was designed to be a classification of student behaviours which represent the intended learning outcomes of the educational process, i.e. outcomes-based assessment (Heywood, 2000).
There is context and history for changes to Bloom’s original classic work. This original Taxonomy was revised in 2001 (Anderson et al, 2001) where the domain names became transformed into active verbs (i.e. “knowledge” became “remembering”). Earlier, the SOLO framework (Biggs & Collis, 1982) moved to a classification expressed in terms of complexity and the quality of the work done rather than just assessing students in terms of the bits they got right or wrong, i.e. it is a journey towards competence. Different designs and representations of the original Taxonomy have also been devised and adapted by different institutions (Heer, 2012) and current needs and contexts have been taken into account in the development of new versions such as the digital taxonomy (Churches, 2008) where the higher order skills in the original Taxonomy are related and levelled to different digital skills.
Initial work: Discussion groups with academic staff in all the faculties, led by the Centre for Learning and Teaching, revealed that the original Taxonomy was useful but that its language needed reviewing for clarity and currency, was difficult to access on the University’s website, was unfamiliar to new staff and needed modernising to support the ongoing Periodic Review and the advent of the University’s new graduate attributes. In addition, the postgraduate (PG) and undergraduate (UG) taxonomies were separate documents and there was a clear mismatch between the UG and PG domains leading to inconsistent practice and confusion. Ethical approval was sought, but not required, as the practice aligned to routine enhancement of teaching and learning activity and involved no primary research.
Some students reported, via the qualitative National Student Survey data and module review, that they needed more clarity about assessment criteria and ‘levelness’ expectations. Staff also requested, as part of their course design work, the need for a useful tool to support the development of robust, level-consistent intended module and course learning outcomes and constructively aligned assessment expectations. In addition, Leeds Beckett University had three new GAs which needed to be integrated into the intended learning outcomes in the undergraduate courses.
Establishing the SLWG: The SLWG was established through an invitation to all faculty academic staff (new and experienced) interested in teaching and learning. The SLWG spent some time discussing the wider context and uses of taxonomy development in general. Informed by Moseley et al (2005) work on thinking skills frameworks, the SLWG focused on exploring a) theoretical understanding of why taxonomies of assessment domains might be needed, b) the value and purpose of a taxonomy, c) how the application of the taxonomy as a framework for thinking to practice might help build knowledge and resource, d) how the SLWG could best approach the work and then promote the new tools for staff to appreciate the updated taxonomy.
The SLWG concluded, after considerable discussion, that they would not i) change the taxonomy fundamentally, ii) adopt a brand new one or iii) make changes that might necessitate immediate regulatory changes. Because staff were familiar with the existing model, a decision was made to just modify the structure and wording of the existing taxonomy (if there was a clear reason to do so) and develop new interactive supportive tools.
The SLWG concluded that an amended taxonomy could:
a) Help the academic teaching staff support students to engage in deep learning and encourage the precise use of words to describe learning activities which makes it easier to develop assessments that reflect the level of demand of the outcomes.
b) Facilitate the design and grading of assessments and support in the planning of the course level assessment strategy.
c) Help in applying high level tasks and develop skills for critical analysis.
d) Clarify assessment-related issues for internal and external examiners by helping them understand the demands of the specific tasks, questions and criteria.
e) Enhance the consistency of module objectives and levelled learning outcomes and make apparent the precise words required to describe activities.
f) For staff to use it to generate a shared understanding of module activities and learning outcomes and level expectations with students.
In addition, as part of its development, staff wanted it mapped to the graduate attributes (GAs), so it was not perceived as ‘standing alone’.
The scope and terms of reference of the SLWG were outlined. In addition, the SLWG
campus services – for example, the availability of/labelling of Halal and Kosher foodstuffs
reviewed a range of other frameworks and taxonomies commonly used as alternatives to Bloom’s Cognitive Domain. For example, Structure of Observed Learning Outcomes (SOLO), Biggs and Collis (1982), Anderson et al. (2001), Churches’ (2008) Digital Taxonomy and a range of taxonomies used in other universities
considered Quality Assurance Agency’s (QAA) Quality Code and Framework for Higher Education qualification descriptors (QAA, 2014)
discussed views on the existing taxonomy’s usage and visibility
discussed its currency in the light of developments in pedagogy, clarity for assessing creativity (Harris, 2008), changes in professional curricula and the advent of the University’s graduate attributes of digital literacy, a global outlook and being enterprising.
There was considerable discussion about our approach to modifying the taxonomy with a recognition that the language needed to suit all subject disciplines and be understandable by academic staff, students and administrators. Group members worked in inter-faculty pairs – liaising with students and staff locally before they brought information on wording back to the main group. The SLWG representatives were asked to seek views from faculty colleagues specifically about the nomenclature of domains, whether domains needed updating or were current, useable, relevant and meaningful as they were. All comments were inputted to a shared Google document to enhance transparency.
Staff and students stated that the current grid shape of the taxonomy (with closed boxes) didn’t necessarily optimise the progressive approach of a continuum of learning complexity between levels.
The inclusion of the Pedagogy/Andragogy/Heutagogy continuum as an educational method which facilitates eventual independence and self-direction was preferred as this is linked to a model which supports optimal lifelong learning (Merriam, 2001; Blaschke, 2012).
It was decided that the domains would need to be finalised before i) the supporting text was revised, ii) supporting activities were added as exemplars, iii) additional text relating to the graduate attributes was added, iv) a possible redesign of the taxonomy to make it more user friendly and interactive took place, v) supporting staff guidance on how to use the taxonomy was generated.
The second meeting collated views from the SLWG colleagues and amendments were made to the domain in the light of the discussion. The agreed principle of the group was that no amendments to the taxonomy would be made unless there was i) strong evidence and a clear evidence-informed reason for doing so, ii) that the majority of the group membership agreed to the change, iii) that the justification for doing so was reasoned and enhanced the clarity and currency of the domain definition to suit the culture of all faculty disciplines.
Our Quality Assurance Department was consulted for advice about quality mapping and regulatory implications as part of the work of the SLWG during January 2015.
The following changes were proposed by the SLWG and ultimately applied to the updated taxonomy
a) The order of the Domains for both PG and UG became consistent with the original order in the earlier (pre 2015) UG taxonomy. (Link 1)
b) The revised PG domains document now matched the UG domains document with the addition of Technical Capabilities instead of Technical Skills as the first domain to both the UG and the PG revised taxonomies.
c) Technical Skills became Technical Capabilities in both the UG and PG taxonomy in order to widen the understanding and scope to all courses. This suits the scope of the professional courses, the fact that the former key skills underpin the newer graduate attributes and the broad remit of the overarching graduate attributes. There was also a clear sense that this change in terminology reduced the sense that the domain is focused on a mechanistic approach to higher education. The ability to have skills was regarded as important but only if they could, for example, be executed, applied, evaluated and critically examined as required by the relevant level of the award.
d) The word “Reflexion” has been removed from the Analysis and Reflexion domain. This was due to i) ambiguity and lack of broad understanding and clarity about its meaning, ii) current research that reflexion was actually part of the reflective cycle and integrated into the concept of “analysis” as “interpretation”. Interpretation has been included in the wording for this domain (Knowles & Gilbourne, 2010).
e) “Reflection” has been separated out from the existing Application and Reflection domain into a domain on its own. This is due to the burgeoning of reflective practice in the sector which encompasses all our courses at both UG and PG level, its widespread understanding by the teaching staff. As a result, it stands alone. The SLWG was happy with this as theory to practice application and understanding is an important component at each level and is clearly understood, distinctive and extrapolated in different ways by the range of disciplines and subject groups.
f) Data Collection and Interpretation in the existing UG taxonomy has been changed to be consistent with the PG domain of Information/Data Collection. “Interpretation” is regarded by the SLWG as part of the “Analysis” process and has been included in that domain. “Information” is now included with Data Collection in both the UG and PG suggested domains to reflect a wider understanding of the way we search for and gather knowledge to contribute to our choice of method for understanding of an area of study.
g) There was some discussion about the Communication domain. This will continue to be named Communication but the new text (Link 2) will be supplemented to reflect “multiple formats” (particularly to reflect the work in the arts-based courses) in order that not just “written and oral” forms of communication were included. Likewise, it was felt that digital and information literacy/skills need to be augmented in the supporting text for each level. A summary of the domain changes for the PG and UG courses is outlined in Figs 1&2 below.
Figure 1: Changes made to the Undergraduate taxonomy of assessment domains
Figure 2: Changes made to the Postgraduate taxonomy of assessment domains
The revised taxonomy (Link 2) was then mapped clearly against the criteria outlined by Bloom et al. (1956) and Anderson et al. (2001) and compared to other assessment domain taxonomies used in other universities.
The outputs from the group were pdf formats of the updated taxonomies for the UG and PG taught courses (Link 2) all searchable and available online, a GA matrix (Link 3) with creative, suggested activities for each domain and an interactive tool (Link 4) which links levelness, domain and suggestions for GA-related activity. These are now located on a new easy-to-find webpage. It has been widely promoted and development sessions have helped support staff as they refresh their courses for Periodic Review.
The SLWG had a responsibility to agree terminology which suited all disciplines, was free from jargon and easily understandable – this took time and persistence. In addition, managing the feedback process, and evaluating the iterations for the taxonomy in a collegiate and collaborative way was challenging for the SLWG. Students also reviewed the wording to check it was clear. Interestingly staff and student comments were similar focusing on the need to i) simplify the language for better comprehension for all, ii) make the domains consistent for UG and PG, iii) remove “reflexion” as a domain and, because of the importance of reflective practice, generate “reflection” as a domain in its own right.
A grid presentation was finally chosen despite acknowledgement from the group that there can be a blurring of learning between levels and a more complex picture of developing higher order thinking through the levels than a taxonomy as a classification and guide can reflect and do justice to. Gaps in the grid were retained in the presentation between levels 7 and the research degree element of the updated taxonomy. Whilst recognising there was a continuum of learning between the levels, the box grid matrix presentation was chosen to enhance staff and student clarity and support a focused understanding of the different domains specific to each level. The practical and informative interactive resource (Link 4) and effective promotion of the resources to students enhanced the dynamic and useful nature of the resource.
The updated taxonomy and its associated tools have precipitated the organisation of a series of tailored course workshops to support staff writing learning outcomes and fully explore ideas of constructive alignment (Biggs & Collis, 1982) using the taxonomy to ensure consistency in their course design and review work. This has helped heighten the focus on enhancing an approach to outcomes-based learning.
Further future work streams will be i) the populating of our GA matrix linked to the domains – this will offer staff ideas for levelled wording they can use in module learning outcomes when they are embedding the GAs, ii) workshops to help staff support their students more effectively in understanding the domains and how assessment expectations can be linked to them.
The follow-up interactive tool and grid (Links 2 & 4), led in collaboration with the SLWG by our learning technologist, will be used to clarify student expectations of assessment in induction programmes and has, in anticipation, been uploaded to the student-facing website as a learning tool.