Colin Gray, Abertay University
Career guidance is a small but distinct profession concerned with supporting people to make choices about their future, to navigate the worlds of work and learning, to plan and implement lifestyles they find rewarding. Career guidance practitioners can be found working with young people or adults in state sector career services, notably Skills Development Scotland (SDS), in schools, universities and colleges, in community based settings, or in private practice. The profession has a well-established entry route, with universities offering initial training programmes leading to a postgraduate diploma. In recent years, the context for this initial training has become surprisingly complex. There are a number of factors to take into account at international, UK, and Scottish levels.
This paper aims to investigate an emerging teaching method for use in online learning with work based learners (WBLs). The method, Bite Sized Learning, will be described below, but first the context is worth describing.
Work based learning (WBL) is said to be the educating of oneself through work and for the purpose of improving the work itself (Gray, 2001). The process of WBL is described as more difficult than traditional education, primarily due to the circumstances surrounding the lives of common WBLs. Busyness is one common thread, with WBLs invariably describing themselves as 'time poor' (Nie, Armellini, Witthaus, & Barklamb, 2011). In practice, this means the life of a WBL will, on average, be more demanding than that of a traditional learner. Furthermore, Nie et al. (2011) found that WBLs travel frequently and have a desire for flexible learning which can be accessed through a mobile device. Mobile access is seen, by the learners, as a way to participate, despite a busy schedule. Flexibility is certainly a key trend, with the ultimate aim being any-time, any-place learning which has the ability to adapt to the learner (Brennan, 2005).
Bite Sized Learning (BSL) has been developed in an attempt to better meet the requirements of WBLs and provide a more effective learning experience for this group in the online space. The method is little explored in the literature, thus far, but variations of the form are in use in a small number of other contexts. For the purposes of this project, BSL is defined in the following way:
1. It is fully online, and accessible anywhere with an internet connection.
2. It is delivered in 'bite sized' chunks of learning, e.g. 30 minutes.
3. Tasks are delivered regularly and often, e.g. every day.
4. Courses run over a short overall time, e.g. one week.
5. Courses are active and social, requiring action and feedback from the participants throughout.
Three key educational theories underpin the BSL method, the first of which is just-in-time learning (Simkins & Maier, 2009). Just-in-time learning advocates course materials which are short, task based and focused on a particular outcome. Secondly, BSL seeks to benefit from social learning practices, drawing much from Lave and Wenger's work on learning communities and communities of practice (Lave & Wenger, 1991). Finally, BSL is underpinned by the work on human attention spans by Robert Pike (1994), namely his 90/20/8 rule. This informs interaction frequency and content chunking within the course.
The core investigation presented here details a qualitative study which was carried out on four instances of one BSL course. This is part of a wider study which includes quantitative work carried out within the same series of courses (Gray, 2015). The aim of the quantitative study was to discover engagement patterns with the BSL material. This study aimed to surface the wider picture by answering 3 research questions:
1. Does the BSL method promote learning in WBLs within HE?
2. Does the BSL method meet the needs of higher education WBLs?
3. Does BSL have advantages over other methods of WBL in higher education institutions?
If research questions 1 and 2 can be answered in the positive, then BSL can be considered an effective option when teaching WBLs in this context. If research question three can also be confirmed, then BSL can be considered an improved method of teaching WBLs in the given context.
The outcomes of this study will provide guidance to educators on how the BSL method can be used more generally and what advantages it might bring to their teaching. It will also inform development of the method in future by other researchers.
This study was carried out on four instances of the course In the Cloud, delivered to academic staff at Edinburgh Napier University between 2012 and 2014. In the Cloud (ITC) was a 1 week course focused on using cloud applications to aid in technology enhanced learning practice. One task was delivered each day on subjects such as using Google Drive and Evernote as teaching tools.
This project was conducted with an Action Research ideology in mind (Avison, Lau, Myers, & Nielsen, 1999). Action Research is an approach heavily grounded in practice and intends to explore and to inform (Wisker, 2001, p. 122). It is commonly used in workplace settings, industry or education, where the practitioner intends to solve a problem or test a hypothesis with the aim of improving their own practice. It is an effective method for researching one’s own work through practical research, and improving that work in future.
The aim of this project is to answer the research questions stated within the introduction. This activity will test the claim that BSL is a method of teaching WBLs which is more effective than traditional methods of WBL in this context.
In order to gather data to investigate the research questions, surveys were run following each of the four instances of ITC. These surveys aimed to investigate the effectiveness of the course, the teaching method, and a request for improvement suggestions. The key aim was to explore students’ feelings on the learning experience and to explore possible reasons behind the trends discovered by the quantitative analysis.
Four surveys were run in total. Details of the survey approach and results are shown in Figure 1.
|
Date |
Survey Sent? |
Participants |
Survey Responses |
Code |
|
February 2013
|
Yes
|
15
|
9
|
ITC-2
|
|
September 2013
|
Yes
|
17
|
4
|
ITC-3
|
|
November 2013
|
Yes
|
7
|
7
|
ITC-4
|
|
January 2014
|
Yes
|
15
|
12
|
ITC-5
|
Figure 1 Details of the surveys run within ITC courses
The survey was sent to 54 learners in total. Thirty-two people responded, giving a total response rate of 59%.
When coding the survey responses, the thematic analysis method defined by Braun and Clarke (2006) was followed.
It has been stated that the details of Thematic Analysis, and coding in general, are often glossed over in the literature (Tuckett, 2005). Studies will state that a particular method has been followed but not cover the detail of how the analysis was carried out. For the purposes of transparency and rigour and to help develop a more detailed picture of this type of analysis within the field, the methods used within this project will be covered here, in detail.
First, the four survey datasets were exported from Google Docs and Ultimate Survey and transferred to Excel for standardisation (e.g. each participant was given a number and columns were headed with question numbers). Next, the data were imported into NVivo, each survey dataset forming one source within the package.
Phase 1 of Braun and Clarke's (2006) process was followed by reviewing the survey questions and then skimming the answers of the first survey (responses from the February 2013 cohort of participants) to become familiar with the data. These survey data (again, February 2013 responses only) were then analysed in detail in Phase 2, and a set of initial codes were generated based on phrases and concepts that appeared to be related to the research questions. By the end of Phase 2, a set of initial codes had been uncovered, based on the February 2013 data, and they were present in a non-hierarchical form within NVivo. Phase 3 involved searching for themes, and this is where codes were grouped together to form those major and sub-themes. Phase 4 then involved a process of reviewing those themes, and, in the first round, led to further reorganising. Phase 5, finally, involved creating definitions for the resulting themes which would better inform future rounds of this process.
The next stage was to move on and repeat Phases 2 and 3 for the next set of survey data, the September 2013 dataset. In Phase 2, codes were generated based on relevant concepts which appeared in the survey data. When these codes fit existing themes, they were integrated, and when they didn't, they were placed on the top level of the coding tree. Then, in Phase 3, the themes were examined again and modified based on the new codes. This often involved reorganising and refining themes as new concepts were integrated, or entirely new themes were created to accommodate codes if necessary.
This author found that it was worthwhile repeating Phases 2 to 5 for each of the four course instance datasets shown in Figure 1. Phases 2 to 4 form the main work in generating and refining data, while Phase 5 provided reflection on the work done so far and forced critical thinking in the generation of a definition. This repeating cycle formed the iterative process that is often described within Grounded Theory development and Action Research in general.
In the case of this project, this work resulted in four iterations of the Phase 2 to 5 cycle, characterised by the coding of anything between four and 12 individual survey responses. In other projects this could be replicated by splitting research artefacts (one full participant survey submission in this case) into groups of around ten and repeating the cycle for each ten surveys. This author feels that, using this procedure, later cycles may be carried out with larger groups of artefacts as the codes and themes become more refined. The benefits of this approach, found during this work, are that it seems to reduce the risk of erratic findings somewhat. Codes are discovered and developed initially, but are then refined through a number of cycles. This seems to produce a natural process of review which may develop more robust coding structures.
This section will report on the four major themes which emerged during thematic analysis and which best address the research questions stated in the Introduction.
Each theme description which follows begins with a ‘means’ statement. This is the first part of thematic analysis described by Saldaña (2009). Each ‘means’ statement is intended to state the meaning of the code within the context of this project. This may differ from the literal meaning of the word or phrase in English or may define only a sub-set of that phrase. This is felt to be valuable because codes must be created in common language, but often that language may not be precise enough to convey the true context of a code. The meaning statements move towards defining what that code represents within this research.
The Course Format MEANS the effect that the structure and delivery method of the course had on learning effectiveness for the students.
This theme is primarily concerned with the bite sized learning format and the opinions learners formed around it. It is also concerned with drawing out the positive and negative effects the format had on learning effectiveness, and changes that could be made to improve the structure based on learner feedback.
Course format was the largest theme to emerge from this project, with 86 references. This is to be expected given that the format, BSL, is the main subject of the research questions.
The task length sub-theme contains any reference to the length of each individual task and was the most prevalent aspect of Course Format, with 35 references.
As an initial validation of the BSL hypothesis, participants commented on the fact that the short, 30-minute tasks were a desirable element of the course: “the 30m a day advert for course was attractive.” This demonstrates that one of the key ideas behind BSL – accessible, bite sized tasks – is something that attracts WBLs to learn.
Some comments also demonstrated the factors that were attractive to the learners: “short daily tasks over a week or so definitely motivate me to plug away at it and not have a big mental barrier about participating.” This suggests that the bite sized nature of BSL may help to negate the inertia in taking part in learning, allowing quick, barrier free chunks of learning to take place.
The fact that short tasks were particularly attractive to learners is backed up by the number of comments referring to the contrary, that the tasks ended up taking longer than the 30 minutes promised: "I found the tasks took longer than an hour a day."
It is obvious from this feedback that the time estimation for the material was, in many cases, too low. The number of responses concerning this highlights the fact that this 30 minute task promise was a major draw to the course. This moves towards answering research question 2 in the positive: Does BSL meet the needs of WBLs? The short task structure is indicated as a very desirable element; therefore, if this was implemented correctly, it will start to meet the needs of WBLs.
Another sub-theme of Course Format, which is Duration of Learning, refers to the duration of the course as a whole. In the case of ITC, the duration was 1 week, with one task being delivered on a daily basis. Within this theme, almost every reference suggested that the time period was too short.
One participant stated that the course was, “...a bit fast and furious,” suggesting that a task every day proved too much to keep up with, even when the task is bite sized. Five of the 27 references within this sub-theme suggested “2 weeks” as a specific time period, and some comments clarified this by stating that a task every 2nd day may be a more sustainable pace.
On the daily tasks themselves, there were a large number of positive comments relating to the daily delivery concept, including: “Keep on with the short daily tasks,” and, “The daily tasks that built on what was done the day before also made it easier to manage.” This suggests that daily delivery is desirable and may offer advantages, such as building on concepts at a regular pace. This is backed up by a reference within the Task Length sub-theme, stating that the short tasks “definitely motivate me to plug away at it.” This concept of plugging away demonstrates one of the advantages of daily tasks – the rhythm and habit that is achieved when something short occurs every day.
In the context of ITC, it is clear that people struggled to keep up with the material, even though they see the daily delivery method as advantageous. As shown within the Task Length sub-theme, it is possible that difficulties were mainly attributable to over-long task lengths. This means that positive comments directed towards the daily delivery method may be taken as validation of the format’s effectiveness, despite the number of negative comments that accompany them.
Flexibility MEANS offering options to the learner in HOW they learn.
This theme refers to the flexibility offered to the learner in how they take part in a bite sized learning course. This falls into a number of sub categories, including time of participation, location of participation, order of consumption, method of participation and pace of participation.
Flexibility was the second most common theme found within this coding process (46 references). The theme is closely related to Course Format as it is built into the BSL course format itself, and is one of the prime considerations of the design. Therefore, some elements of the Course Format theme parallel or overlap with the theme of Flexibility.
As discussed in the Introduction, work based learners value flexibility as a key attribute in any learning they undertake (Brennan, 2005). For this reason, when BSL was designed, flexibility of learning was a core consideration. Therefore, it was to be expected that this theme would arise when discussing learning with a group of work based learners in a BSL context.
When asked about advantages in the BSL format, flexibility, in general, was mentioned more often than any other concept (14 references). References ranged from a simple one-word answer, “flexibility”, to more detailed phrases, “Delivering the course in this flexible manner.” These references are useful as an indication of positivity towards the course format, and the frequency of appearance seems to confirm the choice to prioritise flexibility in the design of BSL.
Time flexibility was the most numerous of the flexibility sub-themes (22 references). These comments centred on the ability to take part in a course “at a time that suits me”. The constant appearance of “time” demonstrates how this is a constant barrier or concern to work based learners.
Respondents mentioned work and schedule often, and the main advantage they found in bite sized learning is that it can “fit around my work tasks”. The barrier presented here is well demonstrated by the following quote: “I constantly miss Staff Develp [sic] things which are on when I teach.”
A related element of time flexibility is that of length and frequency. This quote demonstrates the concept: “It meant I could...fit it in to a stray ten minutes here or there. I like to do a bit of work then reflect on it later so it suited me well.”
As in many areas within this study, the quote is intrinsically comparing BSL to larger blocks of learning; the traditional method for work based learning, where a learner will be occupied in one block of learning for an hour, a half day or even a full day.
One quote reflects the desire to learn in very short bite sized chunks of learning, and appreciated the ability to take time to reflect, before moving on to the next day's concept (“..fit into stray ten minutes…then reflect…”). This reflection time is much harder to achieve in traditional workshops where each student, arguably, may need more or less time to do so. Furthermore, the previous quote (“I constantly miss Staff Develp [sic] things which are on when I teach.”) stated that a participant will often be entirely unable to attend this type of learning intervention, simply because it will not fit within their schedule. In the latter case, BSL allows the learner to participate, when otherwise they would not, and the former demonstrates where BSL has an advantage over these traditional methods when it comes to WBL.
This sub-theme contains any reference to delivery of all course materials at the beginning of the course period. This is opposed to drip feeding material in the daily delivery format favoured by BSL. This is similar to the Buffet Model of course delivery (Twigg, 2003), which allows full choice to the learner in when and how they interact with material, thus allowing the learner to draw from the buffet of learning as they choose.
Participants enjoyed the fact that they could go back and consume material in a different order, or catch up if required: “I was able to go back to the bits I missed.” This suggests that bite sized chunks lend themselves well to a more flexible or adaptable learning sequence.
On the other hand, the following comment suggests that the learner would prefer a full choice of materials at the beginning: “...my strategy was to pick and choose what information I found useful and leave anything that I didn't think would be useful for me.” This concept was supported by a number of other references which stated a similar idea.
When aiming for maximum flexibility, the daily delivery model is at least partly in conflict with that aim. It is expected that learners will participate every day, although they have a choice in when and where. This raises the question over whether all materials should be delivered at the start of the course to offer further flexibility. In the Course Format section, the daily delivery format was found to have a range of advantages, and so this is one area where further investigation would be particularly valuable.
Measures of Success MEANS phenomena which suggest effectiveness on the part of the course.
This theme intends to uncover evidence of success on behalf of the BSL format. This is key in answering all three of the research questions stated in Introduction. Kirkpatrick's four-level model of training evaluation (Kirkpatrick & Kirkpatrick, 2006) informed the success measures detailed here, particularly levels one to three. The four levels are shown here for clarity:
|
1. Reaction – How do participants react to the training?
Think in terms of perceptions of value, feelings towards the instructor,
thoughts on presentation, etc.
|
|
2. Learning – How much knowledge have the participants
gained as a result of the training?
|
|
3. Behaviour – How are trainees applying their training in
ways that change their common behaviour?
|
|
4. Results – Did the training meet the aims that were set
down at the beginning? This may be anything from increased employee
satisfaction levels to improved sales.
|
Figure 2 Kirkpatrick’s Four Level Model of Training Evaluation (Kirkpatrick & Kirkpatrick, 2006)
This sub theme collects positive comments around the course, particularly related to Effectiveness, Useful and Personal Attachment. Effectiveness and Useful collated statements which referred to those words in particular, or suggested the concept. Personal Attachment collected comments on how the course related to the individual in a positive way; for example: “it worked well for me.” The references which are contained within this sub-theme all provide evidence of a positive reaction to the learning materials, demonstrating success at Level 1 of Kirkpatrick's model.
Behavioural change is placed at level 3 of Kirkpatrick's model and is a strong indicator of learning. It indicates that concepts or ideas have been fully integrated by the learner, changing their operating patterns long term, as opposed to simply complying with course tasks in order to complete. Therefore, if there are measurable changes of behaviour in BSL participants, this suggests that learning is taking place.
Ten references in total indicated some type of change of behaviour attributable to taking part in ITC. For example: “Since getting to know how Evernote works, I now open it up when I come into the office”. Measuring skill in the use of Evernote is difficult, but this quote evidences a regular change in behaviour as a consequence of course participation. Similarly, another participant reflected that they have “embraced Evernote in a big way in the last fortnight.” This is clear evidence of a change of behaviour, and success at Level 3. Furthermore, the first statement, “Since getting to know how Evernote works,” indicates that the participant has achieved one of the key course outcomes: Ability to use Evernote in your own work. This confirms success at Level 2 of Kirkpatrick's model, specifically aimed at learning assessment, and so reinforces the learning indicated by the numerous references to changes of behaviour.
These indicators of success are not confirmation that BSL is more effective than any other mode of learning, but they do suggest that BSL is an effective teaching method. When this indication of effective learning is paired with the advantages of other BSL specific themes, such as Flexibility and Course Format, it provides validation for the format as one which is an effective teaching method with advantages over alternative online course formats in a WBL context.
The Priority of Learning theme contains all references to the priority participants give the learning activities throughout the week.
Priority is a prevalent reference, with many learners stating a similar view to this quote: “My engagement dropped off towards the end because I suddenly had too much work to do and the course was lower on the priority/urgency list.” Priority sometimes is not mentioned specifically, but phrases such as, “I think I'll do it later and then don't get there,” suggest that it remains low priority compared to other tasks.
This highlights a particular disadvantage of online learning, which may be even more important with respect to BSL: flexible, distance learning may not carry the same attention and urgency as face-to face-learning. This quote demonstrates these thoughts: “It was sometimes more difficult to clear out 30 minutes at the office – compared to physically attending a course when you are away from the office environment.”
In a distance learning context, the flexible nature of the task causes it to be lower in priority, simply because it can be completed at any time. “...It's easier to push to the side … than a single block!!” as one participant said, comparing the small flexible tasks to one solid face-to-face learning session.
The daily delivery method within BSL is an attempt to overcome this disadvantage, creating a higher priority for smaller chunks of learning, e.g. the idea that this needs to be done today and it won’t take long. But a disadvantage may come in the form of quantity of effort. This effort has to be made every day for five days, as opposed to the one-off effort of getting up and attending a single longer learning session. Furthermore, short tasks may be easy to de-prioritise, or to put off, because they may be perceived as less valuable simply due to volume of content. They may also be perceived as easier to catch up with, even though the references around falling behind and catching up prove this not to be the case.
The purpose of this paper is to present an investigation into the effectiveness of the Bite Sized Learning (BSL) format of course delivery. The BSL format (one or two week course period, online based and comprising short, daily tasks) is designed around the needs of work based learners, which mainly stem from their time-poor nature (Nie et al., 2011). These needs centre on flexibility, both in time and location (Nie, et al., 2011; Brennan, 2005).
This section aims to discuss the lessons learned around how effective the format is as well as strengths and weaknesses found within BSL. This will enable the reader to evaluate the method's value in their own context. Furthermore, suggestions for improvements to the model will be made based on these findings which will be useful to any researchers who wish to replicate and develop the method.
In the context of this research, the measures of success theme addresses research questions 1 and 3, stated in the Introduction, and moves towards validating BSL as an effective, and sometimes advantageous, teaching method.
When changes of behaviour and opinions around effectiveness and usefulness are evidenced, it indicates the method is a legitimate promoter of learning, answering research question 1 in the positive.
Next, feedback on the unique elements of the course format starts to suggest whether the method is more effective than others. For example, participants state that the flexibility present in BSL allows them to participate when standard face-to-face workshops wouldn't, and this is followed by evidence of a change of behaviour in some participants. The combination of these outputs suggests that BSL is a convincing learning method which is more effective for those learners than other methods. This provides a positive answer to research question 3 within the higher education context of ITC, and versus traditional one block of learning methods, either face-to-face or online.
It came to light during thematic analysis that there were advantages to the short task, daily delivery format and that the format, in general, “worked well” for many WBLs. But, in parallel, time flexibility was one of the most prevalent sub themes within the study. This produces a degree of conflict between two of the key features of the BSL format.
To take flexibility first, the prevalence of positive references to flexibility justifies the focus on this in the design of BSL, as predicted by the literature. This finding provides strong evidence for a positive answer to the question: Does BSL meet the needs of WBLs? In contrast, a number of references suggest that flexibility is a contributor to a lack of prioritisation in learning, enabling it to be “pushed to the side” more easily than scheduled blocks of learning.
With regard to the short task, daily delivery format, references suggest that this structure “worked well” and provides effective motivation to participate regularly and build a habit of learning, as opposed to more flexible online courses where everything is available at once. The tendency in the latter is to leave a large amount of work until the last minute, leading to incompletion. Daily delivery, on the other hand, encourages regular participation in accessible tasks which allow the participant to stay caught up and, as stated by one participant, “definitely motivate me to plug away at it.”
While the problems associated with underestimated task lengths make it difficult to draw definitive conclusions, this author would suggest that a combination of flexibility, accessibility and structure may provide a structure for more successful WBL. Flexibility allows participation despite busyness. Short, accessible tasks reduce the inertia barrier in getting started. And a daily structure motivates people to stay involved and not push aside their learning.
Despite work done to mitigate the risks as much as possible, thematic analysis is always a subjective process, at least in part. The themes and codes discovered during this work may have emerged slightly differently under a different author. It is felt that the cyclical process outlined in the Process section, where codes are developed, reviewed and refined through a number of cycles, helps to reduce this concern, but it remains something to be aware of.
This study was conducted over four separate course instances. While every effort was made to deliver each course in the same manner, the delivery will no doubt have differed slightly each time, even if just in participant contributions. To mitigate the effect of variations between courses, themes were, unless stated otherwise, only developed when confirmed by multiple references and if they seemed to refer to the general course format rather than particular details of that instance.
It is hoped that this project has demonstrated the BSL method effectively and has shown evidence for the advantages it offers in engaging WBLs in the higher education context. Further work must be done, as detailed in the Future Work section, to develop this method, but it provides a theoretical basis for the creation of professional development interventions for WBLs in future.
In order to continue this work and develop the Bite Sized Learning model, this project suggests the following experiments.
Firstly, in order to truly evaluate the effectiveness of the daily tasks format, a test should be run comparing BSL to a buffet model course of the same content. This would be intended to test the suggestions of a few participants around greater flexibility, removing the daily structure constraint and allowing the student to participate through the week in the manner which they choose. This would test the effectiveness of the daily tasks in building habit and encouraging longer participation.
Next, it would be interesting to test two identical courses delivered over different durations and with different task frequency. For example, the current ITC course format could be tested against one which is 2 weeks long, and tasks are delivered every 2 days. This would assess whether more time per task would reduce dropout due to “falling behind”, whether it would encourage more social interaction, and whether it would lead to fewer concerns about asynchronous support. The experiment could also assess whether a less regular task actually loses its habit-building potency and results in lower retention.
Finally, it has been stated that flexibility may increase the tendency to “push to the side” learning and lower its priority in a busy work schedule. To resist this tendency, ways must be found to make the motivation to participate greater. One possible method for this is gamification. Using gamification, it may be possible to make learning progress much more visible, and thus rewarding, both to the learner and their peers. If it is made obvious to the learner what they have completed so far, what they have missed and what progress their peers are making, then this may raise the priority for participation in their own minds. A completest mind-set, promoted by gamification, can help to keep this priority high, as can competitiveness when comparing oneself with peers.
Gamification is simple to implement within most virtual learning environments and so will be tested within the next phase of the larger project. It can be tested within Moodle using the Checklist plugin and in any other learning environment by displaying a progress chart for each user.
Colin Gray is an online educator teaching a range of subjects, from Podcasting to Moodle. He researches how to make online learning more effective for work-based learners, particularly staff in Higher Education and entrepreneurs in small businesses – an unlikely pairing, he realises. Colin can be found at the Abertay University graduate school, or online at ThePodcastHost.com, TELTeacher.com and @elearningcolin. 1207628@abertay.ac.uk