Bloom’s Taxonomy

Benjamin Bloom, leading a group of educational psychologists, published taxonomy of learning behaviors in 1956 (when he was 44---just a trivia thrown at you) and ever since, educators and teachers are creating learning materials and assessments that, respectively, teach and test the learners for particular skills. This taxonomy was named ‘Bloom’s taxonomy.’

Bloom’s taxonomy identifies three domains of learning---in other words, it mentions that there are primarily three types of learning:

1. Cognitive Domain: This is the intellectual domain that deals with the thinking part of the learning. It has to do with the knowledge that a person has or gains. The skills that a person acquires in (or owing to) this domain are used more in science and commerce subjects.
   
   
2. Affective Domain: This domain deals with the attitude of the learner---this is the emotional part of the learning so to speak. It has to do with how a person feels about something, say a piece of art. The skills that a person acquires in (or owing to) this domain are used more in humanities or arts subjects.
   
   
3. Psychomotor Domain: This domain deals with the manual or physical skills of a learner. It concerns whether the learner is able to ‘do’ things; such as, creating something by hands, e.g., a sculpture. The skills that a person acquires in (or owing to) this domain are used more in sports and drama.

Bloom’s taxonomy presents a detailed study on cognitive and affective domains, leaving the psychomotor domain for future educational psychologists and researchers to explore, by giving the explanation that our education system does not have enough of the psychomotor skills to teach or assess (which I think is not true).

I have so far used Bloom’s taxonomy for the cognitive domain; and therefore owing to my limited exposure, this is the area I will touch upon in this article.

Cognitive Domain

Bloom’s taxonomy classifies cognitive thinking into six levels of complexity. Each level is signified by a noun and is hierarchically arranged. Each succeeding level builds on the preceding level and is thus more complex. Unless you have mastered the preceding level, moving on to the next succeeding level successfully is fallacious. You can use these levels as an effective framework to build learning objectives and assessments for a learning piece.

Let’s look into these levels first to understand how the learning objectives and assessments can be designed around them.

Bloom Levels

The six definitive levels (a.k.a. stages or skill sets) addressed in the cognitive domain are:

1. Knowledge: At this level, the learner is able to recall the information delivered---the learner remembers ‘who, what, where, when’ from the learning. For this Bloom level, the verbs you can use to design the learning objectives and assessments: list, identify, define, describe, recognize, select, state (remember what is taught). Example: What are the six stages of cognitive learning in Bloom’s taxonomy?
   
   
2. Comprehension: At this level, the learner is able to reword the learning in his or her own words because now he or she has understood the learning. For this Bloom level, the verbs you can use to design the learning objectives and assessments: explain, summarize, reword, understand, paraphrase, generalize, exemplify (not only recall what is taught but also comprehend and understand it). Example: Explain the need of Bloom’s taxonomy in the corporate and education sector.
   
   
3. Application: At this level, the learner is able to use or place the learning in a new environment---the learner applies the learning to outside-the-classroom environment. For this Bloom level, the verbs you can use to design the learning objectives and assessments: use, produce, solve, produce, apply, demonstrate, prepare (apply the learning to a new situation). Example: Demonstrate how you moved up the different Bloom levels for a new skill that you learned lately.
   
   
4. Analysis: At this level, the learner looks at the learning independently---the learner is able to differentiate between facts (the absolute) and inferences (the subjective or the relative; point of views by others). At this time, the learner disintegrates the learning into chunks that make more sense of the organizational structure of the learning. For this Bloom level, the verbs you can use to design the learning objectives and assessments: analyze, compare, contrast, examine, deconstruct, differentiate, infer (able to look at the learning from a critic’s and independent point of view). Example: Compare and contrast the LOTS and HOTS of the cognitive domain of Bloom’s taxonomy.
   
   
5. Synthesis: At this level, the learner creates something new. This is the time when learner reuses information of different types in a new, novel manner. For this Bloom level, the verbs you can use to design the learning objectives and assessments: develop, synthesize, create, design, compile, generate, devise (make an original piece of work, contribute something anew---in verbal or physical form). Example: Create a new classification system for the cognitive domain.
   
   
6. Evaluation: At this level, the learner judges views and works. For this Bloom level, the verbs you can use to design the learning objectives and assessments: review, judge, justify, interpret, defend, evaluate, support (wear a judge’s hat and review the content for assessment). Example: Review Bloom’s taxonomy for writing effective course objectives and assessment.

Note: Knowledge and comprehension are known as LOTS (Lower Ordered Thinking Skills) and Application, Analysis, Synthesis, and Evaluation are known as HOTS (Higher Ordered Thinking Skills).

Wording the Course Competencies and Assignments

The verbs, mentioned against each Bloom level, identify the cognitive level at which the learning material is aimed. In other words, the choice of verbs indicates the level of learning that a learner will attain after taking the learning. You, as a course writer, can use these verbs to write your course competencies, learning objectives or outcomes, and the assessments. The learner, on the other hand, can take a closer look at the verbs used in the learning outcomes (of the course syllabus) to understand what the course has to offer and what (level) he or she is expected to achieve after taking the course. So, the verbs in the learning objectives of your learning piece define what your learner will learn.

For each Bloom level, there are certain activities that are more appropriate than the others as assignments. Next, let’s look at those activities.

Designing the Course Assignments

While designing assignments or assessments, here are the activities that can be used to demonstrate the respective level of learning:

Bloom Levels 1 and 2 (lesson objective or competency is knowledge and comprehension)

1. For non-gradable assignments, you can use ‘Background probe.’ It is better to starts at LOTS at the beginning of the course; so, you can ask them questions with which they would be familiar with---ask about their educational and skill level background, any previous knowledge related to the subject that they might have---this is the background probe and covers Bloom levels 1 and 2: knowledge and comprehension. Students will write what they already know (level 1) and understand (level 2). Generally background probe is not graded and is used to check whether the students have the right knowledge and understanding; if yes, it is perfect; if no, it becomes an area for intervention---the facilitator should correct the misconceptions and the wrong knowledge.
   
   
2. For grading, you can use quizzes, MCQs, and fill in the blanks. Here, you can just ask the learners to identify or list whatever they have been taught in the introductory lesson.

Bloom Level 3 (lesson objective or competency is application)

1. MCQs
2. Essay questions
3. Short class project

You can use some discussion-based assignment in which students get to interact with each other so that they can apply whatever they have learned.

Bloom Level 4 (lesson objective or competency is Analysis)

1. Essay questions
2. Case studies
3. Class or group projects

Bloom Level 5 (lesson objective or competency is Synthesis)

1. Project
2. Portfolio
3. Performance

Bloom Level 6 (lesson objective or competency is Evaluation)

1. Reflecting on creations by self
2. Evaluating others work
3. Doing Cost benefit analysis (CBA)

You can use these broad-level assignments ideas (and more as you think apt) for designing activities that let you test the learner’s skill for a particular Bloom level.

Conclusion

Bloom’s taxonomy has found multiple uses and wide acceptance. At the same time, it has experienced lot of criticism primarily because it is the first classification of education to come up with different domains and levels of thinking, feeling, and doing. In spite of the criticisms and numerous alterations, modifications, and suggestions to the original Bloom’s taxonomy, it remains the most understood, accepted, and applied technique in the field of creating learning objectives and designing assessments. The biggest proof for it being the client requirements for designing the learning courses---most of the clients ask the course to be developed and assessed on Bloom level and I am yet to come across a client that asks to work on a taxonomy other than Bloom’s (no offence meant for the modifications to Bloom; I just want to state that it is working so far in its originally stated form).

Mobile Learning: Go Nomadic

We have been hearing a lot about it---mobile learning (mLearning). Researchers, IT Futurists, and many scholars predicted mobiles to change the way we learn 10 years ago. Ten years down the line, we have just reached the base of that learning pyramid … the one that mLearning promises.
mLearning is the learning that uses electronic media that users can carry with them anytime, anywhere, such as handheld sets, personal digital assistants, blackberry, smart phones that are enabled with portable technology. I understand that mobile has been the greatest revolution in the recent past and has got access to almost each hand. The outreach of mobiles is far greater than PCs, the present media that hosts the majority e-learning programs. Now that the basic device to dispense learning is almost with all, from a rickshaw puller to an MNC CEO, the audience profile that can be addressed through the mobile umbrella is huge. mLearning enjoys the advantages of making the learning mobile—you can access it with ease from a public bus as well as from your garden. It has penetrated the comfort zone of the young generation—this cohort is born and brought up in a world impossible to imagine without mobile devices; their comfort with these tools help mLearning work as a new and effective learning delivery medium. People who are hesitant of speaking up in traditional classroom setup collaborate effectively in a virtual environment that mLearning provides. Although the outreach of mobile sets is greater than the PCs because of the essential lower cost of the mobiles and their portability, there are some very obvious and not so obvious challenges that mobiles pose in the way of mLearning:

• Small screens: Small screen size can present lesser content at a given time and users can quickly lose interest if the learning nuggets are not small. To me, as a learner, to use my stylus of the phone often, or to keep horizontal and vertical scroll rolling almost every 20 seconds is annoying and good enough to suspend my interest in learning. The effort and time for getting one’s hands on the words of long mLearning training can be discouraging.
  
  
• Varying OSs and handhelds' specifications: If I as an organization need to prepare a mLearning program, I need to be aware of the mobiles that my intended audience will use and then develop the learning program that will run well on those OSs and handhelds. On the other hand, if I as an organization, need to buy a mLearning training for my employees, I need to ensure that the training that I buy and further deliver to the employees is compatible with their handhelds and if the disparity is too high for the handhelds, I need to buy the training-compatible handhelds along with the training module—an investment that will cost me over and above the learning module cost.
  
  
• Audience suitability: Although mobile devices are ubiquitous, their degree of use differs, depending on the comfort level that the user feels with his or her handheld. On one hand, a student might use his/her handheld for downloading/uploading learning lessons, collaborating with peers, instant messaging, and so on apart from the usual phone dialing and SMS features; on the other hand, a home maker might rather restrict his/her handheld usage to making and attending calls and SMSs. Therefore, it is imperative to identify the correct audience that can benefit from mLearning for it will not have same popularity and effectiveness among different groups, e.g. professionals vs. home makers and Techies vs. nonTechies. You need to be sensitive to the audience while determining whether to use or not to use a mLearning program.
  
  
• Learning content appropriateness: Imagine going through a 30-minute technical training on your mobile phone. How many minutes will you honestly look at the screen? That answers it all. mLearning is useful for delivering small nuggets of information, some ready-to-use tips, touch-up facts, and small training modules on product knowledge for people who are on the go. mLearning works well when coupled with other modes of instruction such as an ILT. After going through an ILT or a hard copy of information, for example, the user can be asked to submit answer to the assessment questionnaire using the mobile set or get the link for a quiz on mobile or share the experience through the mobile. Any learning that is short, reinforcement-type, meant for quick reference can be best facilitated by mLearning. mLearning can be used effectively to retrieve user information, dispense crisp information, and test the knowledge objectively.
  

Simple mobiles (without WAP/GPRS connection) can be used for mLearning wherein the SMS application can be used as a means of communicating by the users. Advanced mobiles (with WAP/GPRS connection) however will allow the uses to authenticate on relevant learning web sites, upload/download applications/documents of learning, collaborate, and complete the learning by submitting the quizzes and assignments. Such a learning application can be supported by a small portal and a learning management system to make the mLearning effective, traceable, and not surprisingly mobile.
mLearning as a concept started in early 2000s and it has come a long way since then. Like any other new technology and associated advancements, mLearning still has a long way to go—a long way to reach the comfort and impact of traditional training media.

Knowledge Management: Need of the Hour

Knowledge management is an amalgamation of two loosely defined terms: knowledge and management. Different people come up with different definitions for knowledge management. As I have understood it, knowledge management involves processes, techniques, and tools that enable the experts in an organization to identify, explicate, capture, document, and disseminate the knowledge they possess, which the organization can re-use (either in specific context or in a different context).

Important aspect to note here is that it is Knowledge that is to be managed and not data or information, which knowledge often is confused with (with that said, to say that knowledge management is confused with data management or better still information management would not be untrue either). Let’s look closer at these three terms: knowledge, information, and data. Knowledge captures the pattern that a set of information might indicate and a set of data makes information. In other words, there is a progression in the meaning, complexity, and use of these three terms: data, information, and knowledge (in ascending order). Data is without any context, it is just a statistical number e.g., if I say 5 million dollars, it is a data, without any context and therefore without any meaning. If however, I say that a medium business ZYX Consultants (I know you feel the name is reversed) is expected to have a turnover of 5 million dollars in 2008, it becomes information, which makes sense with its context and gives me some information. However, whether you can benefit or not from this information depends on whether you can get access to more of such information set and observe a relation in that information set. Continuing with the same example, if (to keep it simple) I say, that ZYX had a turnover of 1 million dollar in 2004, 2 million dollars in 2005, 3 million dollars in 2006, 4 million dollars in 2007, then one might see a pattern in the information set available and say “ZYX’s turnover is progressing linearly from 1 million to 5 million dollars for the past 5 years.” Now, the statement in quotes becomes knowledge that has context (ZYX’s turnover, over past 5 years) and a pattern (linear increase from 1 million to 5 million dollars). This knowledge (although the example given is simple and does not owe itself to multiple uses) when made available to all people in the organization (ZYX, here) can have multiple advantages. For example, sales and marketing people, among others, can claim the organization’s track record of a healthy and consistent turnover to their prospective clients; business strategy and management people can dwell into what worked for the organization and how and what can be done to see an exponential turnover instead of a linear one. This is a very simple example of what data, information, and knowledge is; how these terms relate to each other; and how basic knowledge can help people perform better at their jobs.

With the terminology made clear and assuming that knowledge as a term is specifically clear, let’s proceed further to understand why is KM at all important (the topic of the article focuses on this however it comes late in picture because I feel it is better to understand the basic stuff before going deeper into whys and why nots of KM) and what it demands.

Why KM?

KM, as a practice, started to gain momentum in 1980s and there is not a single but many reasons why KM has gained popularity and all the leading organizations have KM on their agenda, whether on a big or a small scale. Let’s look at the reasons that make KM indispensable in this challenging corporate environment:

• Knowledge Means Business: More you know, more adept you are at your business. It is true about an organization and equally true about the people that make an organization. These days businesses offer services/solutions more than ever before and an integral part of it is knowledge---knowledge that employees of the organization possess because of their experience and formal education (at times). This knowledge sometimes is captured in tangible form (documents, templates, proposals---explicit knowledge) that people produce as a part of their work products; however, most of this knowledge is captured well in the mind of the employees in intangible form (tacit knowledge). Skillful and experienced employees are a rich reservoir (I apologize for treating people inanimately; however “reservoir” is the first word that came to my mind at this point in time) of knowledge---they know what to do in which situation, which action steps to undertake to strike the deals in a better manner, how to make informed decisions, how to tweak the processes if the need be, how to manage risks superiorly in a given situation, so on and so forth---the list being endless. This knowledge when captured in their heads can do good to the organization/business only when these brains are directly involved in projects requiring these skills. Alas, they cannot be omnipresent to guide and facilitate what other employees are doing; however, their knowledge can be. The whole idea of KM that includes identifying, explicating, capturing, and disseminating this knowledge can help a wider employee base. KM system provides a framework using which lesser experienced employees (I am not talking about number of years of experience here but the richness and exposure of it) can benefit from the knowledge shared by veterans/more skilled/informed experts. Therefore, KM system enables employees to perform better and henceforth enhance the business.
  
  
• Virtual Environment: In today’s global environment, most of the businesses operate out of different locations. Distances between the employees means most of the employees will not get a chance to interact with the experts or the people who hold more knowledge in their minds. KM system however can be made available in virtual environment and thereby all the employees can learn from the knowledge of experts that KM captures.
  
  
• Reducing Workforce: Most of the organizations are making their workforce leaner (now with recession shouting aloud, this statement weighs more) to realize higher turnovers. From business perspective, this laying off however has also a downside to it. With people goes their knowledge. A system such as KM provides as a framework to capture knowledge that employees have in their minds and retrieve it as and when needed. Even if the employee leaves, the knowledge that the person had with him remains with the organization.
  
  
• Impressing Customer: Many organizations are making their KM system(s), or a part of it (which makes better sense), available to their customers. This communicates the seriousness that the organization feels about managing knowledge and offering better solutions. Impressing the customer gives you better chances of grabbing potential deals.

This is what KM gives you; let’s see what it demands of you (as an organization).

What KM Demands?

I view KM as an activity that engages soft skills (interviewing subject matter experts in a manner that aims at transforming their tacit knowledge into explicit knowledge) as well as technical skills (deciding which knowledge can be made reusable to the organization for including in the KM effort, tagging the explicated documents/proposals/templates/checklists with appropriate key words that can will be searchable, identifying suitable databases, data management tools, information management tools, knowledge management tools, and so on). These skills are the essential prerequisites for KM, an activity that is important because of the reasons mentioned above.

So, if you want your business to grow more with the same resources and make the intelligence broth available in your people’s head to all in the organization, go for KM. It is the need of the hour. The longer you wait to implement it in your organization, more knowledge you might be losing and/or withholding from your workforce, which might be unhealthy to your business. The results of a KM activity might not be very encouraging at the beginning because it is a resource-consuming activity that starts yielding results after a while; however, once a KM system is in place and your people get comfortable using it, you would be surprised at the increased efficiency, better productivity, enhanced client engagement, and last but not the least, a more knowledgeable workforce in your organization.

Understanding Kirkpatrick Model: Training (Learning) Effectiveness Measurement

How important is it to evaluate and improvise on you work? As instructional designers, we surely do evaluate and review a lot of work done by SMEs, graphic designers, content developers, and courseware engineers to add value to the training and learnings that we develop; however, how often do we try to evaluate the trainings that we develop? When was the last time you evaluated the training that you developed? Lost. Are you?

Evaluating training/learning is the last phase (E - evaluate) of the instructional design model called ADDIE. Often considered as a cost- and time-consuming process, evaluation is frequently neglected. Let’s look at some statistics to get real. Only 3% of the trainings delivered in U.S. companies are evaluated. This small figure speaks louder considering that U.S. companies spend more than 300 billion dollars in training every year. It is amazing that the claim to deliver these trainings is that they will generate or save business revenue and such a claim is easily let loose by scrapping off the evaluation phase in the instructional design model.

Trainings are always seen under the cost column in an organization’s budget spreadsheet. Unless and until, evaluation is done, this cost cannot be justified. Increasingly, senior management is asking to justify the training costs and such justifications cannot be made at the spur of the moment.

To justify the business cost of the training (which includes the idle time of the employees gone in training and the time and effort spent on training development, delivery, and maintenance), it is imperative to evaluate how business results get impacted by trainings. In addition to supporting the business need for trainings, evaluating trainings helps identify which trainings are effective from the organization’s perspective and what features can be improved in the training. It helps identify relevant and redundant trainings too.

How to Evaluate a Training?

Kirkpatrick model, devised by Donald Kirkpatrick in 1959, provides a framework for evaluating trainings. The model outlines 4 levels at which training evaluation can be carried out. Each level builds on the other --- a given level looses its meaning if carried out without the predecessor level. Time, cost, and effort to carry out the evaluation increases with each successive level. Let’s quickly look at these levels.

Level 1: Reaction: Reaction of the Learners: At this level, you assess the reaction of the learners soon after the training gets over. Get to know how the learner feels about the training. Learner’s reaction can be captured in “Happy Sheets” or “Smile Sheets,” which might contain questions such as “How do you feel about the training, Are you satisfied with the training, Was the delivery mode effective, Are you satisfied with the way trainer delivered the training, Are the facilities amiable for this training, Do you think it was an effective training?” and so on. You can give learners more room for reflecting on training by asking them broader questions on what they think could have been better and if there were any gaps in the training module they attended. This level is the most basic level at which any training can be evaluated. Although, in my opinion, this level does not help in gauging the effectiveness of a training, it surely does tell whether the training was utterly ill-designed and/or ill-delivered to take it off the training map.

Level 2: Learning: Change in Learning: This level aims at evaluating whether there has been an increase in skills, attitudes, and knowledge of the learner post training. You can make such an evaluation by conducting pre- and post-training tests. This will, however, not be a measure of how the learner will perform on the job (OTJ). It only gives an indication if the learner has learned something (skill, attitude, and/or knowledge) to remember after the training. Because this level is above Level 1 evaluation, it requires more effort and cost as compared to the level 1 evaluation.

Almost all training programs, whether technical or soft skills, can be easily evaluated at Levels 1 and 2 vis-à-vis Levels 3 and 4, which are better suited for technical skills. Also, when compared to Levels 3 and 4, training effectiveness measurements at Levels 1 and 2 require less effort, cost, and time, both on the learner as well as on the organization’s part.

Level 3: Behavior: Behavior that Learner Exhibits OTJ: At this level, you need to assess whether the learner has taken the learning a step ahead of remembering (Level 2), i.e., whether the learner has taken the learning to the job. Such an evaluation can be made, 3 to 6 months after delivering the training, by using Behavioral Sheets in which inputs can be taken from the customers, peers, and supervisors of the learner. Behavioral Sheets can contain questions such as “Has the leaner been able to demonstrate XYZ skill in his work/interaction? Is the learning reflecting consistently in the learner’s behavior? Is there any particular behavior change that the learner is exhibiting in past 3-6 months? What is it? Is the learner able to help others with the skill set that he or she has recently acquired after training? Is the earner aware of the behavioral change he or she is demonstrating?” and so on. In case you are unable to get the appropriate response on the Behavioral Sheets or other such templates, try gathering this data from interviews and/or 360-degree feedback.

One problem that I see with Kirkpatrick’s model Level 3 is that following it you can measure effectiveness of a technical training more successfully than a soft skills training, however, it is equally important to understand the effect of behavioral change post technical training as is post a soft skills training. So, how do we evaluate effectiveness of a soft skills training w.r.t. behavioral change --- the only change that we look out for after a soft skills training? Thoughts invited.

Level 4: Results: Business Results: Fourth level of evaluation assesses how far the training has been effective to contribute to the increase in the business revenue of the organization. It would involve assessing how training has impacted the organizational performance measures such as percentages, turnover, retention, wastage, and so on. It might need to involve intervention of business intelligence systems wherein performance (learner) measurement systems can be integrated into business (management) data and the data computed for the impact that the learner’s improved performance has on the business bottomline. It is not easy to make evaluations at this level because the business results are a lot dependent on the external factors and evaluating the impact of the training on business results might get clouded. Howsoever tough evaluating results at this level be, this is the only level that actually matters to the higher management and is capable to justify the training costs.

Please note that Levels 3 and 4 require inputs from the line managers --- learner’s inputs might not be adequate. These are the levels that actually determine the effectiveness of the training and whether it is beneficial for the business cause or not.
My Take

If I was to come up with a quick action plan to determine training/learning effectiveness, I would consider the following for sure:

• See how learners fare: How learners fare in the in-line exercises (of big modules) and assessments (of all modules) is a fair indication of how effective the training has been to the learner.
  
  
• Ask for their experience: The learner can be asked to share the learning experience elaborately. Learner becomes the customer here and can judge our product (the learning) in a better way. Many a times, looking at the learning from second pair of eyes helps to identify the training gaps. Once you are aware of the gaps (this part evaluates the effectiveness of the training), you can fill them up to deliver a more wholesome training (this part is the impact of evaluating the training).
  
  
• Monitor their work at regular intervals: Ask the learners as well as their peers, customers, and managers for their feedback on the skill imparted by the training at regular intervals, say 0.5, 1, 2, 3, 6 months. Ask them what is lacking in the skill that they learned in the training, how comfortable they are w.r.t. demonstrating that skill under different work conditions, and so on. This will help you evaluate how effective the training has been from learner’s perspective.I am still a bit confused as to how to evaluate training from business results perspective. To me it appears that filtering out the external factors is too big a challenge and requires careful administration of the impact of learner’s performance improvement on the business bottomline. Can an LMS do that? I am very doubtful. Suggestions, thoughts invited!

I personally feel that any training that does not contribute positively to the business results is futile, from organization’s point of view; and therefore, all trainings should be evaluated, at one level or the other, to determine their worth of delivery to the learners, by the organization.