Creating Educational Objectives

Educational objectives outline the skills and behaviors the student, resident, or fellow will be able to demonstrate after the teacher has completed a lecture, daily bedside instruction, 1-month elective, or fellowship training. Objectives should be developed for every instructional activity because they are a road map. They guide the teacher in developing an appropriate curriculum, they set unambiguous expectations for the learner, and they serve as a reference for evaluation and feedback.^2,3

Developing educational objectives involves three steps.^2,3 First, using action verbs (e.g., defines, explains, demonstrates, identifies, summarizes, evaluates), the instructor describes a specific behavior the learner must perform to show achievement of the objective. An objective such as “teaches concepts of airway management” is not adequate because it defines what the teacher is doing and does not clearly describe what the learner should be demonstrating. Therefore, it neither serves as a road map for the teacher or the student, nor does it identify a clear behavior the teacher can evaluate.

Second, the teacher should describe the conditions under which the behaviors are to occur. For example “given a scenario using human simulation, the student will evaluate the airway and demonstrate effective bag-mask ventilation” or “given a patient with sleep apnea, the fellow will outline a plan for management of the difficult airway.” Finally, the criteria for acceptable performance should accompany the objective—that is, “bag-mask ventilation will be followed by successful laryngotracheal intubation within 30 seconds.”

Bloom and Krathwohl developed a classification of educational objectives to assess three domains: cognitive, affective, and psychomotor.^4,5 Objectives related to acquisition of knowledge are described in the cognitive domain, objectives related to the demonstration of attitudes and values are described in the affective domain, and objectives related to the acquisition of skills are described in the psychomotor domain.^4,5

When teaching students a specific clinical skill—for example, how to manage a patient with hypotension—the teacher must establish that the learner has first mastered the lower cognitive domains, knowledge, and comprehension. Learners will not be able to initiate an appropriate treatment for hypotension or evaluate effectiveness of treatment unless they can first list the causes of hypotension and describe the effect of preload on stroke volume. The teacher must be able to identify where learners are in the cognitive domain and help them reach the higher domains such as synthesis and judgment. To accomplish this, the teacher needs to develop educational objectives asking the student to predict the consequence of an intervention or evaluate the effectiveness of treatment. Table 228-1 lists the levels of Bloom’s cognitive domain with the examples of action verbs and provides examples of questions that could be asked during lecture or teaching rounds to force the learner to higher levels.

TABLE 228-1 Bloom’s Taxonomy for Cognitive Domain

Educational objectives specifically related to critical care medicine training programs should be developed in accordance with the expectations outlined in the Accreditation Council for Graduate Medical Education (ACGME) program.⁶ In addition to listing the specific cognitive and motor skills that must be taught, the ACGME has also developed general core competencies that focus on patient care and not just knowledge acquisition.⁶ The six competencies include medical knowledge, patient care, interpersonal and communication skills, professionalism, practice-based learning, and systems-based practice.⁷ Examples of educational objectives for each competency are shown in Table 228-2.

TABLE 228-2 Educational Objectives for the ACGME General Competencies

Motivating Students to Learn

The next step in teaching as a manager is to motivate the students to want to learn. To accomplish this they must first value what is being taught. For them to value a specific goal, they need to understand why it is necessary to incorporate the material into their clinical practice.^8,9 The affective domain addresses educational objectives that relate to valuing and applying the material. The lowest level of the affective domain is receiving, in which the students attend lectures. Higher levels in the affective domain are concerned with getting the learner to incorporate material into daily patient care.⁵ These higher levels are accomplished by creating an environment that is conducive to learning. Table 228-3 lists specific activities the teacher can use to achieve higher levels in the affective domain. For example, the instructor should explain why certain educational goals have been chosen, why they are important, and what the consequences of failing to incorporate them are. Most importantly, the teacher needs to be aware of any inadvertent behaviors that may inhibit learning—providing negative feedback in front of others or demonstrating negative body language, for example. Because the teacher’s goal is to facilitate rather than inhibit learning, the teacher must recognize and change any behaviors that are barriers to learning.

TABLE 228-3 Teaching for Affective Learning

A particularly effective tool to get students not only to learn but also to apply their cognitive skills to patient care is to put them in “simulated crisis situations” and allow them to make clinical mistakes and then attempt to manage the consequences. For example, as part of the airway management course for critical care medicine fellows, they are given an opportunity to manage a simulated patient with respiratory distress. If they sedate and paralyze the mannequin before obtaining all equipment for intubations, fail to verify intravenous access for fluid resuscitation, and do not evaluate the airway for potential difficulty, they will then have to manage a hypotensive patient with inability to intubate. Making this mistake in a simulated environment and experiencing the potential complications in real time has proven successful in getting fellows to learn and incorporate their cognitive and motor skills into their patient care. Simulation technology is described later under learning experiences.

Learning Experiences

There are numerous instructional methodologies a teacher can use to achieve educational objectives. Because adult learners prefer active learning, a curriculum that requires them to process information, participate in problem solving, and defend clinical judgment increases their enthusiasm for learning.⁹

Unfortunately, traditional methods of instruction such as lectures provide little opportunity for interaction, but because they are an efficient means of conveying a significant amount of information, they are frequently used. Despite being an efficient method for the teacher, they are not as effective as other strategies in helping the learner acquire clinical skills.¹⁰ In addition, much of what is taught is not retained, especially as the quantity of new material in the lecture increases.¹¹ Finally, because didactic sessions are not interactive, the teacher does not have an opportunity to assess whether the learner understands the content and its applicability.

Small group sessions that incorporate problem-based learning and interactive workshops are more effective because they engage the students, force them to defend their decisions, and explain how they evaluate outcomes.¹⁰ Steps involved in developing a problem-based curriculum are to encourage the group to clarify any concept that is not understood, define the problem, analyze the problem, and outline a management plan.¹²

Newer instructional methodologies involve technology. Since 1992, students have had the ability to access the Internet, hyperlink to additional resources, and search for reference material with potential cost savings both in terms of dollars and time compared with traditional instruction.^13,14 Whereas surveys demonstrate that learners are satisfied with Internet-based instruction, there are no studies to show Internet-based learning is more effective than other educational methods for increasing cognitive function or efficiency of learning.¹⁵

Lectures, small group discussions, problem-based learning, and Internet-based instruction are all effective in helping the learner acquire knowledge. However, none of these methods teaches students or residents how to apply these skills to real-life situations. It is essential that a curriculum includes instruction that gives students an opportunity to learn how to manage unstable patients before they are expected to manage them in the clinical environment.

Each year 44,000 to 98,000 patients die because of medical errors.¹⁶ It is possible that giving students an opportunity to manage complex problems and anticipate consequences of their interventions in an environment where their mistakes do not result in untoward outcomes, where feedback is immediate, and where students can repeat their performance until they acquire these skills might improve patient safety.

Such instructional opportunities exist and have been available for years in the form of simulation. Simulation is defined as any training device that duplicates artificially the conditions that are likely to be encountered in an operation and may include low tech, partial task trainers, simulated patients, computer-based simulation, and whole-body realistic patient simulation. Since the 1960s, simulators have been used to teach crisis management to personnel in military, aviation, space flight, and nuclear power plant operations.¹⁷ Work in cognitive psychology and education theory suggests that more effective learning occurs when the educational experience provides interactive clues similar to situations in which the learning is applied.¹⁸ In other words, teaching management of unstable patients in a simulated environment, providing instruction, and evaluating learning is more effective than didactic sessions.

What initially began as computerized software with a separate torso apparatus has evolved into complex whole-body computerized mannequins with a functional mouth and airway, allowing bag-mask ventilation and intubation.^19,20 The chest wall expands and relaxes; there are heart and breath sounds and real-time display of physiologic variables including electrocardiogram, noninvasive blood pressure, temperature, and pulse oximetry. The human simulator has individual operator controls for upper airway obstruction, tongue edema, trismus, and reduced cervical range of motion. These computerized human simulators require trainees to integrate cognitive and psychomotor learning along with multisensory contextual cues to aid in recall and application in clinical settings.^21,22 This type of simulation has been successfully incorporated into curricula to teach management of obstetrical emergencies, management of difficult airway in the operating room, crisis management in the operating room,^20,23 and management of unstable patients for critical care medicine trainees. Examples of learning objectives for third-year medical students, fourth-year medical students, and critical care medicine fellows using the simulator are listed in Tables 228-4 to 228-6. Note, all objectives are written in terms of behaviors the student must perform, thus giving the teacher clear guidelines for evaluation.

TABLE 228-4 Learning Objectives for Third-Year Critical Care Medicine Course

TABLE 228-6 Learning Objectives for Critical Care Medicine Fellows

In addition to providing the learner with the opportunity to practice specific scenarios such as those outlined in Table 228-5, the simulator can be used to teach crisis management skills.²⁴ Gaba and colleagues recognized the similarities airline pilots and physicians face during crisis situations.²⁴ To bring order to the chaos that often accompanies a crisis, the team leader, whether he or she is an airline pilot or a physician, must demonstrate specific behaviors to effectively manage the situation. The leader must clearly identify himself or herself as the leader and be exempt from any responsibility other than providing orders. For example, when team leaders become involved with other activities, such as inserting intravenous catheters or performing laryngotracheal intubation, they lose oversight of the entire crisis. The leader must demonstrate effective communication skills by assigning specific responsibilities to specific team members. Identifying the nurse who will administer 1 L of normal saline wide open is more effective than asking someone to start some fluids. The leader should identify the essential members and ask nonessential personnel to step back. Finally, the team leader needs to “close the loop” by asking members to report when a specific task has been completed. Studies of anesthesiology residents have demonstrated that training using simulation technology can improve performance in a simulated crisis.²⁵

Another form of simulation is computer-based simulation. Recently Paladino et al. developed a computer model of human pulmonary pathophysiology which is a “simulation-based approach to Graduate Medical Education: Mechanical Ventilation SAGE-MV.” This model serves as an initial step in microsimulation technology. The program provides initial ventilation settings; waveforms of flow, airway opening pressure, and lung volume; arterial blood gases; and mean arterial pressure. The program identifies goals of therapy and allows the learner to titrate tidal volume, positive end expiratory pressure, flow, respiratory rate, inspiratory duration, and FIO₂ to achieve these goals. Finally, the program describes any adverse outcomes, stores the success rate, and allows for all-directed learning.

Although no study has unequivocally demonstrated improvement in actual patient outcomes, there have been studies demonstrating that virtual-reality training improved surgical skills during gallbladder resection.²⁶ Finally, a randomized trial comparing a didactic curriculum versus training with simulation for resuscitation of trauma patients in the emergency department failed to show significant difference between lectures and simulation.²⁷ Despite this, organizations such as the Institute of Medicine endorse simulation as a tool to teach novice practitioners problem-solving and crisis-management skills.

Evaluation

Evaluation is an essential component of any education curriculum and should address whether the goals and objectives of the course were met. When developing an assessment tool, it is important to define what is being tested (the educational objective), define the behavior that indicates the task has been performed, select the testing method, and determine the acceptable standard for performance.²⁸ Some goals, including acquisition of knowledge, can be evaluated using written examinations or multiple-choice questionnaires. However, written examinations do not evaluate higher cognitive skills such as evaluation and cannot predict whether the learner has become clinically competent and can exercise safe clinical judgment. Written examinations lack validity unless they are simply evaluating knowledge.²⁸ In addition, they tend to reinforce surface or superficial learning by rewarding students for memorizing facts for recall.

Chart-stimulated recalls are utilized to evaluate the student’s higher cognitive capabilities. Whereas multiple-choice questions evaluate knowledge, the chart-stimulated recall requires students to defend the workup, evaluation, diagnosis, and treatment of specific cases. As with other examinations, there must be predefined scoring rules, and those conducting the oral review must be trained in how to administer and score the examinations.²⁹

Performance-based examinations can be utilized to assess clinical competency, psychomotor skills, and judgment.³⁰ An example of a performance-based examination is the Objective Structured Clinical Examinations (OSCE), which were developed by Harden and colleagues in 1975.³¹ The examinations consist of several “clinical stations,” each with its own specific educational objectives. The OSCE requires the learner to recall knowledge, outline a treatment plan, interpret a study such as an electrocardiogram, or perform a specific motor skill. These examinations are reliable and valid^32–34 and have been used to assess competency following medical school electives, for surgical and emergency medicine internships, and for licensure to practice by the Medical Council of Canada.^33–36

Some potential disadvantages of OSCEs are that they are labor intensive, they fail to simulate reality because they are broken down into separate stations, and students must rely on the person giving the examination for physical findings or response to treatment. These limitations can be overcome using the human simulator, which allows the teacher to evaluate a student’s cognitive and psychomotor skills in real time.

Checklists should be developed, and all observers participating in the evaluation should prospectively agree on what constitutes a successful performance (interrater reliability).^37,38 Because students receive immediate feedback, their analytic and evaluative skills can be assessed and, when necessary, they can be instructed how to perform the task appropriately. Both computer-controlled simulators and OSCEs have been shown to be better than written examinations in predicting whether students can solve clinical problems.³⁹ Gaba and colleagues have shown that technical skills can be assessed reliably from videotapes of the learner’s performance on the simulator; however, behavioral skills, such as clinical decision making, were less reliably assessed.²⁴

Probably the most common method of assessing clinical competency is to evaluate the learner’s performance in real-life clinical situations. Several evaluation tools can be utilized in this environment. Global rating scales are used to evaluate patient care, knowledge application, interpersonal, and communication skills. These evaluations are typically conducted in retrospect and are used to summarize a performance at the end of a clinical rotation. This type of rating has the potential to be highly subjective, and if those performing the evaluation have not been trained, the results may reflect evaluation bias and lose validity.²⁹

Psychomotor skills such as evaluation of airway management, bag-mask ventilation, intubation, central catheter insertion, and chest tube insertion are evaluated with procedure logs. Checklists should include the specific behaviors that have to be demonstrated to achieve a satisfactory evaluation.²⁹ An example of a procedure log for intubation is demonstrated in Table 228-7.

Communication and interpersonal skills can be evaluated by peers, staff, and families using 360-degree reviews and patient surveys. The 360-degree review is a tool completed by those individuals (nurses, respiratory therapists, families) working with the learner. The difficulty with this review is making sure staff understand the intent of each question, coordinating the distribution, and collecting the completed examination reviews.²⁹

Finally, patient surveys are used to obtain feedback on communication, interpersonal skills, and professionalism. They are reliable if there are 20 to 40 patient responses per student, which limits the use of this tool.²⁹

Providing Effective Feedback

The final step in being a manager of learning is to effectively utilize feedback to enhance learning. Too often feedback is used to fulfill an administrative function; it is provided as a summative report once the rotation is complete. Effective feedback enhances affective learning, and when used inappropriately or done poorly, it can inhibit learning.⁴⁰

Students want feedback: they want to know how they are performing and how their performance can be improved. Most students receive inadequate feedback during their training. Explanations for lack of feedback include a teacher’s concerns that the feedback will result in unintended consequences, will damage the student-teacher relationship, or will result in students evaluating the teacher as having performed poorly. None of these consequences will occur if the feedback is delivered correctly. Formative feedback is the only way to ensure the success of students, telling them what they have done well and, if necessary, what they need to do to achieve an educational objective. Without effective formative feedback, the behaviors go uncorrected, and the student develops a system of self-validation: “I did well because no one told me otherwise.”

For feedback to effectively change behavior without causing unintended consequences, several rules should be followed. First, all feedback should be based on how the student performed regarding a specific goal and/or objective of the program.⁴⁰ This is another reason teachers must develop clear educational goals. They serve not only as the framework for the curriculum but also as a reference for feedback. If feedback is provided in the context of specific performance, there should be no untoward consequence.⁴⁰ For example, if the goal is for the learner to demonstrate effective bag-mask ventilation with appropriate chest excursion and adequate oxygen saturation, then the goal was either achieved or it was not. This is a statement based on an objective and is not a personal affront unless the feedback contains judgmental language. Therefore, it is important not to tell the student he or she did a “terrible job.” Second, feedback must include a description of how to succeed. In the example presented, if the patient was not effectively ventilated, the teacher should suggest repositioning the head, inserting an oral airway, and performing two-person bag-mask ventilation so there is a better seal with the mask. Third, the specific behavior the learner demonstrated should be addressed and not just interpreted.⁴⁰ If students are late to rounds, do not assume they do not care or are lazy. Stating the expectation that rounds begin at 7 AM and that the expectation is for the trainee to be prepared by then assigns no judgment. Fourth, for feedback to be effective, it should be an expected component of the learning tools.⁴⁰ Students should be informed during orientation that they will receive daily feedback on their performance of the stated goals and objectives. Without successfully implementing feedback, the model of teaching described by Irby is incomplete.⁹

In conclusion, a teacher who begins every educational session with clear objectives, creates an environment where students want to learn, applies different educational strategies, evaluates learning, and provides formative feedback will help his or her students to successfully achieve the educational objectives. These guidelines are applicable for developing a bedside teaching session, a 1-month rotation, or a year-long curriculum for critical care medicine fellows.

Key Points