Nurse Educator's Guide to Assessing Learning Outcomes » (2nd Edition)

This guide for nurse educators shows how to develop valid classroom multiple-choice exams as part of a systematic assessment plan. The role of multiple-choice testing in assessing critical thinking is discussed, and guidelines are offered for generation of learning objectives, development of the test blueprint, creation of multiple choice items, and analysis of test reliability and individual item responses. There is also material on the language of assessment, administering and scoring a test, establishing evidence of reliability, assigning grades, and instituting item banking and test development software. The author is a nurse educator and educational assessment consultant.

Annotation c. Book News, Inc., Portland, OR (booknews.com)

Acknowledgments     iii
Preface     xiii
The Role of Assessment in Instruction     1
The Process of Assessment     1
Ethical Responsibility     3
Assessment and Self-Efficacy     3
Assessment Inadequacy     4
Assessment Competency Standards     5
The Need for a Systematic Approach to Assessment     7
Assessment Instruments     7
Summary     8
The Language of Assessment     9
Assessment     9
Measurement     10
Evaluation     11
Formative Evaluation     12
Summative Evaluation     12
Instructional Objectives     13
Learning Outcomes     13
Blueprint     13
Item Bank     14
Test     14
Interpreting Test Scores     15
Criterion-Referenced Tests     15
Norm-Referenced Tests     16
High-Stakes Tests     17
Grade     18
Test Bias     18
Reliability     19
Validity     20
Evidence Based on Test Content     21
Evidence Based on ResponseProcesses     22
Evidence Based on Internal Structure     23
Evidence Based on Relation to Other Variables     24
Evidence Based on the Consequences of Testing     25
Face Validity     25
Basic Test Statistics     26
Summary     26
Developing Instructional Objectives     27
Role of Objectives     28
Focus of Instructional Objectives     29
Stating Instructional Objectives     30
Specific Objectives     30
General Objectives     31
Learning Outcomes     32
Mastery (Performance) Objectives     34
Developmental Objectives     35
Framework for Writing Objectives     37
Number of Objectives     38
Number of Learning Outcomes     39
Taxonomies     41
Using Instructional Objectives     43
Criteria for Effective Objectives     46
Complete     46
Appropriate     46
Sound     46
Feasible     47
Relevant     47
Open-Ended     47
Delineate Student Behavior     47
Shared with Students      48
Summary     48
Assessing Critical Thinking     49
Accreditation Outcomes     49
Establishing Evidence of Validity     50
Defining Critical Thinking in Nursing     50
Sample Definition of Critical Thinking in Nursing     51
Instructional Objectives and Learner Outcomes     52
Behaviors Associated with the Characteristics of Critical Thinking     53
Possesses an Analytical Attitude     53
Demonstrates Sensitivity to Context     54
Questions Assumptions     54
Bases Inquiry on Credible Sources     55
Considers a Variety of Solutions     55
Pursues Ongoing Evaluation     56
Tri-Level Critical Thinking Objective     56
Critical Thinking Assessment Plan     57
Multiple-Choice Exams and Critical Thinking     59
Summary     61
Implementing Systematic Test Development     63
Initiating Test Development     63
Scheduling a Semester's Exams     64
Identifying the Purpose of the Test     64
Determining the Length of the Test     66
Selecting What to Test     66
Selecting the Appropriate Assessment Format      67
Weighting the Content and Course Objectives     69
Developing a Test Blueprint     71
Reviewing the Blueprint     73
Preparing Students for a Test     74
Sharing the Blueprint with Students     74
Pop Quizzes     75
Determining How Difficult a Test Should Be     75
Summary     79
Selected-Response Format: Developing Multiple-Choice Items     81
Advantages of Multiple-Choice Items     82
Limitations of Multiple-Choice Items     82
Relevance of Multiple-Choice Items     83
Item Writing Logistics     83
Style Guide     84
Electronic Item Development     84
Multiple-Choice Format     85
Stem Formats     86
Question     86
Completion     86
Item Writing Guidelines     87
General Guidelines     88
Characteristics of Effective Stems     91
Qualities of Effective Options     95
Developing the Correct Answer     103
Designing Effective Distractors     105
Characteristics to Avoid     109
Alternate National Council Licensure Examination (NCLEX) Items      115
Fill in the Blank     116
Point and Click     117
Multiple Response     117
Chart/Exhibit     118
Drag and Drop     118
Item Rationale     119
Question Difficulty     119
Framing Questions in Terms of the Nursing Process     120
Item Shells     121
Peer Review     123
Summary     123
Writing Critical Thinking Multiple-Choice Items     125
Characteristics of Critical Thinking Items     125
Sequential Reasoning     126
Language     127
Best Answer Format     129
Novel Problems     129
Applying Learning Outcomes     131
Critical Thinking and the Nursing Process     133
Summary     134
Selected-Response Format: Developing True-False and Matching Items     135
The True-False Item Format     135
Advantages of the True-False Item Format     136
Disadvantages of the True-False Item Format     136
Guidelines for Developing True-False Items     137
Characteristics of Effective True-False Items     138
Flaws to Avoid With True-False Items     141
Forms for True-False Item Writing     144
Types of True-False Items     144
Matching Exercise     148
Advantages of the Matching Exercise     149
Disadvantages of the Matching Exercise     149
Guidelines for Developing Matching Exercises     149
Summary     151
Constructed-Response Format: Developing Short-Answer and Essay Items     153
Short-Answer Item Format     154
Advantages of the Short-Answer Item Format     154
Disadvantages of the Short-Answer Item Format     154
Guidelines for Developing Short-Answer Items     154
Characteristics of Effective Short-Answer Items     155
Essay Question Format     159
Forms of Essay Questions     159
Advantages of the Essay Question     162
Disadvantages of the Essay Question     162
Guidelines for Developing Essay Items     163
Suggestions for Scoring Essay Items     164
Developing Essay Items to Assess Critical Thinking     164
Scoring Rubrics     166
Holistic Scoring Rubrics     166
Analytic Scoring Rubric     166
Developing Scoring Rubrics     167
Summary     169
Assembling, Administering, and Scoring a Test     171
Assembling a Test     171
Arranging Items     172
Editing and Proofreading     174
Formatting     175
Providing Directions     177
Reproducing     180
Maintaining Security     181
Administering a Test     182
Computer Test Administration     182
Physical Environment     184
Psychological Environment     184
Academic Dishonesty     185
Scoring a Test     190
Scoring a Classroom Exam     190
Statistical Analysis     191
Student Review     192
Summary     193
Preparing Students for the Licensure Exam: The Importance of NCLEX     197
The NCLEX Application Process     198
Development of the NCLEX     198
The NCLEX Practice Analyses     198
The NCLEX Test Plan     199
NCLEX Item Development     201
Characteristics of NCLEX Items     202
Cognitive Levels     202
Critical Thinking Ability     202
Alternate Formats     202
Characteristics of the NCLEX     203
CAT Format     203
Passing Standard     204
Logits     205
Examination Length     205
Preparing Nursing Students for the NCLEX     207
Curriculum Focus     207
NCLEX Test Plan Consideration     207
Classroom Test Development     208
Cross-Referencing the NCLEX Test Plans     209
Cross-Referencing for Integrated Processes     210
Standardized Examinations     211
Progression Policies     213
NCLEX Predictor Exit Exams     213
Facilitating Student Success     217
NCLEX Prep Courses     218
Summary     219
Establishing Evidence of Reliability and Validity     221
Reliability     222
Measures of Reliability     222
Factors Affecting the Reliability Coefficients of Classroom Exams     226
Homogeneity of the Test Content     228
Measurement Error     231
Validity     234
Content Validity Index     234
Interpretation of the Validity Coefficient     235
Summary     236
Interpreting Test Results     237
Overall Test Data Analysis      238
Measures of Central Tendency     239
Measures of Variability     241
Reliability Coefficient     242
Standard Error of Measurement     243
Mean p Value     244
Mean Biserial     245
Score Distribution     245
Histogram     246
Individual Item Analysis     248
Assigning Test Scores     251
Flawed Items     251
Returning Scores to Students     253
Summary     253
Laboratory and Clinical Evaluation     253
Relevance of Objectives     253
Elements of the Clinical/Laboratory Experience     254
Identification of Clinical Action Elements     255
Identifying Objectives and Outcomes     256
Assessing Learning Needs     256
Planning Clinical Learning Experiences     257
Instructing and Evaluating     257
The Professional Role     259
Designing and Using a Performance Tool     259
Using a Taxonomy     259
Assessment of Mastery Learning     261
Assessment of Developmental Learning     263
Portfolio Assessment     265
Implementing a Clinical Evaluation Tool      266
Summary     267
Assigning Grades     269
Grading Principles     269
Philosophy of Grading     271
What Meaning Should Each Grade Symbol Carry?     271
What Should Failure Mean?     273
What Elements of Performance Should Be Incorporated in a Grade?     273
How Should the Grades in a Class Be Distributed?     273
What Components Should Go Into a Final Grade?     274
How Should the Components of the Grade Be Combined?     274
What Method Should Be Used to Assign Grades?     274
Should Borderline Cases Be Reviewed?     274
What Other Factors Should Influence the Grading Policy?     275
Developing a Grading Plan     275
Weighting Components of a Grade     275
Pass/Fail Grading     281
Adjusting Grades     281
Scaling Grades     282
Giving Extra Credit     282
Dropping the Lowest Grade     282
Summary     283
Instituting Item Banking and Test Development Software     285
Establishing an Item Bank     286
Editing Items     287
Selecting a Test Development Software Program      291
Hardware Requirements     292
Item Banking Facility     295
Item Importing and Exporting     295
Word Processing     296
Item Classification     296
Supplemental Fields     297
Test Assembly     297
Scoring and Reporting     298
Storage of Item Date History     299
Grade Book Facility     299
Cost     300
Implementing Test Development Software     302
Appointing Test Bank Administrators     302
Establishing Procedures     303
Incorporating Textbook Item Banks     304
Summary     306
References     309
Appendices     319
Basic Test Statistics     319
Basic Style Guide     321
References     327
Targeting Cognitive Levels for Multiple-Choice Item Writing     329
Sample Item Stems for Phases of the Nursing Process     333
Assessment     333
Analysis     334
Planning     335
Implementation     335
Evaluation     336
Steps for Implementing a Systematic Assessment Plan     339
Relationship of Critical Thinking Characteristics to the Nursing Process     341
Index     345