Educational Testing and Measurement: Classroom Application and Practice » (9th Edition)

Thomas Kubiszyn is currently Professor of Educational Psychology at the University of Houston. He received his M.A and Ph.D. at the University of Texas at Austin, and he received his B.A. at the State University of New York at Buffalo.

Gary Borich is currently Professor of Educational Psychology at the University of Texas at Austin. His research areas include applied data analysis, program evaluation, and teaching effectiveness.

Practical assessment strategies for today’s challenges

The Eighth Edition of Kubiszyn and Borich’s jargon-free, reader-friendly guide continues to equip teachers-in-training with effective assessment strategies, up-to-date information, and down-to-earth guidance for meeting today's challenges in measuring student progress.

This revision includes practical, balanced, integrated coverage of the many implications for teachers of high-stakes testing, the important and controversial No Child Left Behind Act, and the 2004 revision of the Individuals with Disabilities Education Act.

The new edition of Educational Testing and Measurement enables you to:

• Prepare yourself and your students for optimal performance on high-stakes (and high-pressure!) tests.

• Contribute to valid, day-to-day assessment of student progress in today's increasingly diverse classrooms.

• Develop valid alternative assessments and accommodations appropriate for special education students.

• Keep up with the latest developments in the assessment field, including performance and portfolio assessments.

• Examine PRAXIS test preparation scenarios and questions.

• Explore real-life assessment situations.

• View an online video on The ABCs of School Testing, produced by the Joint Committee of Testing Practices.

About the Authors

Thomas Kubiszyn is currently Professor of Educational Psychology at the University of Houston. He is a member of the American Psychological Association’s Committee on Psychological Tests and Assessments, and is their representative to the Joint Committee on Testing Practices.

Gary Borich is currently Professor of Educational Psychology at the University of Texas at Austin. His research areas include applied data analysis, program evaluation, and teaching effectiveness.

Also available from Wiley/Jossey-Bass Education

A Teacher's Guide to Classroom Assessment: Understanding and Using Assessment to Improve Student Learning

Susan M. Butler and Nancy D. McMunn, SERVE, Greensboro, NC

ISBN: 0-7879-7877-9

Chapter I An Introduction to Contemporary Educational Testing And Measurement 1

Tests Are Only Tools: Their Usefulness Can Vary 1

Why We Developed This Text: Enhancing Test Usefulness 2

Technical Adequacy 2

Test User Competency 3

Matching the Test's Intended Purpose 3

Matching Diverse Test-Takers to the Test 5

Test Results and Diversity Considerations 6

Tests Are Only Tools: A Video Beats a Photo 6

Defining Some Test-Related Terms 8

Tests, Assessments, and the Assessment Process 8

Types of Tests/Assessments 10

Recent Developments: Impact on Classroom Testing and Measurement 13

Education Reform Meets Special Education Reform: NCLB and IDEIA 14

The Impact on Regular Education Teachers of the IDEIA and NCLB 15

Other Trends: Technology, Globalization, and International Competitiveness 16

Competency Testing for Teachers 17

Increased Interest from Professional Groups 17

A Professional Association-Book Publisher Information Initiative 18

Effects on the Classroom Teacher 19

About the Text 21

What if You're "No Good in Math" 22

Summary 22

For Discussion 23

Chapter 2 High-Stakes Testing 25

Comparing NCLB and State High-Stakes Testing Programs 25

High-Stakes Testing: A Nationwide Phenomenon 27

High-Stakes Tests Are Only Tools 28

Why Does High-Stakes Testing Matter? 29

Promotion and Graduation Decisions Affect Students 30

Principal and Teacher Incentives Are Linked to HST Performance 32

Property Values, Business Decisions, and Politics and HST 32

The Lake Wobegon Effect and HST 32

The History of High-Stakes Testing 33

Education Reform 33

Standards-Based Reform 33

Types of High-Stakes Tests36

Criterion-Referenced High-Stakes Tests 36

Norm-Referenced High-Stakes Tests 41

Benchmark Tests and High-Stakes Tests 41

The High-Stakes Testing Backlash 42

Is There Really a High-Stakes Testing Backlash? 44

What Do National Organizations Say About High-Stakes Tests? 45

AERA's Twelve Conditions for HST Programs 46

How Can a Teacher Use the Twelve Conditions? 48

Helping Students (and Yourself) Prepare for High-Stakes Tests 49

Focus on the Task, Not Your Feelings About It 49

Inform Students and Parents About the Importance of the Test 50

Teach Test-Taking Skills as Part of Regular Instruction 51

As the Test Day Approaches, Respond to Student Questions Openly and Directly 53

Take Advantage of Whatever Preparation Materials Are Available 53

Summary 53

For Discussion 55

Chapter 3 Response-to-Intervention (RTI) and the Regular Classroom Teacher 56

What Is RTI? 56

What if You Have Not Heard of RTI Before? 57

How New Is RTI? 57

Do Regular Education Teachers Need to Know About RTI? 57

An RTI Scenario 58

How Important Is RTI to Regular Education Teachers? 60

Can a Special Education Law Reform Regular Education? 61

How Is RTI Supposed to Help Students and Schools? 61

RTI Definitions, Components, and Implementation Approaches 62

RTI Definitions 62

RTI Components 63

RTI Implementation Approaches 68

How Widely Is RTI Being Implemented1? 71

Some Benefits of RTI 72

RTI: The Promise and Some Controversies 72

Technical Issues: Reliability, Validity, and Fairness 72

Implementation Issues 73

73

75

Chapter 4 The Purpose Of Testing 76

Testing, Accountability, and the Classroom Teacher 77

Types of Educational Decisions 79

A Pinch of Salt 82

"Pinching" in the Classroom 83

What to Measure 84

How to Measure 85

Written Tests 86

Summary 87

For Discussion 87

Chapter 5 Norm-Referenced and Criterion-Referenced Tests and Content Validity Evidence 89

Defining Norm-Referenced and Criterion-Referenced Tests 89

Comparing Norm-Referenced and Criterion-Referenced Tests 93

Differences in the Construction of Norm-Referenced and Criterion-Referenced Tests 94

Norm- and Criterion-Referenced Tests and Linguistic and Cultural Diversity 95

Norm- and Criterion-Referenced Tests and Validity Evidence 97

A Three-Stage Model of Classroom Measurement 98

Why Objectives? Why Not Just Write Test Items? 100

Where Do Goals Come From? 101

Are There Different Kinds of Goals and Objectives? 102

How Can Instructional Objectives Make a Teacher's Job Easier? 106

Summary 107

For Discussion 108

Chapter 6 Measuring Learning Outcomes 110

Writing Instructional Objectives 110

Identifying Learning Outcomes 110

Identifying Observable and Directly Measurable Learning Outcomes 111

Stating Conditions 112

Stating Criterion Levels 113

Keeping It Simple and Straightforward 114

Matching Test Items to Instructional Objectives 115

Taxonomy of Educational Objectives 117

Cognitive Domain 117

Affective Domain 120

The Psychomotor Domain 123

The Test Blueprint 123

Content Outline 125

Categories 126

Number of Items 126

Functions 126

Summary 128

For Practice 128

Chapter 7 Writing Objective Test Items 130

Which Format? 130

True-False Items 132

Suggestions for Writing True-False Items 134

Matching Items 135

Faults Inherent in Matching Items 135

Suggestions for Writing Matching Items 138

Multiple-Choice Items 139

Higher-Level Multiple-Choice Questions 144

Suggestions for Writing Multiple-Choice Items 147

Completion Items 148

Suggestions for Writing Completion Items 151

Gender and Racial Bias in Test Items 151

Guidelines for Writing Test Items 152

Advantages and Disadvantages of Different Objective Item Formats 153

Summary 155

For Practice 156

Chapters 8 Writing Essay Test Items 157

What Is an Essay Item? 158

Essay Items Should Measure Complex Cognitive Skills or Processes 158

Essay Items: Extended or Restricted Response 159

Examples of Restricted Response Essays 161

Pros and Cons of Essay Items 162

Advantages of the Essay Item 163

Disadvantages of the Essay Item 163

Suggestions for Writing Essay Items 164

Scoring Essay Questions 166

Scoring Extended Response and Higher Level Questions 168

General Essay Scoring Suggestions 172

Assessing Knowledge Organization 172

Open-Book Questions and Exams 175

Some Open-Book Techniques 178

Guidelines for Planning Essays, Knowledge Organization, and Open-Book Questions and Exams 182

Summary 183

For Practice 184

Chapter 9 Performance-Based Assessment 185

Performance Tests: Direct Measures of Competence 185

Performance Tests Can Assess Processes and Products 186

Performance Tests Can Be Embedded in Lessons 186

Performance Tests Can Assess Affective and Social Skills 188

Developing Performance Tests for Your Learners 189

Step 1 Deciding What to Test 190

Step 2 Designing the Assessment Context 192

Step 3 Specifying the Scoring Rubrics 195

Step 4 Specifying Testing Constraints 201

A Final Word 202

Summary 202

For Discussion and Practice 203

Chapter 10 Portfolio Assessment 205

206

Ensuring Validity of the Portfolio 206

Developing Portfolio Assessments 207

Step 1 Deciding on the Purposes for a Portfolio 207

Step 2 Identifying Cognitive Skills and Dispositions 208

Step 3 Deciding Who Will Plan the Portfolio 208

Step 4 Deciding Which Products to Put in the Portfolio and How Many Samples of Each Product 208

Step 5 Building the Portfolio Rubrics 209

Step 6 Developing a Procedure to Aggregate All Portfolio Ratings 214

Step 7 Determining the Logistics 217

Summary 220

For Practice 221

Chapter 11 Administering, Analyzing, And Improving The Test Or Assessment 222

Assembling the Test 222

Packaging the Test 223

Reproducing the Test 225

Administering the Test 225

Scoring the Test 227

Analyzing the Test 227

Quantitative Item Analysis 228

Qualitative Item Analysis 234

Item Analysis Modifications for the Criterion-Referenced Test 235

Debriefing 240

Debriefing Guidelines 241

The Process of Evaluating Classroom Achievement 242

Summary 243

For Practice 245

Chapter 12 Marks And Marking Systems 246

What Is the Purpose of a Mark? 246

Why Be Concerned About Marking? 246

What Should a Mark Reflect? 247

Marking Systems 248

Types of Comparisons 248

Types of Symbols 253

Combining and Weighting the Components of a Mark 254

Who Is the Better Teacher? 255

Combining Grades into a Single Mark 256

Practical Approaches to Equating Before Weighting in the Busy Classroom 259

Front-End Equating 260

Back-End Equating 260

Summary 263

For Practice 264

Chapter 13 Summarizing Data And Measures Of Central Tendency 265

What Are Statistics? 265

Why Use Statistics? 266

Tabulating Frequency Data 267

The List 267

The Simple Frequency Distribution 263

The Grouped Frequency Distribution 268

Steps in Constructing a Grouped Frequency Distribution 270

Graphing Data 273

The Bar Graph, or Histogram 274

The Frequency Polygon 274

The Smooth Curve 276

Measures of Central Tendency 280

The Mean 281

The Median 282

The Mode 287

The Measures of Central Tendency in Various Distributions 289

Summary 290

For Practice 292

Chapter 14 Variability, The Normal Distribution, And Converted Scores 293

The Range 293

The Semi-Interquartile Range (SIQR) 294

The Standard Deviation 295

The Deviation Score Method for Computing the Standard Deviation 299

The Raw Score Method for Computing the Standard Deviation 300

The Normal Distribution 302

Properties of the Normal Distribution 303

Converted Scores 307

z-Scores 309

T-Scores 314

Summary 315

For Practice 315

Chapter 15 Correlation 317

The Correlation Coefficient 318

Strength of a Correlation 319

Direction of a Correlation 319

Scatterplots 320

Where Does r Come From? 322

Causality 323

Other Interpretive Cautions 325

Summary 327

For Practice 328

Chapter 16 Validity Evidence 329

Why Evaluate Tests? 329

Types of Validity Evidence 329

Content Validity Evidence 330

Criterion-Related Validity Evidence 330

Construct Validity Evidence 332

What Have We Been Saying? A Review 333

Interpreting Validity Coefficients 334

Content Validity Evidence 334

Concurrent and Predictive Validity Evidence 334

Summary 339

For Practice 340

Chapter 17 Reliability 341

Methods of Estimating Reliability 341

Test-Retest or Stability 341

Alternate Forms or Equivalence 343

Internal Consistency 343

Interpreting Reliability Coefficients 346

Summary 349

For Practice 350

Error-What Is It? 351

The Standard Error of Measurement 353

Using the Standard Error of Measurement 354

More Applications 357

Standard Deviation or Standard Error of Measurement? 359

Why All the Fuss About Error? 360

Error Within Test-Takers 360

Error Within the Test 360

Error in Test Administration 361

Error in Scoring 361

Sources of Error Influencing Various Reliability Coefficients 362

Test-Retest 362

Alternate Forms 362

Internal Consistency 363

Band Interpretation 364

Steps: Band Interpretation 365

A Final Word 369

Summary 369

For Practice 371

Chapter 19 Standardized Tests 372

What Is a Standardized Test? 373

Do Test Stimuli, Administration, and Scoring Have to Be Standardized? 374

Standardized Testing: Effects of Accommodations and Alternative Assessments 374

Uses of Standardized Achievement Tests 376

Will Performance and Portfolio Assessment Make Standardized Tests Obsolete? 377

Administering Standardized Tests 377

Types of Scores Offered for Standardized Achievement Tests 379

Grade Equivalents 379

Age Equivalents 380

Percentile Ranks 381

Standard Scores 382

Interpreting Standardized Tests: Test and Student Factors 384

Test-Related Factors 384

Student-Related Factors 390

Aptitude-Achievement Discrepancies 395

Interpreting Standardized Tests: Parent-Teacher Conferences and Educational Decision Making 398

An Example: Pressure to Change an Educational Placement 399

A Second Example: Pressure from the Opposite Direction 404

Interpreting Standardized Tests: Score Reports from Publishers 407

The Press-On Label 407

A Criterion-Referenced Skills Analysis or Mastery Report 408

An Individual Performance Profile 412

Other Publisher Reports and Services 412

Summary 413

For Practice 415

Chapter 20 Types of Standardized Tests 417

Standardized Achievement Tests 417

Achievement Test Batteries, or Survey Batteries 418

Single-Subject Achievement Tests 419

Diagnostic Achievement Aptitude Tests 420

Standardized Academic Aptitude Tests|420

The History of Academic Aptitude Testing 420

Stability of IQ Scores 421

What Do IQ Tests Predict? 422

Individually Administered Academic Aptitude Tests 423

Group-Administered Academic Aptitude Tests 424

Standardized Personality Assessment Instruments 425

What Is Personality? 425

Objective Personality Tests 426

Projective Personality Tests 427

Summary 428

For Discussion 429

Chapter 21 In The Classroom: A Summary Dialogue 430

High-Stakes Testing and NCLB 435

Response-to-intervention (RTI) 436

Criterion-Referenced Versus Norm-Referenced Tests 436

New Responsibilities for Teachers Under IDEIA 437

Instructional Objectives 437

The Test Blueprint 438

Essay Items and the Essay Scoring Guides 438

Reliability, Validity Evidence, and Test Statistics 439

Grades And Marks 441

Some Final Thoughts 441

Appendix A Math Skills Review 443

Appendix B Preparing For The Praxis II: Principles Of Learning And Teaching Assessment 450

Appendix C Determining The Median When There Are Multiple Tied Middle Scores 460

Appendix D Pearson Product-Moment Correlation 462

Appendix E Statistics And Measurement Texts 464

Appendix F Answers for Practice Questions 465

Suggested Readings 471

References 475

Credits 481

Index 483