It's Now More Important Than Ever to Validate Assessments

With the shift to entirely online learning modalities—ranging from traditional eLearning, microlearning and gamification to virtual coaching and classrooms—it is imperative to consider a critical aspect of the learning process: assessments. Assessments can be broadly categorized into two types:

• Formative assessments: Often referred to as assessments for learning, designed to provide feedback and guide ongoing learning activities.

• Summative assessments: Also known as assessments of learning, aimed at evaluating learners' knowledge or skills at specific milestones.

This discussion will focus on the latter. Why is this important now? In the context of fully virtual learning, it is essential to have mechanisms that allow us to:

• Measure learners' progress throughout the learning process.

• Assess their comprehension and mastery upon course completion.

• Evaluate long-term knowledge retention at intervals following the conclusion of their training.

To achieve these objectives effectively, summative assessments must adhere to three critical principles:

• Fairness

• Reliability

• Validity

Fairness pertains to the perception of equity from the learner's perspective. While assessments need not be easy, they must ensure that all individuals are evaluated under equitable conditions, fostering a sense of impartiality and legitimacy.

Reliability reflects the consistency of assessment outcomes over time, across different versions, and within the structure of the exam itself. This is determined through standardized statistical measures that ensure uniformity and dependability in the results.

Validity, a multifaceted construct, is paramount to the integrity of assessments. While there are at least four types of exam validity, our primary focus is on content validity, which entails ensuring alignment between the questions and the course’s learning objectives. Key considerations include whether the questions are appropriately structured, if they adequately cover the stated objectives, and if they evaluate knowledge that is essential and reasonable for the employee to possess.

Furthermore, the establishment of a defensible passing score is critical. Arbitrary thresholds often fail to meet legal and professional standards, underscoring the importance of a methodical approach to cut-score determination.

Additionally, the employment of a comprehensive assessment system capable of delivering item-level data is indispensable. Such systems enable the measurement of exam reliability, facilitation of question validation, and revision of assessments as needed to maintain their relevance and efficacy.

It is imperative to emphasize that validity is not represented by a singular numerical value but is rather an ongoing process. Constructing valid assessments requires the integration of advanced modern assessment systems in conjunction with the expertise of a qualified psychometrician.