Evaluating the Item Descriptor (ID) Matching Method in a Face-to-Face and Synchronous Virtual Environment

This book expands well-known frameworks for evaluating standard setting workshops through a multi-phase study of the Item Descriptor Matching method in face-to-face and virutal contexts. At its core are the author's unique UATD framework and quantitative cut score method, with AI analysis of panel discussions and new ways to tame the CEFR.

This book turns the page on standard setting, calling for a time of change. Expanding Cizek & Earnest''s (2016) evaluation framework, it delivers a comprehensive mixed-methods investigation of Ferrara & Lewis'' (2012) Item Descriptor Matching method, using a multi-phase design across face-to-face and virtual workshops. At its core lies the author''s Unified Alignment & Test Development (UATD) framework, which embeds a unique quantitative principled cut score approach to calculate defensible, trustworthy thresholds grounded in theory. Using corpus linguistics and AI models to analyse panel discussions, the book shows how innovative methodologies enhance the validity and robustness of CEFR-linking studies. It also translates the panelists'' bottom-up and top-down strategies to tame the CEFR into innovative activities for the familiarisation stage. The result is an expanded, transparent, and forward-looking practice that strengthens the validity, fairness, and impact of standard setting.