Validity is the structured argument that the proposed interpretation and use of scores are justified by evidence. It is not a single coefficient, report or expert opinion.
From claim to observable evidence
Define the intended score interpretation, use and consequences. Then state the warrants that must hold: representative content, appropriate response processes, dependable scoring, defensible internal structure, relevant external relationships and acceptable consequences.
Analyse the examination as a system
Inspect specifications, item-development and pretesting records, form assembly, administration, access arrangements, scoring and rating, standard setting, results, appeals and security. A weakness in one link can qualify the interpretation supported by the whole chain.
Use methods suited to the design and sample: item difficulty and discrimination, rater agreement and severity, reliability or generalisability, standard error, classification consistency, subgroup analysis and evidence of differential impact.
Report limitations and act on use
Technical reporting should state data provenance, sample sizes, missing data, methods, uncertainty, limitations and decisions taken. Results without context invite over-interpretation.
Monitor intended and unintended consequences, complaints, appeals and foreseeable misuse. Material changes or harmful score use require investigation and proportionate action, including revision of guidance, boundaries or recognition scope.