Issues in developing valid assessments of speech pathology students' performance in the workplace

Abstract

Background: Workplace-based learning is a critical component of professional preparation in speech pathology. A validated assessment of this learning is seen to be 'the gold standard', but it is difficult to develop because of design and validation issues. These issues include the role and nature of judgement in assessment, challenges in measuring quality, and the relationship between assessment and learning. Valid assessment of workplace-based performance needs to capture the development of competence over time and account for both occupation specific and generic competencies.Aims: This paper reviews important conceptual issues in the design of valid and reliable workplace-based assessments of competence including assessment content, process, impact on learning, measurement issues, and validation strategies. It then goes on to share what has been learned about quality assessment and validation of a workplace-based performance assessment using competency-based ratings. The outcomes of a four-year national development and validation of an assessment tool are described.Methods & Procedures: A literature review of issues in conceptualizing, designing, and validating workplace-based assessments was conducted. Key factors to consider in the design of a new tool were identified and built into the cycle of design, trialling, and data analysis in the validation stages of the development process.Outcomes & Results: This paper provides an accessible overview of factors to consider in the design and validation of workplace-based assessment tools. It presents strategies used in the development and national validation of a tool COMPASS®, used in an every speech pathology programme in Australia, New Zealand, and Singapore. The paper also describes Rasch analysis, a model-based statistical approach which is useful for establishing validity and reliability of assessment tools.Conclusions & Implications: Through careful attention to conceptual and design issues in the development and trialling of workplace-based assessments, it has been possible to develop the world's first valid and reliable national assessment tool for the assessment of performance in speech pathology.

Keywords

Author Keywords

Workplace-based learning Clinical Education Competency-based assessment

Index Keywords

learning Humans Decision making workplace Quality control student reliability Literature review validity Competence Educational Measurement job performance Content analysis Speech-Language Pathology clinical assessment tool performance measurement system Rasch analysis speech analysis

Author Affiliations

The University of Sydney, Sydney, NSW, Australia

School of Humanities and Social Science, The University of Newcastle, Australia, Callaghan, NSW, Australia

School of Medicine, The University of Queensland, Brisbane, QLD, Australia

Funding & Acknowledgements

No funding information

References 10 References

1 Alexander, Helen A., Physiotherapy student clinical education: The influence of subjective judgements on observational assessment, Assessment and Evaluation in Higher Education, 21, 4, pp. 357-366, (1996)

2 Validity Standards for Educational and Psychological Testing, (1999)

3 Supervisory Process in Speech Language Pathology and Audiology, (1988)

4 Advances in Measurement in Educational Research and Assessment, (1999)

5 Barrie, Simon C., Understanding what we mean by the generic attributes of graduates, Higher Education, 51, 2, pp. 215-241, (2006)

6 From Novice to Expert Excellence and Power in Clinical Nursing Practice, (1984)

7 Expertise in Nursing Practice Caring Clinical Judgment and Ethics, (1996)

8 Evaluating the Quality of Learning the Solo Taxonomy, (1982)

9 Higher Education Research and Development Society of Australasia Annual International Conference, (1999)

10 Bond, Trevor G., Applying the rasch model: Fundamental measurement in the human sciences: Second edition, Applying the Rasch Model: Fundamental Measurement in the Human Sciences: Second Edition, pp. 1-340, (2007)

Issues in developing valid assessments of speech pathology students' performance in the workplace

How to Cite