The landscape of validity and reliability practices from applied performance analysts: Establishing a best practice framework

Whilst Theoretical Performance Analysts benefit from established validity and reliability frameworks, their practical application faces several challenges for Applied Performance Analysts (APAs). We explored the current practices and perceptions regarding validity and reliability by APAs to formulate best-practice recommendations. An open and closed-answer survey was completed by 175 APAs from a range of sports and countries, with responses analysed based on experience (early-career APAs vs experienced APAs). Findings reveal a desire for robust validity and reliability methods, particularly among experienced APAs. Collaborative approaches amongst APAs and other stakeholders were identified when determining performance indicators, yet decisions for live data collection were primarily made by APAs. Familiarisation and reliability processes varied, specifically the frequency of checks, with experienced APAs dedicating more time to learning new systems and checking data more frequently. Despite the existing theoretical frameworks, time constraints and personnel changes pose challenges for practical implementation, with APAs calling for an applied framework. These insights inform the development of a best-practice recommendations framework for collecting, analysing, and presenting accurate and meaningful performance analysis data in applied settings.