Can adopting critical approaches to evaluation enhance the understanding and value of knowledge exchange? The power of voice and silence in evaluation

Paper’s Importance:
Knowledge exchange (KE) has been defined as ‘the wide range of activities HEPs [Higher Education Providers] undertake with partners’ (Research England 2023 a) to ensure that the knowledge they teach and create ‘can be used for the benefit of the economy and society’ (Research England 2023 b). We suggest it could be conceived as a form of HRD, ‘shaping individual and group values and beliefs and skilling through learning-related activities to support the desired performance of the host system’ (Wang et al, 2016). Given its increasing importance (Marzocchi et al, 2023; Atta-Owuso and Fitjar, 2023), we explore how KE is evaluated in a KE case study and whether its value might be enhanced through greater application of critical HRD evaluation approaches.

Theoretical base:
KE evaluation can focus on process or outcomes (Upton et al, 2014). The UK government’s current approaches predominantly assess outcomes, or the mere occurrence of knowledge exchange (Research England 2023c). Where outcome-based it tends to focus on commercialisation metrics leading to concerns that certain forms of KE may be incentivised over others (Rossi and Rosli, 2015).

A focus on commercial outcomes defined by a narrow range of stakeholders mirrors some orthodox approaches to HRD evaluation such as derived from Kirkpatrick focusing on impact on job performance and achievement of organisational goals (Reio at al, 2017). Other approaches assess alternative outcomes such as increased equity for marginalised stakeholders (Bierema et al, 2024), recognising alternative or wider stakeholder needs. Such approaches may reflect critical HRD by challenging assumptions about interventions (Corley and Eades, 2014) and/or highlighting conflicting interests (Sambrook, 2021).

Research purpose and methods:
This research assesses whether existing mechanisms of KE evaluation enhance its value, for which groups, and how that might be affected by the application of approaches derived from critical HRD. It seeks to answer:

-How do case study stakeholders evaluate the KE?
-To what extent would these evaluations be captured by current dominant mechanisms for evaluating KE?
-To what extent would these evaluations be captured by approaches derived from orthodox HRD practice?
-How might applying critical HRD approaches to evaluation increase the value of KE?

Thematic analysis of interviews with stakeholders in a longitudinal case study was undertaken in January/February 2022 and December/January 2023.

Implications for HRD practice:
The research sets out potential implications of different approaches to evaluation for diverse KE stakeholders. It supports those involved in learning from KE to select evaluation approaches that reflect on the purpose and value of KE.

Conclusions:
Applying outcome-based approaches to HRD evaluation could ensure that KE enhances the performance of the host system according to its definition of value rather than adopting government-defined value definitions, while some process-based approaches could improve performance in line with the goals of stakeholders. However, these might run counter to critical HRD aspirations which may be better served by evaluation approaches which support individual reflection on learning regardless of system benefits. Additional critical HRD approaches (Bierema et al 2024) could also provide relevant methods of evaluating KE.