Unpicking the fabric of methodological challenge: Evaluating arts-based activities for people living with dementia

Arts organisations, individual artists, museum educators, care staff, and members of the public are increasingly involved in delivering arts and cultural activities for those affected by dementia in their communities. To compete on equal terms for funding with pharmacological, clinical, or psychosocial treatments and interventions, convincing and auditable evidence for the benefits of the arts is required. This can be difficult to provide. To support good quality grass-roots provision, and to ensure that high-level public health commissioning is effective and well-targeted, it is crucial that the field as a whole understands how to better evaluate the work it is doing.

There has to date been no extended study of the methodological challenges facing evaluators across the sector. This research aimed to identify and describe the methodological challenges facing evaluators as they are reported within the literature, and to explore how they are experienced by evaluation stakeholders.

The study included a large and systematically-informed narrative synthesis of the literature using a hermeneutic framework. This was followed by interviews with evaluators, researchers, artist practitioners and arts managers and commissioners. These were conducted and analysed using a critical realist informed approach to grounded theory.

Commonly experienced methodological difficulties are identified through the literature review, along with a set of sensitising concepts: value, context, ethics, meaning, and use. The metaphor of ‘fabric’ is introduced to describe methodological challenge and the way in which these concepts may be woven through it. In interview, participants described anxieties and divisions experienced in evaluation when they had to make and act on decisions about what might be valuable or meaningful. They struggled to reconcile epistemic and non-epistemic values, particularly when trying to introduce standards of rigour to their practice. Interviewees’ responses are discussed in the light of key concepts for the field of arts and dementia, including cultural value, evidence-based medicine, ethics and rights, and quality.

To help address the challenges identified for the sector as a whole, a novel values-informed approach to evaluation is proposed. This is based on the idea of arts and dementia activity as an ‘ethical practice’. Foundations for this approach are suggested, comprising attention to ideas of complexity, multidisciplinary and collaborative working, methods innovation, and an acknowledgement of the problems posed by lack of evaluation capacity and resource. Recommendations are made for practice and future research to support exploration and testing of these ideas.