×

How to Be Flexible in Group‑Based Evaluations

By Emily Preston‑Jones, Senior Evaluation Manager at M·E·L Research, with a foreword from Dr Holly-Taylor Dunn, Head of Evaluation at M·E·L Research

In 2024, M·E·L Research was commissioned by the Welsh Government to evaluate the Culture, Heritage and Sport Anti-racist Wales Action Plan (ArWAP) Grants Programme. In this article we explore how to be flexible and truly listen in group-based evaluation interviews.

This was an important, yet complex evaluation.  We knew that in order to design an evaluation that would provide Welsh Government with the insight they needed to plan future grant funding programmes, it was vital that we work closely with people in Wales who had lived experience of racism. Our Senior Evaluation Manager, Emily Preston-Jones, led on embedding lived experience into the design and delivery of the evaluation and below she shares her reflections of that journey.

Our evaluation found that the grants programme made meaningful progress toward fostering anti-racist change across Wales’ cultural sectors. Delivered through three strands—national institutions, local organisations, and grassroots groups—the programme was praised for its inclusive design, flexible management, and emphasis on lived experience. Community mentors and co-production approaches enriched delivery, while funding enabled increased engagement, capacity-building, and representation for Black, Asian and minority ethnic communities. Our report recommends scaling inclusive practices, strengthening support structures, embedding lived experience, and investing in legacy infrastructure to build on the programme’s strong foundation and support long-term systemic change.

When M·E·L Research was commissioned to evaluate the Welsh Government’s Culture, Heritage, and Sports Anti‑racist Wales Action Plan grant programme last year, I knew our success would hinge on more than just well‑crafted research materials. It required being present, listening deeply (and perhaps most importantly) leaving ego at the door and knowing we weren’t the experts.

Emily
Emily Preston-Jones, Senior Evaluation Manager at M·E·L Research

Before setting foot in Wales, we convened an expert advisory panel made up of individuals with lived experience of racism and the very barriers the programme aims to dismantle. Over several workshops, the advisory group co‑designed our evaluation approach—reviewing draft questions, suggesting culturally appropriate approaches, and flagging potential sensitivities. Their input was invaluable and greatly reinforced the importance of listening not to respond, but to understand. For example, we heard from the panel that the limited time we had with them was best used absorbing their intel, and any minutes spent reassuring that we’d already considered a suggestion were minutes wasted. Fragility or desire for acknowledgement was not the priority – and from that point on, we spent the time listening, not explaining. 

At M·E·L Research, we recognise that many participants carry the weight of past traumas—whether from violence, discrimination, loss, or something completely unanticipated—and that traditional data collection can inadvertently re‑traumatise. To safeguard well‑being, we:

By training our team on these principles, providing researcher debriefs, and designing protocols that minimise triggers, we honour participants’ dignity and gather richer, more authentic data without exploiting or extracting their very real and important experiences.

Between February 2025, I led field visits across eight culture, heritage, and sports initiatives, from weaving workshops to dance rehearsals. In total, we met over 100 community members, of whom 67 provided feedback. To accommodate different communication styles and comfort levels, we:

brands people riX6HX3gCZk unsplash
Reflection with post‑it note exercise

This mixed‑method and agile approach ensured we captured both the warmth of real‑time dialogue and the considered reflections of quieter voices.

Our research often focuses on giving a platform to communities whose perspectives are too frequently overlooked. In Wales, we worked closely with grassroots organisations serving ethnically minoritised groups, refugees, young people and older adults, adapting our materials to multiple languages and engagement styles. By co‑facilitating sessions with trusted local partners, we created spaces where seldom‑sought voices felt safe to share their experiences, challenges, and aspirations.

Remote surveys can be efficient but they lack the immediacy of face‑to‑face engagement. By sitting alongside participants during sessions, we could:

marilia castelli jX 7oDSo9 w unsplash

This real‑time interaction was invaluable, especially when participants described how the funded projects had broadened access to sports facilities or deepened their connection to their heritage within the rich cultural backdrop of Wales.

Our philosophy was simple: data collection should never feel intrusive. Through advance sharing of information posters and clear privacy notices, everyone knew why we were there. We positioned ourselves in the background, allowing the workshops, games, and cultural performances to continue in full flow. When people did choose to pause and chat, it was on their terms, often leading to the most heartfelt insights about how the grants had tangibly improved lives.

Perhaps the greatest reward of this flexible, participant‑led approach was capturing community resilience and creativity firsthand. Witnessing the community groups for myself underscored the social value of the Welsh Government’s investment.  We heard of the immense safety and restorative impact that trusted cultural spaces provide, the desire of so many communities to continue sharing, learning and celebrating theirs and others’ cultures, and the long-lasting, positive impact that access and opportunity within culture, heritage and sport can create.

By adapting our evaluation methods to the nuances of each group—guided every step of the way by those with lived experience—we gathered rich, authentic insights. And as we compile our final report, it’s clear: flexibility in group‑based evaluation isn’t just a logistical choice; it’s the cornerstone of respectful, impactful, and truly inclusive research.


Emily joined M·E·L Research in 2024 as a Senior Evaluation Manager, bringing extensive experience in programme evaluation and social research. She specialises in areas such as youth employment, the care system, working with people facing multiple barriers such as poverty and disadvantage, and the aging population. Emily is passionate about DEI and integrating participatory design into research and is motivated to advance M·E·L Research’s empowerment-focused approach to evaluation.

Whether you’re a public body, charity, or organisation looking to understand your audience, evaluate impact, or inform future strategies, we’re here to help.

Get in touch with the M·E·L Research team today via our ‘Get in touch’ form below to see how our expert researchers can support your goals. Alternatively, you can email the team at info@melresearch.co.uk.

Get in touch

    Latest News

    Cookie Overview

    This website uses cookies so that we can provide you with the best user experience possible.

    Cookies are stored in your browser and perform functions such as recognising when you return to our website and helping our team to understand which sections of the website you find most interesting and useful. They help us improve our service to you and others.