In our recent major project, Be Included, the Leapfrog team have been using the social media platform WhatsApp to create a digital co-design community with a group of geographically dispersed health care professionals. The aim of this project is to co-design more inclusive and person-centred evaluation tools that adults with learning difficulties can use to evaluate the services they use. This project has been taking place in weekly cycles, where the participants have been set missions to complete and upload their results to the project WhatsApp group. These missions are based principally on activities that we would typically do in co-design workshop, but with a few adjustments we are making along the way as the project unfolds in response to the unique requirements of co-designing in this way.
The first mission asked the participants to reflect upon current evaluation practices and share these with the group. Here we asked the participants to write a statement and/or take photos to show the group how evaluation is currently done in their organisation, whilst thinking about: the physical things used for evaluation, processes and procedures, how and where evaluation takes place, the questions asked, who the evaluation is for, and what works and what doesn’t work so well.
In response to this, the Leapfrog team received an array of approaches, which included: easy-read questioners, focus groups, photo stories to illustrate specific journeys or developments, tick box forms, visualisations and posters, talking heads to gain natural and spontaneous feedback, and evaluation trees to receive written feedback on leaves that hang on the trees branches. The key insights discussed here in the WhatsApp group was a need to move away from literacy-based approaches, as well as tick box questionnaires which were found to constrain user’s responses, keeping the process as creative and flexible as possible:
‘Evaluation works if it is interactive, interesting and meaningful and not just a tick-box exercise. Key to this is that people genuinely feel that their feedback is valued and so reporting back on any findings or solutions based on what people have given is vital to maintain confidence.’
The needs of funders were also raised in regards to evaluation processes, particularly as there can be a conflict between the methods by which funders collect the information they need, and how this can make the service users feel. For example, through the use of certain kinds of deficit-based language that can have very negative connotations for users.
So, to build upon these insights further, the second mission was set for the participants, which asked them to then think about other new and inspiring ways to do evaluation. Here the participants investigated the ways in which other people and organisations do evaluation. They were encouraged to find out what other people in their field do, any other interesting examples of evaluation they have come across, as well as sharing with the group examples of the good, the bad and the ugly in terms of evaluation processes and tools.
Here the participants shared a range of alternative approaches to evaluation, discussing the use of feedback apps in conferences, Talking Mats, the use of video and photographic evidence, as well as a ‘feeling survey’ one of the participants use in their laughter yoga practice. The discussion that followed focused in on the need for quick and efficient ways of collecting evidence. Feeling pressured to provide this evidence to funders was also raised. Here the participants shared their anxiety around unrealistic expectations, where they are left with little time to actually to their job, let alone attempt to be creative with it.
‘It would be useful to develop how we can evidence happiness, wellbeing and self-worth etc as this often comes into funding… Funders are asking for an evaluation plan which is usual. But thinking of all the ways to show an increase in those intangible and variable qualities is a real challenge.’
Reflecting on both the first and second missions, the Leapfrog team developed a design challenge and presented this back to the WhatsApp group: how can we design a creative tool for evaluation that can meaningfully capture and evidence emotional impact? The next step in the project is to launch the third mission, which will ask the participants to reflect upon what has been discussed so far and to think about new ways of doing evaluation.