Sharing stories is so important for developing greater community and researcher awareness and capacity for working together. So when we heard about a workshop involving health consumers in the analysis of research results a couple of months ago, we asked if the researcher and health consumers involved would be happy to talk about the experience.
In part 1 of this series, we speak with Dr Mary Dahm (@DrMaryDahm on Twitter), the researcher that led the workshop. Mary is part of the Diagnostic Informatics team headed by Prof Andrew Georgiou, based in the Centre for Health Systems and Safety Research, Australian Institute of Health Innovation (AIHI), Macquarie University. There, she is leading several research streams of an NHMRC funded Partnership project into understanding the challenges with and improving the management and follow-up from hospital test results. I asked Mary why she embarked on involving consumers in the research project, how she did it, and what she felt about the value of health consumer involvement.
It’s a long post, but Mary was so open in sharing her processes and insights, that we wanted to make sure you got them all. In Parts 2 and 3 (stay tuned in coming weeks), Maureen and Ben, two of the health consumers involved, will share their experiences and thoughts about getting involved in research generally and this analysis of research workshop .
So Mary, tell us a little about the research project?
We are partnering with NSW Health Pathology, the Australian Commission on Safety and Quality in Healthcare (ACSQHC), Health Consumer NSW and AIHI on a research project investigating how to make test result management communication and follow-up safer and more effective.
One of the 3 key areas of the project is to enhance to contribution of consumers in results management. For example, what information do consumers want? How do they want it? Are consumers involved in the way that they want to be, and if not, can we find out how they want to be involved and how to work together with them.
How did the project evolve?
The overall project is a 5 year project, and we are about half way through.
To kick it off, in 2016, we had a stakeholder forum to draw the research partners and key stakeholders like clinicians, pathology providers, policy makers, medical indemnity and health consumers together to find out what are the current issues, what is currently being done about it and if we want to change things, how will we know that our changes had an impact in 10 years time. We worked through that and the contributions in that forum informed the research questions and interview questions in the subsequent ethnographic observational study.
The study involved talking to clinicians in hospital emergency, pathology, and medical imaging departments. It also involved talking to patients in emergency departments about the way they had been spoken to about their test results on that day as well as to a more general questions about what they know about what results they and their GP can access, the usefulness of information, and what they would change if anything.
Over a year we conducted more than 30 patient interviews across multiple hospitals and what we found is that you can’t make generalisations about people. You can’t equate health or computer literacy with an actual interest in having access to or involvement in managing results.
We analysed the results, but felt it was also important to have consumers inputting into that analysis as one way of avoiding tokenism. So we held the Consumer Reference Group Workshop on 23 July to get their input into the analysis.
In preparing for the workshop, we reached out via Twitter for suggestions on how best to get consumers to provide input, which helped shape how we did things. For example, after analysing the data ourselves, I wanted to send the consumers a list of 10 topics to choose from to focus on at the workshop. But feedback suggested we should limit the list to 4 optional topics so as not to overwhelm. The feedback also informed our process for preparing people with information prior to the workshop.
How did you find the patients you interviewed for the study?
Those interviewed were simply those in the relevant emergency departments when we were there, who had had a test and been told some of the results. Clinicians acted as the gatekeepers of whom we spoke with. They identified those in a state able to talk and introduced us. We then explained what the project was about and if patients agreed, interviewed them for 5-10 minutes.
How did you find the consumers to involve in the analysis of research?
Finding consumers was actually easier than accessing time to interview busy clinicians – it is not a barrier from my perspective as there is enough infrastructure in place to help.
We started by reaching out to the consumers that had been involved in the stakeholder workshop at the start, but as we were now about 2 years on, 1 had sadly died, 2 were still around and another 3 were uncontactable, so we needed to find more.
We requested consumer representatives through Health Consumers NSW (HCNSW), which involved completing a form and HCNSW posting the opportunity through its networks. That gave us about half the people for the analysis workshop. We also found consumers at the NSW Patient Experience Symposium and through a hospital consumer engagement officer, though there was mixed success across the different hospitals with this latter approach.
When people expressed an interest in the project, I spoke to them by phone about the project and its requirements.
We worked to remove the barriers for people to participate by offering to reimburse their time and travel (e.g. taxi voucher or parking). There is no official guideline on reimbursing consumers from HCNSW, so we used the guidance from Queensland Health Consumers Queensland3. I don’t know if we would have had the same interest if we didn’t reimburse.
What were your timelines and activities relating to consumers prior to the workshop?
The first thing I did was recruit the consumers, which started about 3-4months before the event. I allowed about 6 weeks for recruitment as I wanted enough buffer to be able to find a date all the consumers could attend. I gave a deadline for expressions of interest, allowing enough time that anyone on holidays might have time to respond. Most people contacted me within a week.
Once I had identified the consumers, I sent out a Doodle poll to evaluate potential dates, making sure those dates were not on public or school holidays, or other special events.
About 3 weeks before the meeting, I sent the group a selection of 4 topics and asked them to vote which were the most important to consumers. The topics weren’t necessarily all geared toward consumers – we didn’t want consumers to just comment on consumer issues, but wanted to see what level of awareness and opinion they might have about other systemic issues.
A week before the meeting I sent by email an information pack with a 1 pager describing qualitative research and the process, excerpts from interviews on the two highest priorities topics they had chosen, and a taxi voucher for those that needed one. We also offered to send hardcopies of the information for those that preferred that.
Outside of that, we had the usual catering and event management logistics to organise.
What happened on the day?
At the workshop, we split the group of 9 consumers onto 3 tables, with 1-2 researchers per table. We taped the conversation at each table so researchers could concentrate on engaging with the consumers and topics.
We started with introductions to get to know each other, then a 15min or so introduction to the project and where it was up to.
Each table was given the qualitative research one pager previously sent out, as we didn’t ask who had completed the pre-reading. We kickstarted the workshop by doing a 10 min reading and annotation exercise, and asked people to feedback what they found, so they got comfortable with the process of annotation and generating themes.
We then ran 2 x 1hr workshops, one on each topic consumer prioritised prior to the workshop. Tables were given 2.5 pages of interview excerpts on the topic and asked to annotate the interviews and generate themes. Each table then reported back to the whole group on each topic.
Did you achieve what you hoped with the consumers?
One aim of the workshop was to develop consumer generated themes. I don’t think we quite got there. Some tables delivered more than others, but what we did get was a lot of insights. That met another aim we had – to gain insights that would either challenge, inform or validate what we researchers thought we had found.
As it turns out, there were a lot of things that we had not taken into consideration. The consumer analysis was coloured by their own experiences, which is what provides value to us as researchers. We researchers don’t have the same experiences, which is why we need to talk to consumers.
Consumers also helped fill in some missing links in our research. One of the potential shortcomings of our study is that it is very focussed on hospital test result management, where the onus of follow-up and management is traditionally shifted to GPs. But we haven’t talked to any GPs. We had consumers in the room that frequently need to go to their GP to follow-up results and they told us how they might think results get there in time, but they never do or you can’t do anything with them, or all sorts of missing pieces. Though we are unlikely to overcome this shortcoming in this study, the consumer insights meant we are able to bridge that gap a little bit.
What will happen next?
I plan to write up a small report to send to everyone for their input, and then that report will be published online (see link below as this has been completed since talking to Mary). We are then potentially interested in asking if any of the consumers may be interested in working on policy brief for two groups: health services/agencies and consumer representatives that might be sitting on high level committees with knowledge and some pull to help drive change. That’s two way translation – a top down and bottom up approach. We will also be working with Health Consumer NSW and consumers on a method paper as actually including consumers in data analysis and writing academic paper is still uncharted territory in much of health services research.
How do you think consumers felt about the workshop?
The feedback from all the participants was really good. They appreciated the opportunity to be involved in a process most had not done before, and some followed up to raise their hand to be involved in other AIHI projects if the opportunity arose.
I’ll be interested to see what Ben and Maureen tell you about their experience.
So you believe that consumers can add value in the policy space?
Yes. It is not really done very much yet. There are things that big policy organisations do that miss the point. For example, most clinical documents (like referral letters, result reports, discharge summaries) aren’t written with the patient but with another clinician in mind. That definitely needs to change now that these documents are going into My Health Records and patients are accessing them for information.
How have you found working with consumer representatives?
I find consumer representatives really good to work with in research, especially because a lot of them have training. So you have a diverse mix in terms of backgrounds, but they aren’t really representative of the broader community. You still need to talk to patients directly. I am still grappling however with how to get broad consumer diversity on a project. I only started involving consumers about 2 years ago, so am still learning.
How have you skilled yourself to work with consumers?
When I was employed to be the project lead with responsibility, among other things, for consumer engagement, I went out, read up on it, and signed up to be a consumer rep myself at a local health district. This was so I could directly understand the experience (and as a way to give back myself). I did the HCNSW consumer training which gave me a better understanding and appreciation of what we are asking of these people. As a consumer representative, I often get things to review at a day’s notice, and I think ‘Aargh, I have work to do. I really want to help you, but I can’t review your 30page research proposal in 2 hours!’
You can of course use your personal empathy to understand what consumers encounter, but its not like putting yourself directly in their shoes. As a consumer, you get sent things, you provide comment and then don’t hear about them for months. It’s really funny how I can change my hat. If I put on my consumer hat, I can go ‘I wonder what happen to that feedback. I haven’t heard anything.’ Whereas if I put on my researcher hat I go ‘I have this and that and this to do, and I also have to ‘fit’ this stuff in as well’.
I don’t want to ‘fit it in’. I want to make consumer involvement a major component of what I do. But there are so many things that go into running a research project that take up so much time. I feel that we need to educate consumers about how the system works, how research is managed, where the money comes from, how long an ethics approval takes, the whole oversight of research etc.
What is the major challenge in involving consumers?
I think it is a double-edged sword having policies that require consumer input in every grant proposal as I think it can lead to inappropriate engagement and tokenism. Having a consumer read a grant application is not involvement – they should be involved in developing the questions and application. The problem is that there is no money for that.
There is no pre-funding for funding, so how do you fund getting consumer input into a grant application? You either pay for it yourself, or have access to some seed funding. But, that’s a catch 22. If you apply for seed funding and now have to show consumer involvement in your seed funding application, its never ending. That is one of the biggest issues. Research should be co-created, but there is a significant financial barrier.
I believe consumers should be reimbursed for their time. I am doing the research because I am paid to be there. But consumers come just to be there. As an example, one of the consumers took a day of annual leave to be at the workshop, which I appreciate so much. I felt terrible giving her a gift card for $178. But at least it was something to say thanks, that we want you to be involved, and you are valued.
What worked well or not on the day?
Everyone was really engaged, and enjoyed being involved, per the feedback.
We needed to jumpstart the workshop a bit, as not everyone had done the pre-reading and understood the qualitative research process of annotating and identifying themes. The same goes for developing more fleshed out themes. Some tables created mind maps and things like that to develop themes. Others provided dot points that covered what we talked about, but we needed to prompt them ‘so if you had to put that under a theme, what would that be?’ So in future, we would need to think more about how to structure it to get the feedback we want.
Having said that, it was the first time I had ever done this with a group of people that did not have a background in research, and so I think it actually went well. I learned and reflected on how I could do that better in the future. And they were all happy to be involved in the process – most had never done anything like that before. One may have had some experience in it, but others said they appreciated being given that opportunity.
As with most meetings, attendees said more time would have been good. They also enjoyed the opportunity to network with other peers.
So you now have an engaged consumer group, what’s next?
We will be sending out the draft report and asking for their comments. We’ll also be inviting anyone interested to be involved in some policy work, or writing up a research paper on what we found, and the methodological approach to involving consumers and involving them in the writeup and dissemination of the research.
There is still a whole lot more to learn. For example, if I hadn’t had people on Twitter tell me not to give them 10 topics, I would have given them 10 topics. So I need to think about our budget and how often we might be able to do this.
If you would like to learn more about purpose and outcomes from the workshop, or contact Mary, visit: https://www.mq.edu.au/research/research-centres-groups-and-facilities/healthy-people/centres/australian-institute-of-health-innovation/news-and-events/news/news/consumers-enhance-diagnostic-informatics
In Part 2 of this story, Maureen shares her insights as a patient advocate, and health consumer invited to participate in the workshop.
If you are a researcher who has involved patients in aspects of their work, we’d love to share your story. Equally, if you are a patient, carer or health consumer representative that has been involved in research, whether as a participant or partner in research, please reach out. Your stories matter.
Research4Me provides multiple avenues to learn, share experiences, and connect to participate and partner in research. We do this to help make it easier for everyone involved and to deliver more patient-relevant, faster, doable research that is a better experience for those involved. Please consider subscribing to our monthly newsletter, joining our growing list of potential volunteers, and connecting with like minded people to ask questions and share experiences and information in our Research Gamechangers Facebook group. All are welcome.
References (all websites last accessed 19Sep2018):
- Qualitative research methodologies: ethnography. BMJ 2008;337:a1020 https://doi.org/10.1136/bmj.a1020
- PubMed Health: Observational Study Definition https://www.ncbi.nlm.nih.gov/pubmedhealth/PMHT0025839/
- Remuneration And Reimbursement For Consumer Representatives Statement, Health Consumers Queensland, available at: http://www.hcq.org.au/for-consumer-representatives/resources/