When a researcher has personal participation in peoples activities the researcher is using what type of research design?

Health Expect. 2010 Mar; 13(1): 86–94.

Abstract

Background Health researchers are encouraged to involve service users as partners in their research. There is a need to increase the evidence base of involvement, including an accumulation of empirical accounts of involvement practices, demonstrating how involvement influences research and refinement of the concept itself.

Aims To report the development of a pilot study by academic researchers and stroke service users belonging to a user research group to investigate costs of stroke to individuals and families; to reflect on what this example of user involvement achieved and implications for what involvement means.

Methods We conducted a 2‐year ethnographic study that included participant observation, formal and informal interviews with professionals and user group members and documentary analysis. Data were systematically recorded to permit description of processes and reflexive analysis.

Results and conclusions We report on five stages of the research process from service user identification of a research question to interpretation of pilot study findings. Professional researchers led the research process and developed a novel method to involve stroke service users in the development of a questionnaire. Some academic colleagues questioned the value of the proposed investigation as it did not appear to conform to implicit criteria of quality research. We argue that the moral status that user involvement has acquired means that academics’ concerns about quality did not prevent the pilot study from being conducted. We suggest that much of what was undertaken might be considered standard good practice in developing new research studies but also identify additional benefits of user involvement. Implications for conceptual development and evaluation are discussed.

Keywords: ethnography, research, stroke, user involvement

Introduction

Over the past decade professional health researchers in the United Kingdom have been encouraged to involve the public and users of services at all stages of the research process including development of proposals, conduct of studies and dissemination of findings. 1 , 2 Involvement in research is currently broadly defined but the dominant definition – for example, offered in guidance to researchers applying to major UK funding bodies – is that promoted by INVOLVE, 3 which implicitly draws on Arnstein’s well‐known hierarchy of involvement. 4 Involvement can take the form of consultation of lay people by professionals; collaboration between professionals and service users, or it can take the form of ‘user‐led’ research. A fundamental principle is that research that involves users transforms them from research subjects into partners or researchers. 3

User involvement in research is promoted for different reasons. For example, it is argued that user involvement improves research quality by producing research that is more relevant, more likely to be put into practice and lead to improvements in population health. 3 Another argument relates to the desire to democratize this sphere of civic life. Taking its cue from the more or less overtly political arguments articulated by those active in embodied health movements 5 (such as groups of mental health service users) this rationale seeks the empowerment of oppressed individuals and groups, through knowledge exchange and the facilitation of their active participation in the identification of research needs and in the research process. 6

The aims of user involvement and its potential impact are far reaching. However, as has been previously argued, there is not substantial evidence of claims made on its behalf. 7 , 8 If the policy rhetoric that has so far driven the user involvement project is to be substantiated, further evidence of its effects is required. Boote et al. 7 called for more research to assess the effectiveness of ‘consumer involvement’ in research, identifying four key areas requiring investigation. These were clarification of the concept of involvement in health research; generation of evidence of how involvement influences research drawing on a variety of research settings and topics; development of methods to measure and evaluate the influence of involvement on research; and identification of factors leading to successful involvement, starting with consensus of what this might look like, from the perspectives of both service users and researchers.

A number of studies are responding to the call to assess the practices and impact of user involvement in research and the concept itself is being elaborated both through debate and the development of more sophisticated typologies of involvement. 9 Factors promoting the involvement of users in research have been reported to include ‘good working relationships’ between professional researchers and service users enabled by mutual respect and an effort on the part of researchers to promote equality in the face of a relationship characterized by an imbalance of power. 10 A structured approach has been advocated, with appropriate training of service users to enable them to understand and take part in research development and conduct. 11 Paying citizens who are involved in research is a contested issue but some have identified this as an enabler of user involvement. 12 Reported barriers to user involvement include inadequate resources (such as time and money) and the gulf between researcher and layperson created by expert language and paradigms. 10 Studies have also reported that user involvement promotes research quality as it allows research questions to arise out of service user experience (research questions are more relevant) and it provides the opportunity to devise methods that will enhance researchers’ ability to collect data. 13 Reported consequences of user involvement studies include ‘empowerment’ of users, given new opportunities for personal development and new roles for researchers as facilitator, guide, donor of expertise. 10 , 11 , 12 , 13 User involvement has also been described as leading to new problems creating scientific and ethical dilemmas that so far remain unresolved. 14

In this paper we report an experience of user involvement in the development and conduct of a pilot study to investigate costs of illness borne by stroke survivors and their families. We present our data as an empirical example of user involvement practice in a particular research setting and focused on a specific research question. We further aim to use the data to consider two of the key areas requiring research identified by Boote et al. 7 : consideration of how and why involvement influences research and implications for development of the concept of involvement in research.

Context

Relatively little has been reported in the literature about involving stroke service users in research, although there are published reports of involving stroke service users in the development of methods for recruitment to clinical trials 14 , 15 and in the development of a survey of public stroke awareness. 16 Stroke is a major cause of mortality and adult disability. It is estimated that there are 110 000 new cases annually in England 17 with about 300 000 people living with moderate to severe stroke related disabilities. 18 Stroke frequently causes patients to have an increased dependence on others resulting in longer hospital stays, admission to a nursing home and the need for assistance from other people once back home. 17

Most research looking at the cost of stroke focuses on the cost to government and the health services. For example, the recent report on stroke care in the United Kingdom by the National Audit Office estimated ‘the burden of stroke’ including total direct health care costs for the period 2003–04 at £2.8 billion. This included hospital stays, investigations, medications and so on. It also estimated that families pay nursing home costs of £2.4 billion per year. Indirect costs – that is lost income due to death and disability, as well as benefit payments – amount to £1.8 billion per year. 17 However, little is known about additional costs paid by the individual who has a stroke and/or by their family, nor how people deal with such costs or loss of income.

The need for a study investigating the costs of stroke was identified by the Stroke Research Patients and Family Group (SRPFG). This group is a standing forum of stroke survivors/family members and researchers associated with the King’s College London Stroke Research Programme. Research undertaken within the programme includes the on‐going population‐based South London Stroke Register (SLSR), a vehicle for epidemiological and health services research. The SRPFG was established by professional researchers (CM, NF, CW) in 2006 to promote the involvement of people with stroke and family members in the stroke research programme, not merely as research subjects. Activities include 6‐weekly meetings to discuss research findings, plan new studies and produce a biannual research newsletter disseminating findings to SLSR participants. The process of establishing the SRPFG was evaluated through an ethnographic study undertaken by professional researchers CM and NF.

The topic of costs of stroke was identified by group members during a discussion that took place in a regular group meeting. Thus SRPFG members had identified a gap in knowledge arising from their own experiences in the aftermath of stroke that we – group members and professional researchers – sought to investigate. Group members expressed the hope that by identifying the economic burden imposed by stroke they might use the information to highlight their situation and use this in future campaigning directed at local and national government.

Methods

Data for this descriptive account of processes and our reflections on these were recorded during the ethnographic study 19 investigating stroke service user involvement in service development and research in an inner city area of London. 20 The study was approved by the St Thomas’ Hospital Local Research Ethics Committee. The ethnography was conducted by CM and NF and entailed participant observation, formal qualitative interviews with stroke service users and professionals and documentary analysis. Participant observation data were recorded using detailed field notes and a reflexive diary. Formal interviews were conducted with participants including members of the SRPFG and academic researchers and digitally recorded and transcribed for analysis. QSR Nvivo 2.0 (QSR, Doncaster, Australia) was used for data storage and management. NF was primarily responsible for recording ethnographic data and as participant observer aided CM in establishing user involvement: recruiting stroke survivors to the SRPFG; setting meeting agendas; chairing meetings; performing administrative tasks; encouraging stroke researchers to participate in meetings. CM and NF conducted thematic analysis of field notes, the reflexive diary, interview transcripts and documents. Analysis was undertaken concurrently with data collection and iteratively directed data collection. Strategies to enhance the validity of our data and interpretation included: having more than one researcher involved in analysis, sharing drafts of the paper with informants for comment and validation, incorporation of reflexive procedures from the outset of the study, attention to the variety of perceptions and experiences of study participants. 21

Results

The research process

Stage 1: identifying the method

The first task was to identify an appropriate research method. The professional researchers discussed possible methods with a health economist who felt that the topic was of little interest since societal costs of stroke had already been estimated and published. The professional researchers argued that this did not include ‘out of pocket’ costs but discussions went no further. We carried out a literature search to identify methods previously used to measure out of pocket costs to patients. These included various diary methods; an economic study of stroke costs which included ‘out of pocket’ costs 22 ; and a survey questionnaire to investigate generic patient costs that had been developed but not used. 23 This questionnaire was also rather long running to more than 70 pages of annotated questions.

In the next regular meeting of the SRPFG, professional researchers reported what they had found and led discussions about appropriate methods, raising possible advantages and disadvantages of available methods. SRPFG members argued that a diary method was not practical since patients and carers were unlikely to complete a diary in the early weeks and months after stroke when they may also be going through major life changes. The professional researchers proposed using the generic questionnaire to develop a novel stroke specific questionnaire, which could be tested in a small sample of stroke survivors. SRPFG members were enthusiastic about the proposal.

Stage 2: developing the questionnaire

Questionnaire development began with open‐ended interviews with people with stroke and family members to understand the experiences of individuals and families. These interviews were of two types: researcher interviews and guided conversations. Researchers conducted two preliminary interviews (with two couples) to get a broad idea about what topics were important. Data were analysed to identify topics to include in a topic guide for guided conversations. Adopting similar principles to that of the peer ethnographic approach 25 CM developed the guided conversation method to allow SRPFG members to interview each other, as a way of systematically reproducing conversations and exchange of information that take place naturally between group members during meetings. Nine people from the SRPFG took part in guided conversations during a specially conducted meeting. Participants were assembled into three conversation groupings and used the topic guide as a script or prompt to interview each other about their experiences of costs post stroke. Participants reported feeling at greater ease being interviewed by someone who had been in a similar situation as themselves. Two other group members unable to take part in the guided conversations meeting were interviewed by a professional researcher. Interviews and guided conversations were tape‐recorded, and data analysed by professional researchers to finalize topics for inclusion in the costs of stroke questionnaire.

Questionnaire design was led by a medical student with an interest in the cost of stroke (and who subsequently used this material for a student project) assisted by stroke research programme colleagues with expertise in questionnaire design. This drew on interview data and the generic patient costs questionnaire. Members of the SRPFG and researchers read the draft questionnaire to check for comprehensiveness (that all topics had been included) and for clarity. Some changes were made to the wording of the questionnaire to reduce ambiguity.

The questionnaire covered several areas of cost including: payment for adaptations to the home, medications, alternative therapies, changes in diet (e.g. buying diabetic/organic food), nutritional supplements, clothing suitable for disability, transport and direct and indirect loss of family income.

Stage 3: conducting the survey

The questionnaire was administered to consenting participants of the SLSR during the scheduled three or six month follow‐up. This is a face to face interview in which fieldworkers collect a large amount of clinical, social and service use data for the purposes of the population register. Participants included SLSR recruits living at home or in sheltered accommodation but excluded those still in hospital after stroke; living in institutionalized care (e.g. nursing home, residential home, long‐term care, community or private hospitals). Where the person with stroke could not be interviewed (e.g. due to communication or cognitive impairment), the next‐of‐kin or a carer was interviewed. The survey was conducted over six months.

Stage 4: the survey analysis and findings

Response frequencies were tabulated by the medical student, with additional socio‐demographic data and disability level (Barthel Index) drawn from data routinely collected for the SLSR.

Fifty‐five people agreed to take part in the study but one did not provide any information about costs of stroke. Responders were aged 18–86 years (average 69) and 61% were male; 65% were from white ethnic groups, 23% from black ethnic groups, 10% from other ethnic groups; (2% missing). Using the Barthel Index, a measure of activities of daily living widely used in stroke research, 43% were classified as independent; 32% mildly disabled and 24% moderately to severely disabled; (1% missing). Key findings from the pilot study are outlined in Box 1.

Table Box 1

Expenses reported by responders

Stage 5: interpreting the results

The results were presented to the SRPFG in a regular meeting, with professional researchers leading discussion on how they might be interpreted. Researchers reminded group members that this was a pilot study that aimed to test the feasibility of the method, and that a larger study would be required for a more reliable picture of the out‐of‐pocket costs of stroke. Nevertheless the analysis had shown that most survey participants reported at least one out‐of‐pocket expense. The survey also identified a small group of people who paid for adaptations to the home and for private therapy, raising questions that need to be explored in more detail about these costs. Researchers also reported feedback from the fieldworkers carrying out the survey, including their views that participants felt that this was a meaningful research question and were happy to take part. However, relatively few individuals were able to report specific costs incurred, either because they had not kept track of additional expenses, or now left their financial affairs to their adult children. Group members suggested that it would be worth pursuing a larger scale study but also looking in more depth at some topics, such as why some people paid for private therapies, whether orthodox or alternative.

Reflections on the process

Having described the development of the method to assess costs of stroke to individuals and families, we now reflect on two aspects of this process: the particularities of involving stroke survivors and family members, and the position of user involvement research vis‐a‐vis ‘traditional’ academic research.

Involving stroke service users in research

High proportions of stroke survivors have on‐going disabilities, including problems with mobility, speech and cognition. 25 Members of the SRPFG include wheelchair users, people with reduced mobility and communication difficulties. These present logistic problems which require forward planning and involvement activities perhaps require greater time than might otherwise be needed. As a category of service user, stroke survivors differ from other groups where the desire to influence service development and research may be linked to consciousness as an oppressed group and a commitment to social change. Although members of the SRPFG are concerned to see improvements in stroke care, UK stroke survivors do not appear to be politicized, nor do they have a history of activism, collective sense of oppression, or organisation as an embodied health movement, prepared to challenge experts seen as paternalistic or excluding them from decision making processes. To a large extent this shaped the model of involvement that so far we have put into practice, meaning that the professional researchers, although acting on the SRPFG’s desire to investigate the topic of costs of stroke, nevertheless led the development of the pilot study. This entailed using professional researcher skills to conduct literature searches, conduct initial interviews, develop the guided conversation method, conduct and analyse data and prepare results for consideration and interpretation by SRPFG members. Far from being a conscious decision to retain power, this was a pragmatic decision, with researchers proposing procedures and SRPFG members agreeing. It is likely that their agreement was dependent on their level of knowledge and skill, and their readiness to see the professional researchers as the technical experts. It is also possible that this relationship may change as group members learn more about research and become more willing to critique the way that research is conducted.

Situating user research in the academe

Reporting the intention to pursue this user generated study to academic colleagues, the professional researchers were met with different reactions. While some stroke researcher colleagues found the question interesting, others expressed scepticism, suggesting that scientific research could not emanate from ‘subjects’ because of their inevitable bias. More importantly there were concerns about how this piece of work might be reconciled to the need to demonstrate academic authenticity. In other words, questions were asked about why the pilot study was not ‘properly’ funded, whether ethics approval needed to be sought over and above that already obtained for the larger ethnographic study, and what types of publication the exercise might result in. At the same time the professional researchers felt obliged to address possible concerns that SRPFG members might have about the length of time required to conduct the pilot study, especially given their view that the results might be useful in political campaigning. Thus the professional researchers acted as brokers and translators, defending what appeared to be unorthodox to academic colleagues and explaining the realities of academic research to SRPFG members.

Discussion

We have described the process of developing a pilot study of the costs of stroke borne by individuals and families, which took place in the context of an on‐going forum established to promote service user involvement in stroke research. A research topic that emanated from stroke service user experience and concerns was identified and a feasible method was developed. This included developing the guided conversation method that permitted service users to conduct qualitative interviews with their peers. The pilot identified practical problems that would need to be acknowledged in this study, including the problem of recall. Thus it was certainly feasible for users to be involved in research, identifying a topic and participating in the development of a novel research tool. However, it could be argued that development and piloting of a novel research instrument might have followed similar steps even without the self‐conscious user involvement approach we took. Searching the literature, contacting researchers with relevant experience in the field, qualitative interviewing of potential responders for item generation, field testing the questionnaire and review are all standard elements of good practice in questionnaire development. In fact, the topic of costs of illness and costs of stroke had already been identified by another stroke research group 23 without evidence of these being instigated by patient or lay groups.

This raises the question of what additional benefit was provided by commitment to involve service users. We suggest that there are two main benefits. First, the fact that users themselves identified the topic meant that it was pursued at all. In the face of the lukewarm reaction of academic colleagues, we were assisted by the fact that involving users in research is a governance requirement. This in effect sanctioned the work. We were also assisted, we believe, by the moral status that user involvement has come to acquire. 26 In the absence of unequivocal evidence of benefit, the moral power of ‘users’ as a category and the need to carry out user involvement activities meant that while the pilot might be critiqued on academic terms, it was not going to be thwarted. While much of the published literature assumes that researchers have power and service users need to be empowered, we suggest that it may be illuminating to investigate further the moral status of service users as a category and the kind of power this might have in driving the user involvement project.

Secondly, the question raised by the SRPFG in effect reconfigured the topic of costs of stroke as it is usually construed by academic researchers and policy makers. Epidemiology and health economics construct the problem as the burden of stroke, focusing on costs to society. Investigating the expenses borne by individuals and families corrects this to a certain extent showing that the costs are not only borne by society but also by individuals and family members.

Therefore, in terms of how and why involvement influences health research, our experience suggests that this might occur in three ways: it can lead to the identification of questions regarded as important and relevant to service users (although this perception may not be shared by professionals); it can help refine methodology; and it might help reconceptualize problems in ways that incorporate the experience of service users.

The need to develop a more sophisticated conceptual model of user involvement in research has been identified. 7 , 27 Drawing on the literature from a wide range of areas, Oliver et al. 9 have proposed a more complex conceptual framework of public involvement in research based on type of involvement (individuals or members of organized groups), origin of involvement (invitation from professionals or in response to citizen action) and level of involvement defined as consultation, collaboration user‐led. This model represents a development in ways of thinking about user involvement but does not necessarily overcome the problem of thinking about involvement as static rather than dynamic. The type of involvement we achieved might be categorized as both user‐led and collaborative, with the contributions and roles of professional researchers and SRPFG members shifting throughout the process according to the tasks at hand and the available level of skills. As an idea it was user led and user driven as the impetus to do the work came from SRPFG members. In conducting the study it was mostly researcher led with collaboration between researchers and users.

However, our experience suggests that conceptual development of user involvement in research needs to move beyond development of typologies based on processes and to consider user involvement phenomenologically. What kind of phenomenon is this; or rather what kinds of user involvement are being constructed as researchers and lay people put involvement into practice? Among the actors engaged in our story of user involvement, there was a wide range of motivations, views and objectives, suggesting that attention to user involvement as a social phenomenon will require investigation of the goals and actions of individuals (professional and non) and groups who instigate and engage in involvement practices. Rather than involving users as individuals, members of community groups or in response to citizen action, our approach aimed to set up the means to promote on‐going dialogue between academic researchers and people affected by stroke. This means that activities, relationships and expectations of both parties may change over time, as may our understanding of what user involvement is or should be.

Conclusions

The concern to avoid tokenistic user involvement 8 implies a need to identify and promote ‘meaningful’ user involvement. This requires an agreed definition of what might constitute success, itself dependent on the definition of user involvement adopted. Different aspects of user involvement imply different criteria. For example, aspects of success relating to research quality require attention to relevant components such as whether the boundaries of knowledge are being pushed, new questions, new methods and new solutions being identified and put into practice. Evaluating the success of user involvement in empowering service users and democratizing science will require attention to knowledge exchange and changes in knowledge and expectations of both service users and researchers.

Costs of Stroke Study Team

Elsie Aird, Arun Sriskantharajah, Loyola Baptista, Brenda Bartlett, Geoffrey Bryant, Cathy Coshall, Peter Erbe, Chris Faraday, Nina Fudge, Eva Gordon, John Matthews*, Lillian Matthews, Elinor Matthewson, Chris McKevitt, Elizabeth Mitchell, Dave Morrison, Frank Nugent, Jean Oram, Sharon Porter, Jude Redfern, Judith Rosheuvel, Patrick Rosheuvel*, Arun Sriskantharajah, Dorothy Smith, Harry Smith, Sylvia Thomas, Gavin Topham and Alison Webster.

Funding

Stanley Thomas Johnson Foundation; CM was supported by a Department of Health Career Scientist award; the authors also acknowledge financial support from the Department of Health via the National Institute for Health Research (NIHR) comprehensive Biomedical Research Centre award to Guy’s & St Thomas’ NHS Foundation Trust in partnership with King’s College London and King’s College Hospital NHS Foundation Trust.

Acknowledgements

We are grateful to SLSR participants who took part in the cost of stroke pilot study; Amanda Thrift, National Stroke Research Institute, Austin and Repatriation Medical Centre, West Heidelberg, Australia; S Thompson and S Wordsworth for the Health Economics Research Unit discussion paper [03/01] written on behalf of the UK Working Party on Patient Costs.

Notes

† Sadly, John Matthews and Patrick Rosheuvel died before this paper was finalized.

References

1. Department of Health . Patient and Public Involvement in the NHS. London: Department of Health, 1999. [Google Scholar]

2. Department of Health . Best Research for Best Health. A New National Health Research Strategy. London: Department of Health, 2006. [PMC free article] [PubMed] [Google Scholar]

3. Hanley B, Bradburn J, Barnes M. Involving the Public in NHS, Public Health and Social Care Research: Briefing Notes for Researchers, 2nd edn. Involve Support Unit, 2003. [Google Scholar]

4. Arnstein SR. A ladder of participation. Journal of the American Institute of Planners, 1969; 35: 216–233. [Google Scholar]

5. Brown P, Zavestoski S. Social movements in health: an introduction. Sociology of Health and Illness, 2004; 26: 679–694. [PubMed] [Google Scholar]

6. Beresford P. User involvement in research and evaluation: liberation or regulation? Social Policy and Society, 2002; 1: 95–105. [Google Scholar]

7. Boote J, Telford R, Cooper C. Consumer involvement in health research: a review and research agenda. Health Policy, 2002; 61: 213–236. [PubMed] [Google Scholar]

8. Dewar BJ. Beyond tokenistic involvement of older people in research – a framework for future development and understanding. Journal of Clinical Nursing, 2005; 14: 48–53. [PubMed] [Google Scholar]

9. Oliver SR, Rees RW, Clarke‐Jones L et al. A multidimensional conceptual framework for analysing public involvement in health services research. Health Expectations, 2008; 11: 72–84. [PMC free article] [PubMed] [Google Scholar]

10. Staniszewska S, Jones N, Newburn M, Marshall S. User involvement in the development of a research bid: barriers, enablers and impacts. Health Expectations, 2007; 10: 173–183. [PMC free article] [PubMed] [Google Scholar]

11. Hubbard G, Kidd L, Donaghy E. Involving people affected by cancer in research: a review of literature. European Journal of Cancer Care, 2008; 17: 233–244. [PubMed] [Google Scholar]

12. Rowe A. The effect of involvement in participatory research on parent researchers in a Sure Start programme. Health & Social Care in the Community, 2006; 14: 465–481. [PubMed] [Google Scholar]

13. Lindenmeyer A, Hearnshaw H, Sturt J, Ormerod R, Aitchison G. Assessment of the benefits of user involvement in health research from the Warwick Diabetes Care Research User Group: a qualitative case study. Health Expectations, 2007; 10: 268–277. [PMC free article] [PubMed] [Google Scholar]

14. Ali K, Roffe C, Crome P. What patients want – Consumer involvement in the design of a randomized controlled trial of routine oxygen supplementation after acute stroke. Stroke, 2006; 37: 865–871. [PubMed] [Google Scholar]

15. Koops L, Lindley RI. Thrombolysis for acute ischaemic stroke: consumer involvement in design of new randomised controlled trial. British Medical Journal, 2002; 325: 415–418. [PMC free article] [PubMed] [Google Scholar]

16. Morgan LJ, Chambers R, Banerji J, Gater J, Jordan J. Consumers leading public consultation: the general public’s knowledge of stroke. Family Practice, 2005; 22: 8–14. [PubMed] [Google Scholar]

17. National Audit Office . Reducing Brain Damage: Faster Access to Better Stroke Care. London: NAO, 2005. [Google Scholar]

18. Wolfe CDA. The Impact of Stroke. British Medical Bulletin, 2000; 56: 275–286. [PubMed] [Google Scholar]

20. Fudge N, Wolfe CDA, McKevitt C. Assessing the promise of user involvement in health service development: ethnographic study. BMJ, 2008; 336: 313–317. [PMC free article] [PubMed] [Google Scholar]

21. Mays N, Pope C. Qualitative research in health care: assessing quality in qualitative research. BMJ, 2000; 320: 50–52. [PMC free article] [PubMed] [Google Scholar]

22. Dewey HM, Thrift AG, Mihalopoulos C et al. ‘Out of pocket’ costs to stroke patients during the first year after stroke – results from the North East Melbourne Stroke Incidence Study. Journal of Clinical Neuroscience, 2004; 11: 134–137. [PubMed] [Google Scholar]

23. Thompson S, Wordsworth S An Annotated Cost Questionnaire for Completion by Patients. HERU discussion paper [03/01]. University of Aberdeen: On behalf of the UK Working Party on Patient Costs, 2001. [Google Scholar]

24. Price N, Hawkins K. Researching sexual and reproductive behaviour: a peer ethnographic approach. Social Science & Medicine, 2002; 55: 1325–1336. [PubMed] [Google Scholar]

25. Patel MD, Tilling K, Lawrence E, Rudd A, Wolfe C, McKevitt C. Relationships between long‐term stroke disability, handicap and health‐related quality of life. Age and Ageing, 2006; 35: 273–279. [PubMed] [Google Scholar]

26. Dewar B. Beyond tokenistic involvement of older people in research – a framework for future development and understanding. International Journal of Older People Nursing in association with Journal of Clinical Nursing, 2005; 14: 48–53. [PubMed] [Google Scholar]

27. Tritter JQ, McCallum A. The snakes and ladders of user involvement: moving beyond Arnstein. Health Policy, 2006; 76: 156–168. [PubMed] [Google Scholar]

Articles from Health Expectations : An International Journal of Public Participation in Health Care and Health Policy are provided here courtesy of Wiley-Blackwell