/ 5 July 2013

NHS data used to score patient experience is unreliable

Better service outcomes could be attained in the UK's National Health Service by focusing on the individual needs of patients.

Cross-party consensus about the need to improve the quality of Britain's National Health Service (NHS) has placed person-centred care in the spotlight. 

Patient experience, always a part of the service-quality assessment regime, now ranks equally with clinical outcomes in significance. However, our research shows that the current way of measuring patient experience is not fit for the purpose, with a tendency to overestimate how positive patients are about a service. 

In the wake of the report about care provided at Mid Staffordshire NHS Foundation Trust, the government has emphasised that better service outcomes can be achieved only through a "culture in which the needs of patients are central". 

NHS managers and board directors are "trying to improve their intelligence on patients' experience" because they "are certain it needs doing, but [are] not sure how".

The United Kingdom department of health has produced a number of measuring tools that attempt to integrate experience-based indicators such as the quality of relationships, individual needs and concerns, and continuity of care with traditional metrics such as mortality rates, emergency admissions or safety incidents. 

These include the 2012 overall patient experience scores – developed on the principles of patient-centred care – and the 2013 NHS outcomes framework, which has introduced scoring for "responsiveness to patients' personal needs". 

These are important developments. The latter, for example, forms part of the way in which the secretary of state will hold the new NHS commissioning board to account. The health department is developing further methodologies for capturing real-time feedback on aspects of care such as dignity and emotional support.

On the surface, then, we appear to be taking patient feedback seriously in order to make sure care is patient-centred. However, the aforementioned scoring systems, which take as their input patient questionnaires, individual patient interviews or the insights produced by patient and public involvement (PPI) organisations, cannot yield reliable data on complex personal experiences.

To assess such input to scoring systems, we conducted in-depth fieldwork at one of the UK's largest PPI organisations, a national forum of cancer patients and carers supported by the department of health and major cancer research charities.

PPIs are acknowledged by most NHS trusts as one of the most sophisticated and sensitive ways to measure the quality of services, yet our study suggests the approach to them needs to be rethought. In particular, they display a tendency to overestimate patient satisfaction. 

Failings
The main failings include leaving little or no room for patients' own feelings about what criteria are relevant in assessing good practice. For example, patients might want to reflect on how their emotional needs are considered by care providers during treatment, but there is no way for them to put this on the pre-defined agenda. 

Second, patients and professionals find the feedback mechanisms unsuited to developing a relationship in which trust is established over time. Finally, those involved feel a sense of pressure to achieve consensus during the interactions themselves, thanks to inflexible time schedules and overly formal meeting arrangements where the roles of lay and professional participants are starkly delineated, encouraging an us-and-them dynamic.

Consensus is often achieved and PPI meetings are recorded as being successful. Verdicts about service would look very different – and would be more accurate – if such failings were addressed. Our research reveals that there is significant yet overlooked potential for transforming PPI partnerships into forums for meaningful discussions about patient experience that leads to accurate assessments of satisfaction. 

Other more constrained forms of measurement such as patient questionnaires or interviews should be jettisoned altogether. Instead of a one-directional listening process that defines the terms of engagement and has survived largely unchanged since patient experience was taken less seriously, we are calling for a relational approach to measurement, which would include:

• Providing space and time to ­listen to what patients and professionals on the ground have to say; 

• Creating protected time for staff to engage properly with feedback and to learn how to respond; and

• Allowing for different and more nuanced interpretation of the measurement process itself that asks not simply what should be measured, but importantly also how and why.

Anything less pays lip service to the principle of patient-centred care and means shortcomings will remain unacknowledged. – 

© Guardian News and Media 2013

Aris Komporozos-Athanasiou is an ethnographer of public organisations and associate researcher at the Cambridge Institute for Public Health. Mark Thompson is senior lecturer at Cambridge's Judge Business School and strategy director of Methods, which is developing a platform offering open access to healthcare-performance data