Purpose: The research project is a three-and-a-half year study which started in December 2009 and finished at the end of May 2013. It is an evaluation of the effectiveness of the North East Transformation System (NETS). NETS was originally introduced by NHS North East and is centred on the so-called three-legged stool – Vision, Compact, and Method – as the mechanism to bring about large-scale transformational change.
Design/Approach: The research comprised a longitudinal (involving repeated observations of the same variables) mixed-methods study. Research methods used included open interviews, observation, focus groups, documentary analysis, and interrupted time series (ITS). The ITS component comprised five case studies in two of the sample sites. Two rounds of interviews were conducted in Years 1 and 2 of the study with staff in our sample of 14 sites comprising a range of NHS organisations. These comprised acute hospital trusts, mental health trusts, community services, an ambulance trust, and commissioners in the form of PCTs. To examine the NETS method in greater detail four Rapid Process Improvement Workshops (RPIWs) were observed. Focus groups were conducted among human resource managers and knowledge process outsourcing (KPO) leaders.
Qualitative data analysis proceeded using Pettigrew et al’s ‘receptive contexts for change’ framework to explain transformational change in NHS North East. The framework comprises eight factors:
Findings: Undertaking successful transformational change in a complex system takes time and demands consistency, constancy of purpose and organisational stability. The NETS was seriously disrupted by the NHS changes announced in July 2010. Given these ever-shifting changes in the overall context and external environment in which the NETS was occurring, combined with the numerous complexities to be found in any health care setting, it is extremely difficult to arrive at any final conclusions about its success (or otherwise) or impact either on services or the public’s health status in the North East. Even where there may be evidence of change and improvement, attributing these with confidence to the NETS is impossible. Establishing strong causal links as distinct from strong associations and/or correlations has not proved possible.
Notwithstanding the variable impact of the changes on the overall NETS programme, four of the study sites in the sample demonstrated positive impacts. Progress in the other study sites was either slowed, halted or seriously disrupted by the NHS upheaval which resulted in the NETS losing momentum. Leadership style is critical to the success of transformational change and while it was clearly a factor in the progression of the NETS overall, it was also critical in respect of each of the participating organisations.
The four sites which made progress in implementing the NETS were all notable for clear, visible and relatively stable leadership. Despite this, the commitment to embedding deep cultural change proved challenging and fragile. It is also the case that most of the attention of managers and other practitioners was devoted to the Lean tools rather than to the more difficult issues around values and culture which the Vision and Compact (two legs of the NETS stool) sought to address.
Compared with Lean in the manufacturing sector, its application to the NHS involved a far greater degree of being able to manage complexity and numerous competing objectives. Perhaps four, maybe five, of the study sites remained truly committed to the NETS approach while other sites tended to adopt a pick and mix approach combining elements of the NETS approach with other inputs which had been decided upon locally as being of greater value. The absence of adopting a pure NETS approach did not preclude some sites from achieving success in quality improvement and patient safety.
Analysis of the ITS component of the study gave rise to mixed findings with some statistically significant improvements observed; some ambiguous results; several where no evidence of impact of the RPIW could be detected; and some counter-expectation findings. Clear improvements included a reduction in time from arrival of patients with abdominal pain in A&E to being x-rayed (surgical pathway RPIW); reduction of length of stay on the ward for women (purposeful in-patient admissions RPIW); and a large impact on the proportion of patients allocated on the day of referral (community psychosis-admission RPIW). In general, it is difficult to draw definitive lessons from the ITS analysis. Given the reliance on routine administrative data and absence of data on the range of clinical outcomes, it may be the case that the ITS has missed significant impacts in some key outcomes.
Project award: £509,000
Project dates: From December 2009 to May 2013
Practical implications: Two dissemination workshops have been held in the NHS North East to share the findings and to heed the lessons for future change initiatives. The key lessons are explored in the publications listed under Outputs. The final report will be available shortly on the NIHR website.
Contact information: Principal Investigator: Prof David Hunter
Please direct all queries to Gill McGowan, Centre for Public Policy and Health Administrator, Durham University, email: firstname.lastname@example.org
Purpose: There is increased urgency to demonstrate return on investment in relation to public health interventions and explore methods of decision-support for public health priority-setting. The return of the responsibility for public health commissioning to local authorities means that priority-setting will take place within new organisational and cultural settings, which presents new challenges. With local authority ring-fenced public health budgets confirmed, difficult decisions about investment, and particularly in a time of economic stringency, about disinvestment, will have to be made, not just within the ring-fenced public health budget but also across different departments of the local authority.
Design/Approach: This two year study is supporting public health priority-setting in three local authority case study sites across England, through bringing together specialist input from health economics and public health in a series of seminars and targeted decision-making support for public health commissioners. The relevance of prioritisation methods and their impact on spending patterns within and across programmes will be evaluated through a series of initial and follow up interviews with decision-makers in each site.
Findings: No findings available yet.
Project award: £294,263.27
Project dates: From April 2012 to July 2015
Practical implications: The project is seeking to show which priority-setting tools local authority commissioners find useful for public health investment, assessing enablers and barriers to decision-making and deliver recommendations about appropriate decision-making support for determining priorities in public health commissioning within local authorities.
Contact information: Principal Investigator: Prof David Hunter
Please direct all queries to Michelle Cook, Project Administrator, Durham University, email: email@example.com
Purpose: School meal provision was introduced in the mid-19th Century as a public health response to under-nutrition of children. In the late 20th Century, the focus for public health shifted as the obesity epidemic in children emerged, and a number of nutrition-related public health initiatives were introduced. One such initiative was the introduction of the school food policy in England in 2006; primary and middle schools were to be fully compliant by September 2009.
Design/Approach: A mixed methods approach was used to collect data at both school and individual level from two age-groups in Newcastle and Northumberland, North East England. Dietary, anthropometric (body measurement) and socio-economic data were collected using identical quantitative methods pre, mid and post-implementation of the school food policy. Data on food eaten at school (school lunch or packed lunch) and throughout the day were collected. A qualitative approach was used to examine the process of implementation of the policy.
Findings: There were significant improvements in the nutrient content of both school and packed lunches in children aged 4-7yrs; the extent of change was greatest in school lunches. There was less evidence of such changes for children aged 11-12yrs.
The effect of lunch type choice (school or packed lunch) on total dietary intake changed from pre to post-implementation of the school food policy in that those having school lunch had intakes more in line with recommendations. For some nutrients, this was a reversal of intakes prior to the school food policy and demonstrates the impact of the school food policy not only on lunch time intake but also on the total dietary intake of 4-7yr olds.
In contrast to our findings in 4-7yr olds there was limited evidence of the effect of school lunch type on the total diet of 11-12yr olds; the exception was in percentage energy from fat. Mean daily intakes from iron and folate fell from 1999-00 to 2009-10; it is important to note these were both below the Reference Nutrient Intake (RNI) in 1999-00 and remained so in 2009-10.
The process evaluation suggested that schools coped well with challenges involved in implementing the school food policy. The knowledge and skills of catering staff, their ability to adapt to new processes and ways of working were important factors as was the level of commitment from senior managers. It was evident for both age groups that the food choice available was only one factor in the decision to have school lunch or packed lunch; the dining room experience and encouragement offered to children is part of this choice.
Project award: £459,480
Practical implications: Our findings, particularly for 4-7yr olds, have demonstrated the potential for school lunch to have a positive impact on the total diet. To maximise this impact there is a need for a concerted effort to ensure full compliance with the policy for all age groups and to encourage and facilitate children to take advantage of school lunch.
Spence S, Delve J, Stamp E, Matthews JNS, White M, Adamson AJ. The Impact of Food and Nutrient-Based Standards on Primary School Children's Lunch and Total Dietary Intake: A Natural Experimental Evaluation of Government Policy in England. PLoS One 2013, 8(10), e78298
Adamson, A., Spence, S., Reid, L., Conway, R., Palmer, A., Stewart, E., McBratney, J., Cather, L., Beattie, S. and Nelson, M. (2013) 'School food standards in the United Kingdom: implementation and evaluation', Public Health Nutrition, 16(6), pp. 968-981.