Impact case studies

REF Impact Case Study submissions were made earlier this year, and we have some time before having to think about research impact again. Or do we?

Well, that depends on how you view research impact, which can be stated simply as the demonstrable good that researchers can do in the world, consisting of the non-academic benefits that arise, directly or indirectly, from research.

In terms of REF accountability, it will be some time before the next submission. And with the overwhelming responsibilities academics have, it would be easy to put this on the back burner. But for the sake of argument, let’s begin by following a line of thinking that emphasises research impact as a means of gaining the highest possible rating in REF impact case studies.

According to the REF guidelines, impact has to have significance and reach. These two terms have often been misunderstood. Significance deals with whether or not the impact of research is meaningful, valuable and/or beneficial. Reach relates to whom the impact affects. It’s generally been assumed that impact has to be big, showy or extensive for it to be rated highly. This is not necessarily the case. But it does have to be valuable or meaningful, and it does have to impact a significant extent and diversity of relevant individuals, systems or organisations within the groups where it’s focused.

For example, let’s say your research involves a new medical intervention to improve outcomes for patients under the age of 5. Based on your research, this intervention begins to be used by 90% of hospitals across England. How should this be judged? First, we have to ask whether the impact of the research, that is, the use of the intervention itself, is valuable: has it produced any beneficial effects for under 5s? Is their health or wellbeing improved by this intervention? If it’s no better than conventional treatment methods, this impact is not particularly valuable or beneficial. The fact that its reach is very good – occurring in 90% of hospitals – then becomes irrelevant. But let’s say there is a significant improvement in the health of under 5s. In this case, the research impact would have both significance and reach.

But what if the intervention helps under 5s, and only under 5s? That is, use of the intervention in older children and adults is not shown to significantly change their health outcomes. Let’s assume the treatment in hospitals covers all patients over the age of 5, and under 5s are treated exclusively in GP surgeries. This new intervention would be used extensively in English hospitals, but it would do little for the group it actually helps because they are not treated in hospitals. The impact in this case would not be significant (since it’s being used with patients it hasn’t been shown to help) and poor in reach (since those treating under 5s are not using it).

However, if this intervention were to be used in these circumstances in just one GP surgery instead of in many hospitals, but it is used consistently with all patients under the age of 5, this research impact would have better significance and better reach. Getting the significance right with small, achievable reach first makes it easier to expand reach as research and implementation develop. Once the value of the research put into practice is demonstrated well in one GP surgery, it makes it easier to extend impact by scaling up to other surgeries, primary care networks, NHS trusts, or perhaps even other countries.