You have to feel sorry for Lord Drayson. At a time when he is doing his best to stand up for science and the funding of science in what are very difficult economic circumstances, where every department in government must show it’s worth, scientists appear to be trying to hack his legs off; by balking at the proposal that they should explain their impact on society and insisting that they should be left to do ‘blue skies’ research. I refer to the recent debate hosted by the Times Higher Education Supplement (THES): “Blue skies ahead? The prospects for UK science“, a discussion which centred around the impact of science on society and how you might increase that impact, how impact is evaluated, the role of blue skies research and the crisis at the Science and Technology Facilities Council (STFC).
In this post I’ll try to explain why scientists are so impassioned about the subject of impact assessment and make a couple of suggestions as to how we might collectively do this better.
Impact means several things in this context: there is the HEFCE Impact pilot exercise, which is retrospective and in pilot phase at the moment and there are impact statements in grant applications which were introduced this year which should provide a prediction of the economic and societal impact of the work proposed. I suspect it is the impact statements in grant applications which are causing the real concern here, and I also suspect that Lord Drayson was talking primarily about the former, and the audience and panel were talking about the latter at the THES event.
Before I go on I will provide a short autobiography to provide context for my comments: I’m currently a research scientist in a large company, I’ve been here for 5 years but until the age of 35 I was an academic scientist, rising to the position of tenured lecturer in the physics department of what is now Manchester University. The opinions I present here are entirely my own.
Putting aside the question of how good a scientist I am; I can tell you one thing for certain: I’m an incredibly bad at writing successful grant applications! Really bad, awful, abysmal. I wrote about 8 over my relatively short period as a grant writer and they all failed (and not even by a small margin). I don’t think I’m alone in this.
Exactly what grant applications mean varies from subject to subject, but in my field: experimental laboratory-scale soft matter physics it’s important: in order to do research you need people and equipment. Typically grant applications are written to get 2-3 years of postdoctoral research assistant, or a PhD student and some equipment. Your department will rate itself on the grant funding it obtains, and you on your contribution to that figure. You will definitely feel that winning grants is something you have to do to succeed in your job. You can eke out an existence without grants, funding students from other sources and helping colleagues with successful grant applications, throwing yourself into teaching but it doesn’t feel like the way you’re supposed to do it.
I find it difficult to put into words how much I loathed the whole grant application process. A grant application requires you to describe the research you’re going to do over the next few years, and how the results will be world-class. The average success rate is about 20% (disputed) in the field in which I was applying. Once written the grant applications are sent to reviewers – actually that means someone just like you – it’s peer review. Reviewers know full well that if they rate an application “excellent” as opposed to “outstanding” in any one of several areas they are damning the application to failure. Reviews go forward to a panel who plough through huge numbers of these things (about 5 times as many as they are going to fund), then rank them. Funding is given to those at the top of the list, working down until the available cash is exhausted. There are serious questions as to whether we can rank schools accurately, here we try to do it for world-class research. It’s a grim process for all involved.
It’s in this context that the new impact statements are introduced, potentially these contribute 25% to the grant decision. As an grant application writer I need this like I need another hole in the head! Writing the science part of the grant application is a work of pure fiction (I think it might have helped if I’d appreciated that when I was doing it), writing the impact statement: “The demonstrable contribution that excellent research makes to society and the economy” is going beyond that. This is forward looking, you’re being asked to quantify the future economic value of that bit of world-class research you’re going to do.
The problem is that university science can take a long time to trickle out, in a case I highlighted yesterday in what is a pretty applied area, research was having a very direct, specific impact 40 years after it was done. Prior to my grant seeking days my work as a PhD student, postdoc and ADR has been funded, at least in part by three separate companies – little word of the economic impact has made it back to me from these companies.
So this brings me to some concrete points:
Point 1: For many areas of research societal, economic impacts are diffuse and long term, and actually the academic proposing the research is not in the best position to determine those impacts. As an industrial researcher I’ve written business cases for doing external research, this is a very revealing exercise which I know I couldn’t have done as an academic because I simply wouldn’t have had the required information to hand. Impact statements should not be required on a “one per application” basis, they should be for whole subject fields and written in consultation with people with actual data on the economic and societal impacts.
Moving on to a second point, Lord Drayson very generously praised scientists in Britain for the quality of the science they do, but said the problem was in how this expertise is translated into wider society:
Point 2: Impact statements are purely about scientists, applying for grants in universities. The onus seems to be on scientists to fix a problem which has two sides. What are we doing about how society and industry interact with science?
In the end impact is about communication, it’s about understanding the preconceptions that other people bring to the party and addressing these preconceptions in the way you communicate. Although I described myself as a research scientist, I’m actually a science communicator in and industrial environment.
This whole blog is really about communicating what it’s like to be a scientist, how it feels, the little details of the tribe. The people I follow on twitter are all very clever, they do lots of different things and from a combination of tweets and blogs you learn about their lives. This is my contribution to that discussion, it’s a societal impact.
11 comments
Skip to comment form
Excellent post that brings a perspective that was largely lacking in the #sciblue debate.
Very much hope you're happier in industry.
@Stephen_curry thanks! I am much happier in industry, there are things I can't do (like spending time to go into something in real depth) but it's great to do science that people really seem to be interested in, immediately. And I do get to go visit universities every so often. Miss doing tutorials and lab classes but not lecturing!
Having had an industrial perspective, I think I'd make a better go at being an academic a second time around.
Interesting post and – if I've understood it properly – it may well have hit the nail on the head. It's not that scientist don't want the impact of research to be considered, it's that they want it to generally be done in a more global way. Science ministers and heads of funding councils should have information (both economic and societal) that they can use to justify their budgets and that can help when budget decisions are made. What I'm worried about – and others I think – is the inclusion of impact assessment on individual research grants or when assessing individual university departments (or even individual universities). Of course, in some sense, the assessment of a researcher or a research department's impact has always happened, but not in a formulaic way. By making it more formulaic, and presumably more rigid, we risk damaging the potential impact of future research.
Good point about the two sides to the impact question – I agree that this did seem to be a confusion at #sciblue.
I was initially skeptical about the impact idea and the affect it would have on blue skies science but, having recently written an impact plan for a NERC proposal, I'm won over. Maybe it is different in more theoretical fields but in my work the impact plan offered the chance to include new and less formal collaboration with the people that might use the results. This is not something I would have done without the inclusion of the impact plan. There don't seem to be many people standing up for it so I'm ready to be shot down but I thought I'd include my experience here.
Aha, now I get. Very helpful indeed! I was rather disappointed with Sciblue myself, it's a good job the non-sci audience wasn't tuned – not great PR for the world of science. Well done Lord D for getting stuck in though. How long that will last I'm not sure if this is the visionary debate he gets!
I think you're right – collectively, as scientists spending public money, we should be making a case for basic research – but having to write two pages of nonsense about how our (yet unfunded and untried) research will impact society is plain silly. It's interesting that, when the classic dinner party question 'what do you do?' comes up, many non-scientists I have spoken to seem to understand perfectly well the value of research for its own sake, and what impacts it has had in the past. So we cannot be doing everything wrong.
Excellent post that elegantly and cogently synopsises arguments many academics have been putting forward for quite some time.
I'd quibble with just two points. Academics are deeply concerned about *both* aspects of the impact agenda (i.e. the HEFCE and RCUK approaches). There was certainly confusion at the sciblue debate regarding the HEFCE and RCUK approaches to impact but that arose entirely from the Science Minister. I don't know a single academic that confuses the "retrospective" HEFCE approach with the "predictive" RCUK impact statement requirement.
Over 13,500 academics have signed the UCU petition related to the HEFCE REF proposals on impact. See this week's THE for an article from Zoe Corbyn and an opinion piece from Sally Hunt on the petition. To put this in context ~ 50,000 academics were submitted for the last RAE.
Second, it's important to realise that academic research spans a rather broader range of areas than just science, technology, and engineering. I say that as a scientist and, incidentally, a nanotechnologist who, like many of my colleagues, could write, in my sleep, impact statements filled with purple prose on the socioecomonic impact of nanotech. These would be filled with hyperbole, bearing no connection with the outcome of the project, and fundamentally disingenuous – but would tick the appropriate boxes. How is this providing best return on taxpayers' investment?
Finally, I must admit that it sticks in the craw just a little to be lectured on accountability to the taxpayer by Paul Drayson. See http://www.monbiot.com/archives/2009/05/12/captive-knowledge/ and http://www.monbiot.com/archives/2006/02/14/a-good-model-for-a-mugging/
Thanks for the comments. I suspect this blog post has been read by more people in 24 hours than have read any of my scientific papers.
I do think impact assessment, and the communication of science, is important – over the next few years there'll be a lot of pressure on budgets and if we're not explaining very clearly why it's good to spend money on science the best minister in the world won't be able to save us! The trick is how best to do it: I actually think you need some industrial / UK employer input on this.
I think there's a real issue over how the grant system is working, which I know EPSRC at least is trying to address (that's not to endorse what they're doing). the grant application system seems to be used to do too many things.
I do like the streamed video / twitter events / blog follow up combination.
Ian,
I agree entirely with your comments re. the communication of science – I, like the vast majority of my colleagues, care passionately about this. It is, however, frustrating and disingenuous of RCUK, HEFCE, and BIS to misrepresent, repeatedly, the concerns of university researchers on impact by trotting out the tired old line about "reactionary ivory tower academics not wanting to be made accountable for the funding they receive".
Unlike a number of #sciblue Twitterers (ermmm…Tweeters?) and bloggers, I was very unimpressed by Drayson's performance on Monday. Kudos to him for participating in the debate and "facing the music" as it were. Rather less kudos for not being better informed and failing to address many of the comments from the floor and from other panel members. (And as I said in the comment above, I don't see why I should take lectures on public accountability from someone whose track record in that area is far from squeaky clean).
Your argument that "… if we're not explaining very clearly why it's good to spend money on science the best minister in the world won't be able to save us" is of course entirely fair. But we *are* explaining very clearly why it's good to spend money on science – from scientific, societal, and economic perspectives. The arguments are outlined clearly at for example,
The Danger of Assessing Research by Economic Impact . Sally Hunt's article entitled The worst of all worlds in this week's THE provides an excellent synopsis.
The difficulty, of course, is that the counter-arguments to the HEFCE/REF/BIS proposals – from both scientific and economic perspectives – of course don't align with government ideology. And that's the primary issue here. Ideology trumps data, statistics, and evidence. If I can paraphrase a statement from a New Labour MP who shall remain nameless: "The problem with scientists when they deal with government is that they think too much like scientists".
Bottom line: The HEFCE and RCUK plans will be detrimental to the socioeconomic impact of science (and, more broadly, academic research).
Right, enough of this for now – back to the lab!
Best wishes,
Philip
I used to run grant assessment panels in some areas of science. The heartbreaking times I wished I was a secret millionaire to be able to fund below the cut-off line… This country needs more investment in blue skies science and technology – but how to improve that when all society seems to value is 'being a celebrity'?
@newshoot from the perspective of one now on the outside. The problem with the grant system is not the amount of cash in it, it's the mismatch between the amount of cash and the amount of cash applied for. Departments tend to use the grant system as a measure of the value of the staff, or at least don't dispel that impression. I can imagine a happy place as a lecturer where, rather than fight for funding for post docs and students and equipment, I became a peripatetic postdoc-like figure. Doing stuff that I found interesting, only required a modest amount of cash, and a skilled pair of hands.