Failing the Turing test

Wherever one goes in the present-day British university, one finds not just general but universal agreement as to the fatuity of the "impact" criterion in the revoltingly named research excellence framework. There is no need for any survey to document this attitude, which is only intensifying as the end of the five-year assessment period draws closer; preparing the submission (as they say) is the small change and the major investment of every department, and every department is united in the incredulous contempt in which it holds the regulations preoccupying so much of its research time.

It is important to force oneself to read some of the REF criteria by way of recollecting that the condition of one's language is the supreme responsibility of the university, whatever discipline one practises. The very heart of intellectual enquiry beats exactly at the point at which the language of thought probes and cuts open the puzzles of experience. But how on earth is this an action to be performed precisely with concepts to hand such as "prioritising", "outcome indicators", "operational implications" and, for these latter are ourselves, "impact beneficiaries"?

The trouble is that it is all too easy to lean on the solidarities of academic life, quote this inane diction, and wait for laughs. It was the great, crazed poet of Modernism, Ezra Pound, who wrote in 1931 that when the custodians of truthfulness and facticity, which is to say academics, allow their work to go rotten - "when their very medium, the very essence of their work, the application of word to thing goes rotten, i.e. becomes slushy and inexact, or excessive and bloated, the whole machinery of social and individual thought and order goes to pot". The most casual listener to the everyday conversation about our common political life, on television, in the yellow press, in Parliament, in Tesco, can hardly doubt the rottenness with which word to thing is currently applied.

George Orwell, taking up Pound's theme in 1946 in a famous essay, said, as we might all say today, "English becomes ugly and inaccurate because our thoughts are foolish, but the slovenliness of our language makes it easier for us to have foolish thoughts." Then he says roundly, "The point is that the process is reversible...there is a long list of fly-blown metaphors which could be got rid of if enough people would interest themselves in the job." Finally, he says - and at this point you should be cheering wildly unless overcome by shame at your complicity with the pro vice-chancellors - "Political language...is designed to make lies sound truthful and murder respectable, and to give an appearance of solidity to pure wind."

The language of the REF is everywhere flatulent and frequently mendacious, never more so than when defining impact or commending excellence. The least the guardians of thought and truth can do, appointed to such a task by their history and their state (although that isn't often how we put things to ourselves), is refuse to use such language themselves, and jeer sufficiently loudly to render the impact criteria of the REF - as written by...by whom? by Saturn's children with degrees in PPE and doctorates in policy engineering, one fears - ridiculous and unspeakable.

It is rumoured, for example, that at the best universities, those chosen by a heavenly choir as the destination for the children of the national and international rich, their submissions will be limited strictly to aspirants for four- and three-star classification. Two- and one-star classifications belong elsewhere, to struggling lecturers in Ulster or Cumbria or some such. A four-star piece of research will be found "instantiating an exceptionally significant multi-user data set or research resource". Three-star REFable work (yes, this is the neologism) must show "the application of robust and appropriate research design, and techniques of investigation and analysis, with intellectual precision". "Robust", of course, is now an adjectival cliche in the cant, as indicating tough, no-nonsense methods, along with the meaningless "generic", the pharmacological-sounding "toxic", the strenuously respectable "appropriate".

It is believed by the helots of the Department for Business, Industry and Skills that, like everything else in the world, impact is measurable by number. So if you are four-star REFable in, say, medical studies, then to appear in The Lancet is the measure of fame, and the journal is so canonised because everybody cites it; contributors, moreover, cite themselves, and the consequent citation index gives The Lancet a score of 33.63, whereas Medical Teacher, an honoured journal much favoured by GPs, struggles along, starved of citation, at 1.494. It is relevant to add that The Lancet score rocketed upwards in terms of its citations when, in a picturesque example, it published in good faith research claiming to discover a causal link between autism and MMR injections in babyhood. Numberless citations in repudiation of certainly unmethodical and probably dishonest work contributed largely to the journal's lordly score.

That, as Thomas Kuhn showed us so conclusively 50 years ago, is how science revolves, sticking grimly to old beliefs and methods until at last the weight of new evidence finally upsets the applecart and everyone pretends they knew what was coming all along.

Impact, unexpectedly, has nothing to do with colleagues and collaborators in a subject, for (para 79) "impacts on research ... within the HE sector (whether in the UK or internationally) are excluded" (last two words in bold). Why? No explanation. It sounds, at least, counter-intuitive. No, it sounds bloody ludicrous. Let us take, in rebuttal, a recent historical illustration.

Over the past 40-odd years, the former Regius professor of history at the University of Cambridge, Quentin Skinner, has developed his distinctive version of "Cambridge historicism". This has been a radical and thoroughgoing restoration of political ideas to the centre of historical explanation. This imperial venture expanded from Skinner's own now-celebrated volumes of essays into a long series of editions of canonical texts, the editing supervised by Skinner and Raymond Geuss according to their principles of contextual interpretation. These volumes, representing the great texts of Western political thought from Aristotle to Mary Wollstonecraft, from Max Weber to Eduard Bernstein, are situated deep in the experience and traditions that gave rise to them. They are returned to the strains and stresses of everyday politics, and presented as answers to the questions that these historical tensions so urgently posed. The editions have sold, worldwide, more than a million copies.

No one can measure what difference Benjamin Constant or Thomas Hobbes might make to the muffled nationwide debate about, say, democracy in China or Iran because, you see, "impacts...within the HE sector (whether in the UK or internationally) are excluded". Impact in the Politburo or on the Supreme Leader is not measurable.

By the same token, the Cambridge series is accompanied by volumes of essays titled Ideas in Context - half a million have been sold. One of these essays, from Skinner's own hand, concerns the 500-year genealogy of the idea of the state itself. This is no doubt addressed to "the HE sector", in the unlovely phraseology of the REF. But it reminds us that the state, a living fiction, is not only the agent capable of assuming on behalf of the people indebtedness that will last a lifetime, but possesses, in its fictive reality, a moral personhood distinct from rulers and ruled. At a time of public hysteria about the oppressive state whether in the US or here at home, this academic history may or may not prove to have an impact on governments or peoples, but it impresses and expresses an irrefutable truth, of unignorable importance in our everyday life.

There is no limit to the examples one could take to illustrate the impotence and vacuity of the concept of impact. It may even be that this latter commonplace is finding acceptance among those of our rulers past and present who have dealt so irresponsibly with university life, since impact has undergone a 5 per cent devaluation since 2008. Nonetheless, it is worth reminding ourselves of the case of Alan Turing by way of affirming the utter elusiveness of our key concept.

Turing, perhaps the greatest British mathematician ever, was born 100 years ago this week in 1912. He was at once recognised as a supreme talent when he graduated from King's College, Cambridge in 1934 and was voted into a fellowship at the college (by John Maynard Keynes, among others) the following year. It was a moment of extraordinary commotion in the mathematical world, pulled one way by Kurt Gödel's incompleteness theorem, which showed that there exist necessary but unprovable assertions. Arithmetic, Gödel said, is not provably consistent.

At about the same time, however, David Hilbert, by then the grand old man of the subject, was pulling in the opposite direction. He had asked of all mathematical procedures that they meet three criteria: they must be complete; they must be consistent; they must be decidable. Hilbert concluded that "there is no such thing as an unsolvable problem". Gödel dispatched his deadly response. Proof in Princeton was merely a mathematical property.

Turing, empirical but no positivist, was early seized of the idea of a mechanical answer to this impasse. His biographer, Andrew Hodges, tells us: "He started from nothing and tried to envisage a machine that could tackle Hilbert's problem, that of deciding the provability of any mathematical assertion presented to it."

Turing took a typewriter, with its upper and lower cases, its moving letter point and changing characters, its spacebar and backspacing facilities, and made that his first model of a finite machine to settle the provability of an arithmetical assertion made according to arithmetic's fixed rules. Furthermore, it is important to grasp that the Turing machine was to answer Hilbert's problem of itself. It would read (Turing's coinage was "scan") a mathematical assertion and decide its provability without human intervention.

There isn't space to tell the whole gripping story. But the point is plain. Turing's machine, incarnated in his long paper, "On computable numbers, with an application to the Entscheidungsproblem", which was published in 1937, was all his own invention. Its apotheosis at Bletchley Park in the Enigma machine lay four years or more ahead, and what pushed it to that fulfilment was an unprecedented violation of British rules of class and gender, seniority and achievement, all brought about - to put it a bit crudely - by the critical losses suffered by British convoys in the North Atlantic.

It would have been damned hard to put an impact value on "Computable numbers", and the journal in which it was published, Proceedings of the London Mathematical Society, would have had an invisibly low citation score. Nor will it do to dismiss Turing as working in prehistory, and to commend the REF impact scale not only as objectively numerical - I have shown the emptiness of that claim - but also as equal in its opportunity, democratic in its conversability, in ways impossible in class-bound 1937. The force of circumstance in intellectual creativity of a rare order is of its nature protean, unpredictable, maybe long, maybe short in gestation. The rhythms of discovery turn on the recognition of genius, for sure, but also on time and chance. On a tiny scale of coincidence, Turing met John von Neumann because he got to Princeton. Against the horizon of history and quite suddenly, in 1938, the government awoke to the volume of German radio traffic and so, two years later, appointed 60 wartime codebreakers, one of them Turing, to an Admiralty department unchanged since 1918. Thus the computer egg was fertilised.

It is the merest footnote to add that another instance of the incalculable force of original scholarship may be found in the brief tale of the political scientist Philip Pettit, whose book on republicanism was published in 1997 and respectfully reviewed in the house journals. It had, however, one significant reader, who was deeply impressed by its argument. That reader was a politician who would go on to become president of the Spanish Republic. He invited Pettit to Spain to advise him and his parliament how best to make their constitution and therefore their republic more genuinely republican. Pettit advised him, Spain was changed accordingly. Pettit was born in the Republic of Ireland, worked in Australia, and is now in Princeton. How, one wonders, would his impact have been measured in another country?

Telling the rousing tale of singular genius cannot, of itself, bring down the cumbrous edifice of the REF. But it can serve to remind us of the rhythms and accidents of all thoughtful research. The vicious thing about the criteriological document is its venal thoughtlessness.

It makes errors of a kind to warrant the sack from a serious department. The most noisome of these has been, by implication, much noted already. It is the blithe, culpably innocent ventriloquising of the language of advertising and propaganda. This poison is most audible nowadays in the embarrassing convention of institutional straplines on letterheads and job advertisements. It is not easy to accuse these slogans of lying because, presumably, lies are told with the intention of being believable, and no one can be said to believe the sometime strapline of a powerful government body, for instance, Ofsted, which was "driving up standards" (presumably these slogans are left hanging on the end of participles because in the present indicative they would too obviously be lies).

The queasy, breezy language of advertising is just as plain to see, however, in the reiterated use by our enemy of the word "excellent". This is surely a case where we can follow Orwell in simply laughing the word off the page. One cannot suppose anybody sets him- or herself the conscious purpose of producing excellent research. The nature of all enquiry is never so crassly self-referential. You try to do the best you can in the time you have. You are compelled by certain questions, maybe; you have stipulated duties to do research in any case; you want to beat the guys down the road: the motives are limitless but none of them illumined by reference to excellence.

What is far more misleading, however, and makes the whole exercise so intellectually grotesque as well as distorting like a hideous tumour the whole conduct of the conversation of academic culture, is the error made everywhere in the REF guidelines (they say guidelines, they mean commands). It is the error of the incommensurability of paradigms. That is to say, their eyes glazed by the half-understood idea that business ventures are all reducible to the algebra of product management, the giddy innovators and magicians of skills at the department suppose that all research is commensurate. They require that this senior lecturer here, working with a group of local doctors in a study of the sympathetic language of patient consultation is doing the same kind of thing as that physicist over there pursuing, with slightly out-of-date equipment, a bouncing particle that just might be of interest to Cern. When both are set beside this admirable professor whose serious and careful book on the importance of beauty as a concept central to a decent education for children is rejected by some zombie evaluator as not recognisable as research at all, it must be apparent even to the minister for universities and science if, as is highly unlikely, he ever thinks about these matters, that something is terribly wrong.

As it happens, this polemic was voiced at a conference earlier this year immediately before the contribution of the very senior civil servant at the Higher Education Funding Council for England who is, it seems, prime architect of the whole ghastly procedure. He broke off his official account in a spirit of outrance, in order to make what turned out to be scurrilous and underhand rebuttals. He asked what the anti-impact guerrilla would do instead, when their whole point is to say that there is no call to do anything of the kind. He boasted that he wrote cheques for £130 million to the University of Oxford and asked for no report on what happened to the money; in which case why not do the same thing everywhere? He claimed that the citation index was no more, which would come as a surprise to the many heads of department earnestly conning its bogus indices right now. He pointed out that impact has been devalued by 5 per cent this time around, as though we could disregard its one-fifth weighting, which still stands. (If they can knock off 5 per cent just like that, why not do the same to the remaining 20 per cent?) He noted that academics everywhere are busy in his service, making the machinery of impact whirr away, and he was immune to the obvious objection that not only are there quislings and somnambulists in every university, but that in any case they have neither choice nor voice in the matter. Naturally he dismissed the Turing parable as so long ago as to be contemporary with the Norman Conquest. He topped off this revolting display with a sycophantic tribute to the minister as ardent in his horticulture along the groves of academe. As, meanwhile, the ghost of Edward Boyle - an education minister with a serious intelligence who would have sacked a creature who said, as David Willetts, with meaningless sanctimony has, "I believe in choice and competition for universities" - glimmered helplessly behind him.

It was another venerable Tory peer who, years ago, rebuked the voices of science and of "practical activity" as taking far too much upon themselves. Of their nature they belong, Michael Oakeshott told us in 1959, to "the conversation of mankind". "In conversation," he went on, "'facts' appear, only to be resolved once more into the possibilities from which they were made" (this remark might give pause to the REF and the ignorant certitude with which it counterposes "facts" to "opinions"). Then Oakeshott says: "This conversation is not only the greatest but also the most hardly sustained of all the accomplishments of mankind."

Insofar as governments pay universities for their research, it is their duty on behalf of the future to pay them to sustain this conversation. The things that governments want, whether for that vague catch-all, "the economy", or for deadly warfare, or for human happiness, will still transpire from that same conversation, but only so long as it thrives.

This polemic is addressed to the continuity of the conversation. It will be no good to those hoping for tips on how to play the system and bag the loot. Yet there are chances for the chancers. For in paragraph 80 we find this contorted and subjunctive concession, heavy with negatives: "There may be impacts within Main Panel disciplines which take forms such as holding public or private bodies to account. Such holding to account ... may have had the effect of a proposed change not taking place; there may be circumstances in which this of itself is claimed as impact."

As Orwell said of a different kind of filthy writing, "Now for a header into the cesspool." It would indeed be excellent research that thus held the REF to account, which so changed it that it never took place again, that the impact of the research proved so forceful as to blow the whole tinpot apparatus to bits, and to disgrace its designers for good and all.