The Prime Minister no doubt knew he would be fanning the flames when he waded into the argument about the admission of black undergraduates to universities like Oxford and Cambridge. We should do him the courtesy of trusting he means it when he says he feels strongly about discrimination in the awarding of university places – and I think he does. In this week’s issue Toby Young marks David Cameron’s essay with tutorial authority, and finds his case wanting.

Particularly valuable among Toby’s marginal notes is his point that you can’t accept applicants if they haven’t applied – and black and working-class students disproportionately don’t. But we enter a vicious circle here. People don’t apply if they don’t think they have a chance, don’t see many faces or hear many accents like theirs among the successful applicants.

And Toby’s argument is prefaced by a shrewd piece of bomb-proofing. Toby says he his talking about BME (black and minority ethnic) students. He then goes on to demonstrate that Indian, Chinese and other minority ethnic groups whose origin is east of Suez, are not a problem. But the PM was not focussing on these. The problem is black students. Despite the race industry’s attempts to conflate all non-whites, black means black: people of African and Caribbean origin. There is a problem here and Toby does not suggest otherwise. His urgent purpose is to add white working-class boys to the list, and he certainly makes that case, but it’s a different – if important – argument.

So what can be done for black applicants? I have become a firm believer in positive discrimination – in questions not of race alone, but of background and of gender more generally. And I should add that systematically overlooked issue for us British: class.

As a younger Conservative and as an MP I took the line Tories instinctively take: all selection should be on ‘merit’, and ‘the best candidate’ should unfailingly win the prize. Labour’s idea of (for instance) all-woman shortlists appalled me and still does; positive discrimination for jobs, university places or parliamentary candidatures in favour of candidates from minority ethnic groups seemed to me to discriminate unfairly against white applicants. Fair was fair and that was that.

I’ve come to believe it isn’t so simple. If of course you believe that black, brown, working-class or female applicants might as groups simply be inferior human beings then you’d have to conclude that this was among the reasons why they tended not to be selected. But what if there are other reasons? Let me suggest three. First, confidence. Secondly, polish. Third, unconscious bias on the selectors’ part.

This third, unconscious bias, is troubling. I was dismayed to try an online test for racial bias the other day, and find that I was ‘mildly’ discriminating against black (not BME) faces. This surprised me. Less surprising (because I’ve mediated many Conservative selection meetings for parliamentary candidates) is the observation that people doing the choosing have more difficulty in seeing (say) a woman as an MP – and so, though they do mean to be fair and apply the same standards to all, will find themselves less impressed by a woman’s performance.

Which brings me to the first of my suggested reasons for bias in selectors: ‘confidence’. In candidate selections I’ve noticed that men have a certain swagger that women often lack. Swagger does not equate to competence, but sometimes forget this when we are (and there’s no other phrase) taken in by an individual’s self-belief.

And this brings me to the third characteristic that can distract us in our quest for pure merit: polish. There really is something about a private education, a certain patina — fluency, articulacy, command — that a good public school can impart. An American lady once remarked that so convincing was an Englishman with a public-school background that you needed to marry him and live with him for ten years before you realised he was basically just thick. Such candidates — be they for a lady’s hand or for a place at Oxford will impress in ways that appear to selectors to indicate merit but don’t.

Successive studies have suggested that, once admitted, state-school pupils do better at university than their independent-school peers with comparable A-level grades. But, like daffodils, children can be ‘forced’ to blossom at the right time in the right way: in this case, achieving examination grades. Private education can do this. But once the university course is embarked upon, with its emphasis on understanding rather than regurgitating, the state-school girl or boy comes into their own. They had previously been held back.

And if we accept that something may often be holding back the black university applicant, or the woman in the jobs market, then surely considerations of ‘merit’ invite us to compensate for that handicap? One can continue to believe in selection by merit, continue to believe in getting the best person for the job or university place, but in assessing merit one will now take intelligent account of [ITAL] potential [ROM] as well as already-demonstrated performance. Once you realise that a car has been being driven with a handbrake on, or a child educated without much encouragement from peer-group, family or teachers, you adjust your expectations of how that car would perform unbraked, or how that child could perform unhindered.

And the odd thing is that any parent, any good teacher, and any good employer knows this instinctively and through experience. We all know the importance and justice of ‘bringing someone on’ — of giving them a chance, that is, that past performance may not appear to warrant, in the belief that they may just need a bit of a boost.

This, of course, is a principled argument for positive discrimination: it is not in itself a workable proposal for adopting the practice at the semi-formalised level at which politics, university admission or human resources have to work. Systematising a good idea can poison it. Any points-based system that adds or subtracts points in a tick-box manner on the basis of race, gender or class, carries its unfairness on its face and will prove counter-productive. Anyway (as Toby Young points out) it would at present be unlawful.

Which leaves us with interview. Here, mercifully, the law has no window into thought-process. Interview, as Toby does not quite acknowledge, can be an important part of university admissions policy. And it is at this stage that David Cameron’s remarks deserve quiet, careful study. We should try to find ways of giving effect to his exhortations in the informal and unacknowledged way that interview can. I’m sure the best universities already do. They should try harder.