I’m still weighing up whether to run for Parliament, but after this week’s reshuffle I’ve concluded I’m in the wrong party. If you’re a middle-aged white male, particularly one who’s been to Oxford, your chances of becoming a Conservative minister are negligible. Unless you’re a pal of George Osborne’s, obviously, in which case it doesn’t matter if you have B-U-L-L-E-R tattooed on your knuckles, you’ll still get promoted.

In the Labour party, by contrast, coming from a privileged background actually seems to help. I’m not just talking about the usual suspects, like Lord Longford’s niece Harriet Harman and ex-public schoolboy Ed Balls. I’m thinking of the new shadow education secretary. Under Ed Miliband, holders of that particular office have been getting posher and posher. Indeed, if you put Andy Burnham, Stephen Twigg and Tristram Hunt side by side it would be a bit like that famous sketch with John Cleese and the Two Ronnies. It’s not that much of a stretch to imagine Twigg saying, ‘I look up to him because he is upper-class, but I look down on him because he is lower-class.’

Hunt’s rapid rise has struck a chord in me because we’re both Hons. Admittedly, as far as titles go, being an ‘Hon.’ because you’re the son of a life peer is about as unimpressive as it gets. I’m sure Tristram has never used it, not even when trying to get an upgrade on Air India (which works, by the way). But the distinction between real ‘Hons’ and bogus ‘Hons’ is probably lost on Joe Public. As far as he’s concerned, we’re all posh gits. It doesn’t help that the new shadow education secretary is called ‘Tristram’. Michael Gove probably won’t need to rub it in by referring to him as ‘the Right Honourable Honourable’, though I expect he will.

Here’s the thing, though. Having a title is less of a handicap in the Labour party than it is in the Tory party. People will point to Cameron’s pedigree as proof that this isn’t true, but it’s precisely because the leader is so obviously to the manner born that the faintest whiff of privilege in the lower ranks is the mark of Cain. He’s desperate to disabuse the electorate of the idea that the Conservative party is the party of the rich. Hence the bizarre spectacle of a millionaire Old Etonian defenestrating loyal, hard-working ministers for being ‘too posh’.

Labour has less of an image problem when it comes to toffs. On the contrary, sticking a few well-brought-up young men on the front bench is good PR. It dispels the notion that the party is hamstrung by ‘the politics of envy’ and reassures Lib Dem refugees in the south-west that their vicarages won’t be hit by the mansion tax. With the Hon. Tristram Hunt in the cabinet, there’ll be no pips squeaking under Labour.

The other reason I’d be better off trying to become a Labour MP is because my father was a prominent member of the party. He wrote the 1945 manifesto and served as Labour’s research director from 1945-50. Any suspicion of nepotism is the kiss of death if you’re a Tory — it reeks of inherited privilege — but not if you’re a socialist. Just look at the careers of Euan Blair, Will Straw and Jack Dromey, the so-called ‘Red princes’. Now, I’m not saying they owe their success to Mummy and Daddy, but the danger that it might look that way hasn’t held them back. On the contrary, their left-wing lineage probably reassures the comrades that they’re good Labour men who will look after the party’s -interests.

You’d think that being a Labour princeling who’s crossed over to the other side might be a help in the Tory party. After all, I’ve seen the best of what Labour has to offer — I’ve broken bread with the heroes of the Attlee administration — and I still prefer Conservatives. Surely this should reassure the party faithful that their policies, their sense of what’s right, their idea of what Britain stands for, are all superior to Labour’s — which, of course, they are. But it doesn’t work like that. Instead, they can never quite overcome their sense of suspicion. In their eyes, I’ll always be a member of the Labour tribe, however loudly I renounce my heritage.

So there it is. I have all the right credentials to excel in the Labour party — a title, an Oxford degree, a VIP father — and yet I’ve thrown in my lot with the Tories, where all those things are terrible disadvantages.