The first time I met Tommy Robinson I told him to fuck off. The English Defence League (EDL) had just formed and Robinson came up to me after a public interview I was doing in London. Without knowing anything much about them, I am afraid I assumed (white, working-class, Cross of St George at demos) that the EDL were a British National Party front. Which was why I ended up advising him of the procreative way in which to travel. He took it very politely, said he understood that I didn’t know their views and then said, ‘We’re not racists — we’re just working-class guys who are losing our country and can’t bear it.’

Last week, four-and-a-half-years on, we met again. Several days earlier Robinson had announced he was leaving the movement he had formed, saying — to some guffawing — that he was no longer able to control the genuine far-right elements who sought to hijack his movement. He was aided by the Quilliam Foundation, an anti-extremism think tank, and it was a close Muslim friend of mine who had been guiding Robinson through some of this who called me up to ask if I would hear Robinson out. I agreed.

How has the reaction been since leaving? ‘Difficult,’ he says. ‘People saying I’ve been bought out.’ It’s not just some former friends in the EDL who have it in for him. While walking through his home town of Luton, accompanied by a film crew, he was attacked by a gang of Muslims who punched him to the ground. ‘I went flying back on my arse,’ he says. ‘Another three came over and said, “We’re going to decapitate you.” It’s all on camera: “You need to be decapitated.” I’m sat there thinking, this is what the world needs to see, this is what middle England needs to see. But then at the same time I said, “Am I going to hate every Muslim for that?” It’s been a difficult week.’ But, he adds, ‘For me it [leaving the EDL] was the right thing to do.’

Robinson had been thinking about leaving since a stint in jail earlier this year. Sent down for travelling on a friend’s passport, he was put in solitary confinement to protect him from Muslim gangs in the jail. It was probably the best thing that could have happened to him. ‘Before I went to jail I was a mess. I was a complete mess — drinking and just sessioning. I was put on solitary confinement for 18 weeks and it was like a break. But that’s then the first time that I started questioning where it was going. Because there is violence. I don’t believe it’s been English Defence League committing it, mainly it’s been provoked. There is racism. I call Muslim leaders apologists all the time. If I don’t admit the problems — which I haven’t done really for four years, I’ve made excuses for a lot of it — then I’m an apologist myself.’

So was the organisation far-right? ‘It’s not a far-right organisation. We’ve been battling from the inside of this movement for four and a half years to be all-inclusive. To not be far-right at all.’ They have had problems. To hear him describe some of the groups which circled around to get onto the EDL’s bandwagon is to glimpse a very dark corner of our society. But Robinson says that the politicians and press did not help. He claims that by calling the EDL ‘far-right’, politicians and the media have actually sent such elements their way. ‘It’s like they want it to be a far-right bunch of lunatics, instead of a bunch of ordinary normal people.’ Why would they want that? ‘So they don’t have to have the debate. So the issues that we’re raising don’t have to be spoken about.’

And there certainly are issues. Robinson was in Luton town centre in 2009 when the Royal Anglian Regiment’s homecoming parade was barracked by the radicals of Al-Muhajiroun. In a disastrous policing decision Al-Muhajiroun were protected by the police and angry locals threatened with arrest. Any politicians or opinion formers who think there is not a problem in places like Luton have never been there.

Robinson boasts of the diverse racial and religious background in which he grew up and in which he appears genuinely comfortable. It wasn’t the diversity but the double-standards that, he says, provoked the EDL. When he and his friends organised a protest to oppose the Islamists, they were prevented from getting to the Town Hall through which Al-Muhajiroun had previously walked. And while Al-Muhajiroun had flyered mosques for their protest with impunity, Robinson says he and his friends were stopped by police from handing out leaflets. ‘They knew it had a potential,’ says Robinson. ‘They put their hands in our pockets, cameras in our faces, made people take their shoes off. I was like, “You didn’t do this to them. What are you doing this to us for? Why are you treating us like this when you didn’t treat them like this?” ’

He now acknowledges, however, that the EDL’s professed message — opposition to Islamic extremism — was lost almost from the outset. Some of this, including some disastrous generalising about Muslims, Robinson now admits to be his fault. Other problems, including the opposition which formed against them, were not. Supporters of Unite Against Fascism often got involved in serious violence when they turned up to ‘oppose’ the EDL. But UAF receive significant political support, whereas from the beginning the EDL were pariahs whom no one in power could dream of supporting. Robinson and his friends see some of this as a class issue and perhaps they are right in part. Certainly there is a disenfranchisement issue. Robinson says he does not know anyone who votes, and Luton borough council, which talks to Islamic extremists, said it would ‘never’ sit down to hear -Robinson out.

He does have a litany of charges against him. Robinson is currently awaiting trial after a tax investigation carried out by the police and says all his immediate family have had their finances examined by the police in recent years. He says his own bank account and all assets have been frozen for four years and his businesses run into the ground. He is apparently allowed access to only a small sum of cash each week.

There is a video of Robinson being arrested while attempting to walk through Tower Hamlets and he tells me that he was arrested for incitement after (by his account) one EDL demonstration went three minutes over its allotted 30-minute running time.

Then there are the ‘constant, constant death threats’. When somebody posted his mother’s address online and promised to ‘chop up’ Robinson’s kids he finally went to the police. He says they told him they could do nothing about it. He began retweeting Twitter threats, but says he was told by police that if he continued doing so he could face arrest himself.

He has been repeatedly attacked. On one occasion, when set upon by a Muslim gang in Luton, the police handcuffed and arrested him. On another, the perpetrator was caught on camera but has still not been arrested. He says he’s given up on the police: ‘They’re scared.’ April this year saw the trial and conviction of six Muslims from the West Midlands who planned to carry out a terrorist attack at an EDL demonstration in Dewsbury last June. They missed the demo by minutes but while returning their car was stopped by police and found to contain nail bombs, knives, sawn-off shotguns and a -message to ‘To the EDL. O enemies of Allah!’

Robinson tells me that he has spent four and a half years being called a racist while fighting to keep actual racists out of his movement. But now he has given up that struggle. There will be those who will rejoice at that. Most of us will breathe a sigh of relief if the EDL’s brawling protests now cease. But even if it does go away, our authorities would be very unwise to keep ignoring the issues that gave rise to this reactive -movement.

As we say goodbye, I cannot help reflecting that our society would never have heard of Tommy Robinson if it had dealt with Islamic extremists with anything like the severity it has meted out to him.