At first sight, the evidence presented in David Anderson’s report into the four terror attacks committed between March and June sounds damning. The security service, MI5, had had three of the six attackers on its radar. The Manchester bomber Salman Abedi, who murdered 22 people, had come to the attention of MI5 in 2014. As recently as the beginning of this year, he had been implicated in criminal activity, which MI5 officers now admit might have led to his attack being thwarted had it been investigated. Khuram Butt, one of the attackers at London Bridge, had been under investigation for two years, yet still he and his two accomplices were allowed the space to plot and carry out their attack, in which they murdered eight people. Khalid Masood, who struck on Westminster Bridge in March, had also been under investigation by MI5.

Yet these failures have to be set in context. While extremism, for political reasons, is frequently attributed to a tiny minority of people, that is not how it must seem to an MI5 officer trying to assess and prioritise risks. According to the Anderson report, the organisation currently has 3,000 ‘subjects of interest’ in its sights, with a further 20,000 people on the books, who have been under investigation at some stage. It can take a dozen intelligence officers to track one suspect around the clock: there are simply not the resources to track them all. Intelligence can never be foolproof.

Andrew Parker, the head of MI5, says the service has helped foil nine serious plots in the past year, not all of which have come to light. The current trial about an alleged attempt on the Prime Minister’s life is an example of what the intelligence services manage to intercept: their success is remarkable when there are such large numbers of people in Britain who are psychologically capable of contemplating attacks. Accurately predicting which ones will actually carry them out in every case is an impossible task.

The public is not party to the information, for example, which led MI5 once to add Khalid Masood to its list of subjects of interest, but it seems clear that although he was a violent criminal who had served two prison sentences for knife attacks, his conversion to Islamic extremism happened largely alone. How could he have shown up on the security services’ radar other than via some dystopian technology capable of the mass reading of minds?

To give an idea of the difficulties faced by the intelligence agencies, according to Peter Neumann, Professor of Security Studies at King’s College, London, MI5 has sufficient resources to monitor full-time between 50 and 60 suspects — just one in every 50 of those it considers to be currently worthy of investigation. Add to that the challenge of pre-empting lone wolves who have never been in contact with any kind of extremist organisation, and it becomes clear just how large the problem has become. It’s not looking for a needle in a haystack as much as trying to find a specific piece of hay.

The biggest faults with our anti-terror campaign lie not with the intelligence agencies but with the criminal justice system. We could do more about the estimated 400 Britons who travelled willingly to join Isis in Syria and Iraq and who have since returned to Britain. There is no issue of civil liberties involved in this as there was, for example, with suspects who were subject to the now-defunct control orders. Every one of these people, simply by travelling to Isis territory, has committed a criminal offence —yet just 14 of them are known to have been jailed.

A dangerous conceit has developed to the effect that former Isis fighters are merely misguided and that with a bit of counselling and deradicalisation they can be introduced back into British society. Max Hill QC, who is David Anderson’s successor as the government’s independent reviewer of terrorism legislation, said of British Isis fighters who have returned and not been jailed that ‘we should be looking towards reintegration and moving away from any notion that we’re going to lose a generation due to this travel’. Reintegration makes sense for the cases where it is possible. But when it’s not, we end up relying on MI5 to track known jihadists who are adept at covering their tracks.

Isis is now in its dying days, having been forced out of Raqqa and Mosul. But its fighters were escorted out in buses laid on by the Syrian forces and dropped off into a safe space where they are free to relocate. As the bombing of al Qaeda in the Pakistani tribal areas demonstrates, committed jihadists tend to relocate rather than retire. So the threat of returnees, radicalised by what they have witnessed and complete with a network of other terrorists, might be about to become greater than ever.

There is a bizarre disjuncture between how we treat Isis fighters in Syria or Iraq — where we are happy to eliminate them via drone strike — and how we treat them if they manage to make it back to Britain, where we hope they will find gainful employment and tend not to go after them.

There are justified objections to the detention or control of people whom we merely suspect might be capable of carrying out terrorist acts. But if the government decrees that fighting for Isis is a crime, then it ought to go after the offenders. That would help the intelligence agencies thwart the next terror attack.