What happened on Streatham High Road last weekend was exactly what I had feared during my time as prisons minister: a recently released convict mounting a terror attack. It was the second incident of this nature in Britain in three months, but the truth is we are lucky that there have not been more like it.

When I was appointed prisons minister in January 2018, I was introduced to the full scale of the problem. In one wing of a prison in Liverpool, half the windows were broken. Prisoners could stick their hands straight out to take drugs from drones. In Lewes, an X-ray scanner — which was meant to detect what prisoners were smuggling back into the cells — had been defunct for seven months.

Ancient Victorian prisons like Wormwood Scrubs needed to be closed (not least to free up land for affordable housing in London) but there was nowhere else for the inmates to go. Across the UK, 82,000 prisoners are crammed into cells designed for 62,000. On one visit, I saw a man walk straight up to the governor, shout ‘Fuck off’, and continue without any reprimand. Violence had tripled in five years. There were more than 10,000 assaults on prison officers a year.

I spotted something else: most terrorist offenders had only been convicted of relatively minor offences, so would be released sooner or later. It meant there was only a short period in which to make these men less dangerous. Deradicalisation is possible and has succeeded in the past. But when it doesn’t work, the impact is horrifying. Prison officers are extraordinary public servants but they are not experts on radical Islam. Hard-working prison imams would admit to me that they struggled to debate jihadist ideology with educated fanatics.

And how to ensure that the terrorists didn’t radicalise others? When serving in Iraq, I saw how the mass incarceration in the Abu Ghraib prison created a terrorist training school. When the US army released these prisoners, many of them formed the core of Isis. Does it really make sense to house terrorists in one place? Ian Acheson, a thoughtful ex prison governor, had proposed solutions. He had pushed to establish specialist separation centres. But Ian had left, saying that he found the system defensive and reluctant to engage with many of his recommendations.

What does a minister do? I began with the basics — saying I’d resign if we didn’t reduce violence. We championed prison officers, wrote a new handbook, doubled the maximum sentence for assaulting officers, fixed broken windows, bought X-ray scanners and improved rehabilitation programmes. But safer and more orderly prisons, although essential, are only the beginning.

It’s amazing how much we don’t know about prisoners. It is almost impossible, for example, to know how many prisoners are university graduates, let alone acquire a rich case history of an individual terrorist. The data does exist — scattered in separate databases from the NHS, police, social services, prison and probation. But putting it together is expensive and legally challenging: the government should at least immediately put all prison and probation records into single searchable files.

We must also train a new cadre of specialists, who know how to work with terrorists and manage risks. And they need to be continually retrained. The kinds of terrorists I encountered in Afghanistan in 2001 (educated, ideologically aware and technologically sophisticated) are completely different from the recent London attackers (less educated, and not part of any formal organisation).

Prisons are isolated places, from which people often emerge more dangerous than when they entered. This can only be fixed if we focus on the fundamental details. That requires fierce independent inspections, more willingness to admit mistakes, and a better balance of compassion and discipline. Only by doing this can we reform those terrorists who can be reformed — and protect ourselves from those who cannot.