Thirteen years ago, when John Reid became Home Secretary, he declared the ministry he presided over ‘not fit for purpose’. He was talking about border control but he might well have been referring to the department in general. When Theresa May became PM, things at the Home Office went from bad to worse. Her paranoid style as PM, developed as a survival mechanism when she was Home Secretary, was disastrous.

The Windrush debacle, the most shameful example of government dysfunction in recent times, was a sign that change was long overdue. And yet this chaotic department, riven with internal feuds and a lack of accountability, now has a gargantuan task ahead of it. Within a matter of months it must produce a post-Brexit immigration system and sort out the EU settlement scheme — knowing that, as with Windrush, failure will have human consequences.

Boris Johnson is committed to a plan that would test the most efficient of organisations. The Home Office has nine months to get this country ready for the most profound changes to its borders in the postwar era. In these circumstances, it was inevitable that tensions would flare, and it is no real surprise that Priti Patel and Sir Philip Rutnam, her former chief civil servant, have fallen out so spectacularly.

Sir Philip saw no irony in making an organised media appearance to complain about an ‘orchestrated briefing campaign’ against him. But as the news demonstrates on a daily basis, there is spin on both sides. Perhaps most telling was Westminster’s assumption that this would all be sorted out behind closed doors because both sides had too much to lose if it were not. Boris Johnson was aware that he hadn’t sent the most competent minister to the department (Patel was chosen for her decisive style, to assure voters that their concerns are being taken seriously). Sir Mark Sedwill, Rutnam’s predecessor, had no desire to have the hood lifted on the department’s institutional failings — all of which Sir Philip oversaw.

An investigation is now under way. If Patel has been proven to bully people — in the proper sense of the word — then she should go. Being abusive towards civil servants is neither defensible nor effective: many ministers may look in horror at the mess they find, but the trick is to take a firm grip and improve things (as Michael Gove did as Education Secretary). However, presenting challenges, and demanding solutions rather than endless lists of problems, should not be regarded as bullying. Working in government is stressful; and working at the department charged with protecting the public is particularly demanding.

Nonetheless, the Tories cannot afford a war with the civil service. The Thatcher government was effective because she identified and promoted dedicated civil servants who worked to transform government — while being aware of the limitations of any machine of its size and nature. It is impossible to reform any government department to make it run as efficiently as a private company, and it is dangerous to assume otherwise.

Tories should also be careful of their own prejudices. Many Brexiteers became convinced that elements in the civil service were trying to keep Britain in the single market and the customs union by the back door. This was true, but only because civil servants were following direction from the likes of Philip Hammond. Treasury civil servants who held up ‘no-deal’ planning or threw up every objection to leaving the customs union were only doing what the Chancellor wanted them to do.

If demands on the civil service are to increase, then so should their pay. This might be seen as rewarding bad behaviour, but you can’t recruit top talent or expect the civil service to be as efficient as the best business if it pays charity-sector wages. The pay squeeze of the last decade took its toll: Tory ministers who had served in the last Tory government and then returned to office in 2010 were shocked by the decline in standards in the intervening period. Some civil servants spelled their own minister’s name incorrectly in letters. If they do that, how likely are they to be able to deliver radical policy? Those who aren’t up to the job should lose their jobs. The Whitehall practice of promoting duds out of the department needs to end. There also needs to be greater accountability. No. 10’s desire to have named officials in charge of projects, and have them stay in post until these are complete, is sensible. Project management is the most undervalued skill in Whitehall and improving that would make more difference to the functioning of the British state than anything else.

Opponents of reform will try to use Sir Philip’s resignation (and the allegations against Patel) to stop any improvements to the current system — and, ideally, stymie the government. They will argue that any attempt to change the status quo is a bully’s charter. But the truth is that under governments of all stripes, the British administrative machine has been found wanting. The surge of powers back to Whitehall from Brussels makes it imperative that the government works better. This requires better leadership from both sides.