‘It was hard to be a tsar,’ Simon Sebag Montefiore writes in his opening sentence, and what follows fully bears this out. In his thought-provoking introduction, he stresses the unique nature of Russian autocracy and its perverse contradictions; the tsar was absolute ruler, yet he was bound by a tangle of restrictions. His subjects were prepared to accept his tyranny and any cruelty its exercise required, but claimed the right to punish him if he failed to provide strong leadership. The system was never meant to give one person tyrannical powers over everyone else. Nor was it intended to work for the greatest good of the greatest number. It was a very different animal from anything we, with our traditions emanating from ancient Greece and Rome, Judaism and Christianity, have ever known. It was, in the words of the French woman of letters Madame de Staël, ‘autocracy tempered by strangulation’.

The Romanov dynasty reigned for just over 300 years, from 1613 to 1917, and while they expanded their empire at a spectacular rate (an average of 20,000 square miles a year), not one of their 20 monarchs slept easy — with good reason. Few died of natural causes.

The founder of the dynasty was the 16-year-old weakling Michael Romanov, a great-nephew of Ivan the Terrible. He was bullied into accepting the crown as Muscovy emerged from the decade and a half of civil war and foreign invasion known as the Time of Troubles, in which a succession of his predecessors, and most of his family, had suffered more or less grisly ends. Surprisingly, Michael succeeded in stabilising his realm politically and imposing a degree of deference to the person of the tsar. He used religion to create a numinous aura around the throne, and involved the nobility in the process of strengthening it. Much of the ritual he introduced, such as the selection of his wives through glamour contests and their segregation in something resembling a harem, was distinctly oriental in flavour.

His son Alexei was tall, strong, clever and ruthless, and well-suited to empire-building. In 1649 he published the first collection of laws. As it came after a long period of instability, the accent was on reinforcing the state, which meant the position of the tsar. It identified a litany of crimes against his person and introduced a new concept: the duty to denounce. His subjects were obliged to report not merely real conspiracies, but also their wildest suspicions regarding possible ill-intent. It prescribed draconian punishments — mild disrespect for the tsar entailed having one’s tongue torn out — and established the protection of the state as the ultimate purpose of Russian law.

Alexei also strengthened the connection between noble status and service to the crown, rewarding the nobility with the right to own serfs, thus institutionalising a pattern which became a mainstay of the Russian system: servility to those above, tyranny over those below. Taken together, his reforms ensured that Muscovy would develop in a different direction from all other European states.

Alexei’s son Peter the Great was determined to modernise his kingdom along the lines of what he had seen on his travels through western Europe. The magnificent showpiece city of St Petersburg testifies to his success in one respect. But while he strove to civilise his subjects, prescribing the clothes they should wear and how they should drink tea, he himself indulged in epic bouts of drunkenness, bestial depravity and juvenile profanity, involving giants, dwarves and grotesquely disfigured cripples cavorting in ribald farce. He was also capable of breathtaking cruelty. He even tortured his own son to death. And his introduction of the table of ranks, which effectively turned every individual into a cog in the machine of state, brought Russia closer to the structure of the horde of Genghis Khan than to any other European society.

Peter’s reign was followed by a murderous merry-go-round, as rival factions backed various members of the Romanov family and the succession leapt from one branch to another until there were none left. The last actual Romanov ruler was Elizabeth, who was succeeded at her death in 1761 by her nephew Peter III, a Holstein-Gottorp. He in turn was succeeded by his wife, Catherine II, originally Sophia of Anhalt-Zerbst, and from then on the dynasty was effectively German, and almost certainly without a drop of Romanov blood.

The German influence attenuated the savagery and put an end to the spasms of violence which had characterised life under the Romanovs. But though Catherine the Great corresponded with Voltaire and basked in the admiration of Enlightened European opinion, her successor was nevertheless bludgeoned to death in his own palace by his courtiers. His son, Alexander I, also won the admiration of Europe for his part in the defeat of Napoleon and his liberal leanings. But he failed to steer his empire in the direction these suggested. At a moment when Russian society was drawing closer to Europe, he backed away. He abandoned his plans to abolish serfdom, the principal obstacle to reform, and, turning to mysticism, clamped down on all expressions of liberalism.

His successors embraced the concept of Russian exceptionalism; that Russia was by its very nature destined to be an autocracy whose only legitimate ruler was one prepared to rule as an autocrat. The 19th-century ‘Romanovs’ thus present a pathetic spectacle of decent, hard-working German royals struggling to personify the soul of Russia. While the parricidal struggles and rackety Rabelaisian revels of the previous centuries gave way to automatic succession and familial gemütlichkeit, these essentially kind and sensitive men nevertheless felt obliged to preside over a repressive and cruel system.

Convinced that a tsar must show military prowess, they embarked on wars that only served to demonstrate the weakness and incompetence of their military machine. Catastrophic defeat in Crimea forced on Alexander II some reforms and the emancipation of the serfs. But that did not prevent him from being blown to pieces by a terrorist bomb. Denied any legal means of expressing opposition, dissenters increasingly turned to terrorism. The intrusive and brutal secret police, the Okhrana, were unable to provide security, and government officials as well as members of the imperial family were assassinated wholesale.

Dodging the bullets and bombs of assassins, the genial, bear-like Alexander III reasserted the prestige of the monarchy by ruling like a real autocrat, controlling every aspect of life throughout his realm. But this only served to set a trap for his pathetic successor Nicholas II, who was simply not up to the job but felt a strong sense of duty to fulfil what he saw as his divinely ordained mission. He also reached for war as a means of enhancing the prestige of the throne, but humiliation by the Japanese, ‘yellow monkeys’ who were expected to crumble before Russian might, fatally weakened it. The development of the navy — which steamed into catastrophe at Tsushima in 1905 — had been entrusted to the flashy dresser Grand Duke Alexei, whose predilection for fast women was accompanied by the delivery of slow ships.

Once again, reforms had to be conceded and representative bodies sanctioned. But Nicholas resolutely clung to the role of autocrat. Reading about his well-intentioned, bungling progress towards his tragic and squalid death is almost unbearable.

With its sordid power struggles, violence and brutality, its cast of magnificent monsters, tragic victims and grotesque ‘holy men’, this is an extraordinary and gripping tale. Simon Sebag Montefiore has plumbed a remarkable range of hitherto unused archives which shed valuable new light on the private lives and behaviour of his subjects, and help explain their vulnerabilities and the source of much of the otherwise inexplicable violence and cruelty.

By turns horrific, hilarious and moving, but ultimately tragic, this is essential reading for anyone interested in Russia or simply puzzled by the workings of the Russian state. It provides ample evidence that whoever is in power, and however incompetently or cruelly they may be governed, the Russians are dedicated to autocracy.