The inbred Habsburg monarchs, who for centuries ruled without method over a vast, ramshackle empire, managed to leave an indelible mark on modern Europe, says Sam Leith