Nearly a century ago, some Chinese water deer swam the River Ore to Orford Ness. They had escaped from the ornamental deer park at Woburn Abbey in Bedfordshire and, perhaps through ancestral homing instinct, headed as far east as possible without falling into the North Sea. They climbed unsteadily on to this strange shingle spit at the end of England and made it home, along with the gulls, thistles, wild poppies and the brown hares who, reportedly, are too burly to be airlifted by the Ness’s marsh harriers and barn owls.

The pioneering deers’ descendants have borne witness since to many faces of human folly. From the first world war to the end of the Cold War, the Ness was home to experiments with airborne weaponry, radar and atom bombs. What Nevada and Bikini Atoll were to the Americans, Orford Ness was to Britain. In the 1950s newspapers and locals were told misleading stories that the scientific work done here had merely civil purpose, namely to aid merchant vessels so that they didn’t run aground on the Ness.

Today I am following in the little deers’ hoof-steps. I’m taking the ferry to the Ness to experience an art project called Afterness. It consists of a walking tour through the ruins of 20th-century military experiments, passing site-specific sculptures and installations, all the while accompanied by poetry and field recordings broadcast into visitors’ headsets.

When in the 1990s W.G. Sebald wrote Rings of Saturn, his unreliable memoir of walking the lugubrious East Anglian coast, he imagined himself on the Ness ‘amidst the remains of our own civilisation after its future extinction by some future catastrophe’. He supposed its abandoned military buildings, whose reinforced roofs were designed to protect against misfiring bombs exploding, resembled pagodas from a dead civilisation, Ozymandian remnants in Suffolk.

Legends of what may or may not have happened here radiated inland like fall-out. Sebald said it was believed biological warfare experiments here were aimed at making whole regions uninhabitable and that, during the war, pipes were laid out into the sea to make a wall of fire that would boil the sea and repel Nazi invaders. Charred bodies, in some versions German and in others British servicemen, were found on the beaches. The truth may never be clear.

Since the end of the Cold War, the National Trust has repurposed this bleak land of secrets and lies as part of the heritage industry with a healing nature walk through the toxic past. Afterness, whose artworks will be in situ until the end of October, embellishes that cultural offer.

None of the art commissioned by Artangel is as easy on the eye as, say, Maggi Hambling’s metal scallop on the Aldeburgh beach further up the coast, or Constable’s paintings of Dedham Vale a few miles down the A12. There is no tea room and you don’t exit through the gift shop. But still the ferry is filled with visitors armed with maps.

From the boat we see what Sebald described, the ruins of Britain’s military past. Or can we? Given the Ness’s post-truth vibe, I say to my ferryman, just possibly the National Trust and Artangel are useful fronts, distracting us from what fresh hells are being concocted here. Maybe Orford Ness, far from being a mausoleum to the Cold War, conceals its current purpose at the cutting edge of diabolical military research. Very sensibly, the ferryman won’t comment.

Once on land, I stumble over shingle (on no account wear sandals) up to a sign reading ‘Danger: unexploded ordnance. Please keep to the visitor route’. The ground has been cleared but, the National Trust warns: ‘It is possible for buried ordnance to work its way to the surface.’ It’s like Ikea but with more interesting wildlife and the non-negligible risk of being blown to smithereens.

I put on headphones given to me by Artangel workers and follow the trail. As I walk, Ukrainian-born Ilya Kaminsky’s site-specific poetry starts up in my ears: ‘Gulls hold this nation in the quarantine of their eyes.’ I like that line, and remove my headphones. I can see no gulls but I hear a kind of electric airborne trilling as if the birds, thanks to some nuclear error, have been digitised. I can’t see the deer either. Hundreds of them, possibly working for MI5, have melted into the landscape and are doubtless spying on me from behind shingle ridges.

I stop at another National Trust sign, inviting visitors to enjoy what it calls this ‘wellness walk’. ‘Listen to the birds, the wind in the reeds and in the distance the sea on the shoreline… Take deep breaths in and out. Relax your shoulders. Take in the stillness around you.’ But, in truth, this is not just a wellness walk but an unwellness walk. Until the Trust sells the spit for redevelopment as luxury flats or Boris commissions a bridge from here to Amsterdam, Orford Ness will remain a land poised between nature cure and poignant reminder of humanity’s impulse towards self-destruction.

I climb steps to the first building on the route, the so-called Bomb Ballistics building, from which boffins watched bombers drop ordnance into the sea, measuring windspeed and plane velocities to optimise, presumably, future kill ratios. In my ears, Kaminsky adds to the mood of desolation. There is no solace to be found in God, I learn, because ‘in fact God is a little insect that climbs up the windows carrying a breadcrumb’.

I walk on to the next building, the octagonal Black Beacon, originally built in 1928 to develop radio systems for marine navigation. Here Artangel has installed what it calls a ‘library of sound’, recordings including the drone of a single bee and the sounds of boots on shingle. I admire Chris Watson, the sound recordist whose work regularly features on Radio 3, but there is a problem with how his and Brian d’Souza’s field recordings figure here. Art in capturing the sounds of nature silences it, replaces it with disembodied simulation. And that’s my worry about much of Afterness. Orford Ness is already a great unintended artwork. What can art do except at most embellish, at worst erase?

Nearby, in a grim abandoned shelter, Alice Channer has installed a sculpture called ‘Lethality and Vulnerability’, made up of bramble-like growths shaped from aluminium bars. Brambles, the blurb explains, are nature’s barbed wire, a fact to which I can attest having lacerated myself recently reversing a narrowboat into a tangle of bushes on the Oxford Canal.

A mile or so further across the shingle is Lab 1 (pictured above), built in the 1960s for the Atomic Weapons Research Establishment’s testing programme. Today it is a ruin with a collapsed roof that Italian artist Tatiana Trouvé has transformed into a Tarkovskyan hellscape, complete with bizarrely lush greenery and a water feature that may have been installed to cool overheated bombs. Trouvé’s installation evokes the refugee crisis and, no doubt, climate change. Blankets, clothing, radios, suitcases are scattered across the ground and rot in the water, as if a neutron bomb has erased humans but left traces of our possessions. But all is not as it seems: the suitcase is bronze and the radio made of marble, as if Trouvé is memorialising present disasters in the materials sculptors used to honour heroes of the past.

At the end of the trail, I enter the Armoury, a shed where weapons were stored awaiting tests. Here Emma McNally has done a rather brilliant piece called ‘The river that flows nowhere, like the sea’. A large-scale drawing stretches across a frame lying on the floor. It depicts migration patterns, subatomic particles, cloud chambers, weather information and more, a meditation on how we try to measure and comprehend our place in the world. But there’s a problem: the result looks so much like an abandoned Ministry of Defence map rotting back to nature in a shed at the end of the world that many of the visitors I saw passed on by without attending to it.

I stagger back to the ferry with Ilya Kaminsky for company. ‘When I have nothing I still have a hair hand for a pillow,’ goes his grimly funny verse. It’s as if I’m on an especially dismal staycation haunted by the spirit of Samuel Beckett. Have you enjoyed your trip? asks the Artangel worker to whom I return the headset. No, I say. Enjoy is absolutely the wrong word.

As I wait, I chat to other visitors. For several, the art is a cuckoo in the Ness, gaining any life from supervening on its locale.

There is more to experience though. Online, there is a 24-hour radio station called Beacon.Black that consists of a sound collage riffing on the Ness’s history of spying and signal jamming. On YouTube, there is a film inspired by the alleged UFO sighting nearby at Rendlesham Forest. That sighting is already part of the leisure industry: Forestry England have a UFO trail there for kids. It also inspired last year’s Radio 4 drama, The Whisperer In Darkness, in which some hippie witches foolishly summon an evil god through a mystic portal off the B1084 with predictably catastrophic consequences.

Now, in Paul Maheke’s film, which experiments, it says in the blurb, with site-specific choreography (think slo-mo parkour) two otherworldly men in raincoats roam the Forest. Like David Bowie in The Man Who Fell To Earth, they may be aliens whose spacecraft has crashed and now have to pass themselves off as humans.

In the film, the pair pitch up on the banks of the River Ore, looking across to the Ness reflecting, as I did throughout my art excursion, on what an oxymoron human civilisation is, and wondering what strange secret things have happened, and perhaps are happening right now, on the other side of the water. If only Orford Ness’s Chinese water deer could talk, the stories they could tell.