There’s no avoiding the Britishness of British art. It hits me every time I walk outside and see dappled trees against a silver-grey cloud that looks like it was painted by Thomas Gainsborough, or look in the mirror and feel the same gooseflesh anxiety as I do when I see a portrait by Lucian Freud. It’s got something to do with the light — that pale, ever-changing clarity that is so kind to clouds and, when Freud has got his model naked under the skylight, so unkind to human flesh.

The phrase the Englishness of English art was coined by Nikolaus Pevsner in the title of a classic art-history book — though he called it an essay on ‘the geography of art’ — based on his 1955 Reith Lectures. This omnivorous student of art and architecture traced the national character of English art deep into the Middle Ages, arguing that English monks put much more human fun and natural detail into their illuminated manuscripts than their brethren elsewhere in Europe. Well, maybe — but this is where claims for the national character of art can fall down. Is it uniquely English to look at human life and the natural world? Surely such realism was taken to new heights in the 15th century by Pisanello and Uccello in Italy and Van Eyck in Flanders, at a time when there were no British artists of their stature at all.

In fact, a more cynical way to define the Britishness of British art is by its clumsiness, ugliness and gleeful incompetence. Pevsner was writing before the recovery of Henry VIII’s battleship the Mary Rose, with its astonishingly hideous carvings made by ham-fisted sailors. The ‘sculptures’ on the façade of Sir Paul Pindar’s house, a survivor of pre-fire London preserved in the V&A, are not much better. You can trace this lack of aesthetic ambition right through to today. Grayson Perry’s pots and David Shrigley’s conceptual art cartoons are sniggeringly gimcrack, and that’s essential to their appeal.

Today’s historical fashion is to lament an exuberant Catholic Britain whose rood screens and doom paintings were wrecked by the Reformation, but if there really were lots of Giottos toiling in the Midlands 600 years ago that didn’t stop kings, nobles and merchants hiring artists from abroad. The striking survival of such works as the Trinity altarpiece, commissioned by the Edinburgh provost Edward Bonkil from the great 15th-century Flemish painter Hugo van der Goes to decorate James III’s royal chapel, suggests that no one in late medieval Britain thought local artistic products equalled those from the continent. This fad for foreign masterpieces became a flood by the 17th century, when treasures including Leonardo da Vinci’s ‘Saint John the Baptist’ and Caravaggio’s ‘Death of the Virgin’ were joined at Charles I’s court by living artists including Rubens and both Orazio and Artemisia Gentileschi. Then something changed.

British art is exactly as old as many other forms of Britishness that are still highly visible today. Visit a race meeting or simply see a horse in a paddock and you might be looking at one of George Stubbs’s paintings of shiny-flanked thoroughbreds under Newmarket skies. Go to Covent Garden or St Martin’s Lane and you can still see the London engraved and painted by William Hogarth. Visit Brighton and it remains the resort that John Constable knew, even if the chain pier he painted has gone.

Horse racing, the seaside and the West End are all inventions of Enlightenment Britain and so is British art as something unique and distinctive. From the Glorious Revolution of 1688 to the death of J.M.W. Turner in 1851 the growing wealth and confidence of British society nurtured a bold native art. This art had a philosophy and it is called British empiricism. In his 1689 work An Essay Concerning Human Understanding, the founder of this philosophy, John Locke, demolishes the idea that we are born with innate ideas. All our knowledge comes from our senses. Or as he puts it: ‘Let us suppose the mind to be, as we say, white paper, void of all characters, without any ideas; how comes it to be filled?… To this I answer, in one word, from experience…’

Empiricism filled the empty canvas of British art in the 18th century with a host of human and natural observations. The reason Gainsborough’s landscapes look so familiar is that he observed the Suffolk countryside with an acutely empirical eye that caught the exact light of the local landscape. The same urge to see without prejudice led Stubbs to dissect horses so he could know them from the inside out. Constable digs deep into the matted earthen texture of the countryside. Turner paints seas so real they seem to heave against the picture surface.

In my book Sensations I argue not only that the ‘sensationalist’ philosophy of the British empiricists drove the superb observations of Britain’s first great painters in the Georgian age but also that it still shapes our art today. Empiricism is, of course, congenial to the British national character as it used to be defined. ‘British common sense’, equability, moderation — these were the traits of the Britons as defined by such mid 20th-century observers as George Orwell and Pevsner himself. Common sense is not always seen as the stuff of artistic inspiration. Indeed Orwell takes it as beyond debate in his 1941 essay ‘The Lion and the Unicorn’ that ‘the English are not gifted artistically’. He meant in music too, which seems hard on Benjamin Britten, but — in spite of our desire today to big up Henry Moore and Barbara Hepworth as British equals to Picasso — most people in the age of high modernism would have agreed. They would have been right.

For the rebirth of British art in the 1950s came when artists rediscovered that raw, tough, empirical eye. The art of Francis Bacon, Lucian Freud, Frank Auerbach and David Hockney is wildly subjective: none of these artists look like each other. Yet they all paint their perceptions as honestly as possible.

Britishness in art is not a genetic inheritance or even an effect of the weather. It’s a philosophical idea. Freud and Auerbach proved that when, as children forced to leave Germany, they consciously embraced all that some might see as conservative and staid in their adopted country’s art. Both were to spend lifetimes painting nothing but what they could see. There has never been a more empirical artist than Freud, who seemed to abhor the idea of making anything up. Most people think that’s what imagination is. Freud showed that it can consist in a passion for truth. His naked portraits of Leigh Bowery and Sue Tilley rank among Britain’s greatest gifts to world art: masterpieces painted in an age that seemed to have left naive figurative painting behind. But of course it’s not naive. It is rooted in Locke’s philosophy. No sheet of paper was ever emptier than Freud’s canvas when he began a new adventure of perception.