Buried deep in the Villa dei Papiri, covered by the molten lava of Vesuvius, lies one of the finest libraries of the ancient world. But excavation may destroy more than it saves

They look like lumps of coal, and when the Swiss military engineer and his team who first explored the buried town of Herculaneum in the 18th century encountered them, that was how they were treated: as ancient rubbish, to be dumped in the sea.

But before being hit by a cascade of molten volcanic rock at more than 400C (the so-called pyroclastic flow that inundated the town), these now-blackened and nondescript objects were part of the library of the grandest villa in the town, where the father-in-law of Julius Caesar was regaled with the epigrammatic gems of his in-house Epicurean philosopher, Philodemus.

They were the papyri on which the ancient world preserved its literature, as the tunnelling archaeologists of 250 years ago belatedly understood. Some 1,800 have so far been recovered, and although both papyrus and ink were carbonised, modern thermal imaging techniques have made it possible to decipher them, with the help of a considerable amount of computing muscle.

Half have already yielded their secrets. None are likely to enter the best seller lists: mostly they are works of Epicurean philosophers, like Philodemus, the one-time resident of the villa. Indeed, although he died a century before Vesuvius’s disastrous eruption, the papyri discovered so far may well have come from his private library. But experts suspect that only a fraction of the papyri inside Villa dei Papiri (”the Villa of Papyri”), as it is known, have been discovered. New excavations in the 1990s revealed two more previously undiscovered floors to the villa, below those already explored. But because the entire villa is encased in tufo, the tough stone that results when the pyroclastic flow hardens, a major task of engineering and archaeology is required to find what more remains to be brought to the surface.

A group of classical scholars is now calling for excavations inside the Villa of Papyri to be resumed without delay. Thanks to the fluke of its preservation within the inferno of the eruption, this is by far the oldest extant library in the world. And nobody has a clue what is in it. It is known that its owner when Philodemus was alive was Lucius Calpurnius Piso Cesoninus, a senator and a wealthy, cultured figure who entertained Roman high society down here at his fabulous country pad by the sea. The villa was full of beautiful vases and statues and other works of art, many of which are now in a museum in Naples.

It is highly probable that Piso also possessed a large library, as became someone of his wealth and culture: not merely the works of Epicurean philosophy that reflected the special interest of Philodemus, but all the other works, Greek and Roman, with which a man of his civilised tastes could be expected to be familiar: the plays of the Greek tragedians, for example, or the dialogues of Aristotle, or Livy’s History of Rome. And given the freakish survival of Philodemus’s collection, it is argued, the rest of the library may be in a similar condition: carbonised but accessible. The figure that has been suggested as the likely cost of bringing them back to civilisation is between €20m (£13.6m) and €30m. But the prize, Robert Harris, author of the novel Pompeii, and the scholars argue could be quite literally priceless: our knowledge of the literature of the ancient world could double overnight, with this single excavation.

But at the Villa of the Papyri all is quiet: no drills or jackhammers batter at the villa’s tufo shell, no new mines are being bored through the rock, no teams of volunteers sift spoonful by spoonful through the recovered debris.

In fact there is nothing going on here at all.

The villa was built a couple of hundred yards away from the town of Herculaneum, set apart from it along the beach that the eruption of 79AD destroyed. Today it occupies a site adjacent to the ruins of the ancient town, separated from it by a seedy lane lined on one side with old tenements and newer but already shabby-looking apartment blocks strung with washing. Groups of British and American and French tourists pad about through the ruins of Herculaneum, which looks like a fragment of Grozny after the Red Army had been battering it for a couple of years.

The tightly packed houses, shops, temples and taverns are built of diagonally set, cream-coloured stones: all are roofless and with weeds and wild flowers sprouting from the walls, though structurally they look in remarkably good shape.

But nobody pads around the Villa dei Papiri site: it is only open for groups with special permission. When I visited this week it was completely deserted. Behind a high concrete entranceway and massive steel gate, more befitting a municipal refuse site than an important ancient monument, what remains of the Villa of the Papyri is wrapped in its rock-encrusted sleep.

And now the scholars are demanding to know why. Last year they formed the Friends of Herculaneum Society, and with Robert Harris have begun lobbying for excavation to begin again as soon as possible.

Professor Robert Fowler, professor of Greek at Bristol University and a trustee of the new society, said: “Everyone thinks it is possible that there is more to be found, because of the very peculiar, one-sided nature of the library as so far discovered: this is one of the great country houses of one of the great Roman potentates. Where are the other philosophers? Where are the Greek poets? Where are the Latin books? If you were under siege by a volcano, would your first priority be to get the books out? We have an obligation to finish the excavation.”

Robert Harris said: “The promise and potential there is immense. This is the wellspring of Western civilisation. There could be the lost dialogues of Aristotle down there, the lost plays of Sophocles, poems of Catullus – it’s just priceless. Anything that could be done to get it out should be done. It’s irresponsible to leave it in the hope that it could be got out in 50 years.

“It’s a battle to keep people interested in history and ancient history in particular. And this is a great story, it captures the imagination of people.

“It’s a thrilling thing that stirs the imagination. The sad fact is that the preservation of what’s already been excavated is essential, but it’s not sexy. I hope both can be done.”

So why aren’t the archaeologists and engineers busy burrowing under the Villa of the Papyri now, as we speak, to bring this hypothetical treasure to the light? The essential reason is contained in a paradox: under present circumstances, the only way to ensure the survival of whatever may emerge from the villa is to leave it exactly where it is, encased in rock.

Suppose new excavations were to start tomorrow, and next year half a dozen lost tragedies of Sophocles, say, were brought to light. Once the spectral imaging technology had got to grips with them, their survival for eternity would be guaranteed; a year or two after that one would be able to buy the Penguin Classics translations in any bookshop.

But what about all the other stuff that would inevitably emerge at the same time as the precious lumps of carbonised papyrus? Because the library, if it exists, will not be uncovered in isolation. Everything interred with it will come to light too.

Herculaneum is unique in that the mantle of rock that encased the town preserved not only the sort of things that can be found in sites all over the world, such as stone and pottery, but organic material as well: papyrus, but also wood, cloth, rope.

Andrew Wallace-Hadrill, the director of the British School in Rome and of the Herculaneum Preservation Project, said: “The ancient town is in a great pit beneath the modern town, and it was preserved by the molten rock like a fly trapped in amber. This is one of the very few excavations in the world where organic material has been so extensively preserved – wood, papyrus, cloth, rope, mostly wood that was carbonised by the pyroclastic flow. Things that survived include furniture, cupboards, beams, doors.”

Herculaneum in this respect is very different from the more celebrated Pompeii, which receives 10 times as many visitors. “At Pompeii you have the feeling of a bare skeleton, impressive for its scale but with too little intimate detail. But in Herculaneum you walk round a corner and come upon a little tavern with a wooden wine rack, storage loft and back office with sliding doors,” Mr Wallace-Hadrill said.

“A villa of this quite exceptional magnificence evidently had many treasures other than its papyri, including other types of documentation (wooden tablets with legal transactions and records) and an abundance of other organic materials, including grain and foodstuffs, fabric and wooden furniture.”

But with rain, polluted air and pigeon droppings assailing the site daily, that unique organic fabric is crumbling fast.

“The Herculaneum site was mostly excavated in the 1930s,” Mr Wallace-Hadrill said. “Now it’s undergoing a conservation crisis – it’s crumbling away. It’s hard to believe if you didn’t see it with your own eyes. Really beautiful fresco decoration flaking off, lumps of plaster that have come away lying at the bottom of walls. We’ve got a whole team of people trying to halt the worst – it’s an emergency first aid job, taking it area by area. It’s an enormous undertaking. If it had been properly maintained it wouldn’t be such a huge problem.

“Stuff falls off the walls in great chunks. It’s like in the scene from the Fellini film Roma where they discover a Roman villa and open the door and this vivid fresco fades in front of their eyes. It’s one thing to bring an ancient town back to life by excavation. But to keep this delicate ‘reborn’ patient alive is a massive challenge. The people who built Herculaneum didn’t build for millennia any more than we do today – they built to last 20 years or so. The staggering thing is the quality of the Roman mosaics that have survived despite what’s been done to them.

“Because of this conservation crisis, I’m almost indifferent on the subject of the papyri. I feel terribly strongly that you’ve got to concentrate on the acute problems of today, rather than what you could do in the ideal circumstances. Ancient cities that have been buried and preserved, are incredibly rare – we can’t afford to throw them away. Let’s get our heads around what’s to be done to preserve it. It’s tremendously important.”

Neither Mr Wallace-Hadrill nor Bristol University’s Robert Fowler have the final say in the matter, of course: Herculaneum’s fate is in the hands of the Superintendent, Professor Piero Guzzo. Listening keenly to the arguments is David Woodley Packard, the American billionaire philanthropist, a scion of the Hewlett-Packard dynasty, whose Packard Humanities Institute is committed to funding the site’s development. Initially on the side of those who argue for a rapid new start to excavation, he too has come round to the view that the first priority is to stop the existing site from disintegrating further. His institute has already spent $2m (£1.1m) doing that.

Mr Packard told the Art Newspaper: “When the Italians decide it is time to resume excavations at the Villa of the Papyri, our foundation expects to be in a position to offer appropriate financial support.”

We can imagine the master of the Villa dei Papiri, grasping the stanchions of a galley offshore from Herculaneum on that nightmarish day in August AD79 and gazing back in horror as the eruption column of Vesuvius slowly collapsed and a cascade of molten rock engulfed his home. He could have had no doubt that all was lost, his fragile papyrus library and everything else.

Two thousand years later, the disaster has turned out to be a miracle. But the only way to ensure that the contents of his home, miraculously spared, will not quickly crumble to nothing is to keep them in their rocky womb a little longer.