Outstanding work! I've participated in the challenge, but didn't get far. One of the questions I had at the time was - if I'm going to use ML to detect ink, could it invent hallucinated letters, or even parts of text, and how to prevent that?
Yes, it's quite possible for ML to hallucinate ink, though it is on a much more local scale, like predicting a slightly longer stroke, filling in more of a character than is actually in the data, etc. Perhaps enough to change a reading of a character or show where ink isnt. It is difficult for ink detection to hallucinate grammatical and idiomatic greek and latin.
What is the input to the ML algorithm? Does it know the surrounding context so that it has a chance to deduce "if this stroke is slightly longer then the end result will be idiomatic greek and latin"?
Just as with redacted documents (consistently blocked terms) or bad OCR jobs (wrong or missing characters), even if only a certain percentage comes out unmangled it is more readable than having no data at all.
A stable base corpus and some dynamic programming will allow you to clean up the remainder[0].
We've been trying to automate since the beginning. A lot of it is automated but it's mostly the easier and less damaged parts of the scrolls. Scanning takes a few days for the biggest scrolls but the amount of human refinement is still a multi month process.
I'm interested to know about the approaches that you tried with the ML, and then decided to not use. In practice, the options are so many. How did you come up with the final approach - and was there a systematic way to decide which options to go for?
I am not on the research team, rather on the production side of things, so my knowledge on that is pretty limited. I think one of the main takeaways from a lot of the research, though, on both the segmentation side and the ink detection side, is that it's a lot less about what models and techniques and such you use, but how good your training data is. Gathering ground truth is hard, and if you don't have a lot of good ground truth, it doesn't matter if your code is perfect, you'll never get results.
That is a general truth of most ML; many models _can_ find the information in the data, if the data is good enough. If it is not, then likely no model can.
I am not a papyrologist or a classicist, rather I'm a computer scientist, so my expertise is unfortunately not in _what_ the scrolls say, rather how we get there. That being said I think and hope that there will be a trove of things that has no known provenance at all, completely lost works that elude the public memory.
There's also the Telegony. Odysseus has a son through Circe who winds up killing him and marrying Penelope. Odysseus son through Penelope, Telemachus, marries Circe. There's some wild stuff that doesn't survive.
That's what was thought, but maybe not -- only one of the three so far looks Epicurean, which is not what was expected. Maybe it's a fluke, but historians are buzzing a bit about whether it might be broader than expected.
The Epicureans were particularly hostile to the Jews and Christians, because Epicureans deny Providence or the active intervention of the divine in human affairs. See Horace Sermones 1.5.
in the paper it says "The recovered text is a philosophical treatise on ethics, and the evidence points to a Stoic work: it turns on human nature, impulse, and the moral progress of human beings, and its final preserved column names Aristocreon — nephew and disciple of the great Stoic Chrysippus — which, together with the language and themes of the text, places it in a Stoic context and dates it to the 2nd century BC."
30 scrolls, maybe? Something like that. I scanned Pherc Paris 4 and Pherc Paris 3 at Beam line 18 at ESRF back in March.
The team did "the campfire scroll" experiment a few years ago to replicate carbonization, unrolling, and ink detection. That is the only case I am aware of. It proved the method could work but it's not a source of say training data; it varies too much from the real scrolls.
The main limitation is time and cost. We have to scan on what is AFAIK the most powerful x-ray beam line in the world. It is not cheap
You had to pay? I understand the machine cost many hundreds of millions of dollars, but I would have thought for academic researchers doing open science, the beamtime is free (funded by the govt / science trusts).
The beam time is unfortunately not free. I scanned Pherc Paris 4 and Pherc Paris 3 in March and had the final shift on the beam. As I was removing the scroll from the scanning pedestal the next team of scientists were already in the lab getting their samples ready. It's a well oiled machine and they've got customers.
The way these things normally work is that the project starts with some sort of a grant. Then that grant pays for all of the costs of the project: peoples' salary, materials used, time on equipment, plus money for the buildings and administration (overhead).
In this case the time on the equipment would need to be included, both a portion of the cost of building/maintaining it, and probably the energy needed to run it. Even where the government is providing the grant (likely here), it still needs to be accounted for.
How do get to do that? As in what did you study to get the prerequisite knowledge, and how did you find this particular job? When I see interesting jobs I'm anyways curious what path lead there
I am a computer scientist. I studied CS in university, worked in the semiconductor industry for a while, got started as a participant in the challenge aspect of the Vesuivus Challenge. They were hiring, I sent in an application, interviewed, and was offered the job.
That's a tough one to give a strong estimate of. Some scrolls are easier or harder to unwrap and read for a multitude of different reasons, mostly due to how damaged the scroll was in the eruption, and how easy or not the ink is to read. IIRC from what we've scanned of the herculaneum collection, none of the ink is easily visible via spectrum alone, so we have to use a lot of ML and physically based rendering techniques to be able to find ink. That also requires unwrapping and segmentation _before_ any ink detection.
For iron gall ink with high enough iron concentration, the ink stands out in the xray volume through simply masking off low values, such as was shown in our campfire scroll experiment a few years ago. No herculaneum scrolls show similar ink.
Most of the evidence so far points towards carbon based ink. I am not sure if any of the scrolls we have scanned show strong evidence of iron gall based ink. I know that there are different types and preparation methods for different carbon based inks, but I do not know if it is possible to determine which kind(s) were used solely from inspecting the xrays.
I am, though, not a papyrologist, so historical ink making, preparation, and usage are not my field.
Did anyone on the team come from a non-science, non-math, non-academia background? Did anyone working on this just teach themselves and start contributing?
Yes. Sean, who was a co-winner of the 2024 prize, IIRC has no formal background in ML, computer science, AI, etc. He is one of our core researchers and the most productive team member.
I've been on the Discord for a couple of years now, and poking around with submissions as well. Sean and the entire team deserve so much praise for all of this work.
It's easy to just read about the breakthrough and see it as one neat, linear line to get there, and hard to comprehend the hours, months and years that so many spent to get there. Big congrats to you, Sean, Nat and the entire team!
I am unaware of those fragments in particular. Though we have scanned a dozen or so fragments, mostly to help guide ink detection, since the ink in them is often more visible in visible and/or near IR light, but can be hard to impossible to detect in the xray spectrum.
You have a potential to rewrite the history of European Antiquity quite substantially. The Herculaneum set of scrolls is enormous and must contain a lot of hitherto unknown.
That comes with a set of peculiar risks. Once your work starts producing something that contradicts previous work of Very Important People, they will lobby to stop you. Be prepared for that.
Science should be neutral and always value new evidence. Scientists as humans are unfortunately subject to all sorts of passions.
Only about 20% of the Herculaneum site has been excavated, so there is high probability that more scrolls exist. The current scrolls were not part of the main library, but more of a private collection at the time.
So imagine how cool it would be to find a full library with thousand of scrolls across many different topics, that can now be read with this technology.
This could eventually completely transform our understanding of Antiquity. It is estimated that only around 1% of the ancient works in Greek and Latin have survived to the present day, much less in other languages such as Punic [0]. Some works and some authors we only know by name because they were alluded to in later texts.
It's also well known that surviving texts survived because they were copied again and again on costly animal skin during the Middle Ages, by monks who had to make a choice and naturally favored topics that were of most interest to them.
I'm not sure what you expect to find that would completely transform our understanding of the time period. The most likely discoveries are going to be filling in details about things we already knew. This period of time was already pretty well documented. Even if we found something amazing like some of Aristotle's lost works, we basically already know what they were and (roughly) what was in them. Really the most interesting and useful finds would be more mundane things like household records, and personal diaries.
Could change our understanding of history - slavery, early Christianity, politics, secularism among roman elite, etc
Or of technology- steam power, mechanical computation (like the Antikythera mechanism, which is the only known example of such a thing until 1300 years later), mechanized production, mining techniques, etc
Disagree. My understanding is that most surviving works have been transcribed repeatedly over the centuries, often times based on preferences of the people living at the time. There’s a big chance that excavation could find deeply heterodox stuff, I think.
I think it's less that the stuff would be considered heterodox, as just not as good/relevant. Like certain texts were used in the Roman world for school, just kind of universally taught to the literate class. The Aeneid was one of these but before it was written the Annales by Ennius was the classic poem everyone had to learn. Then the Annales became less popular, stopped being taught, and now we only have some fragments of it.
> There’s a big chance that excavation could find deeply heterodox stuff, I think.
Heterodoxy (or really, orthodoxy) wasn't really a thing in 79ad, and you're not likely to find much of it in the private library of a wealthy Roman's vacation home. The only forbidden work you're going to see from that era is stuff critical of the emperor.
We've seen this happen already once with the recovery of palimpsests. Outside of a few lucky discoveries, the vast majority of what monks were discarding were things that were not seen as useful - outdated (to them) legal texts, liturgical books, etc.
The exception though would be Greek literature. Greek literacy collapsed in the early medieval era and a large catalogue was probably just scrapped or discarded before even being collected in Monasteries. Herculaneum could represent a legitimate treasure trove in that regard.
We have successfully uncorrupted audiovisual media of what we believe to be an oral retelling of the long lost ending to Chekhov‘s “The Three Sisters.” It turns out the light was on.
As I understand it, Pompeii was basically a city of vice and hendoism. Most of the scroll text so far seems to be the ancient version of porn fanfic. So things really never change.
Every time you feel depressed by the state of tech, and how so many intelligent people seem to work on forcing ever more ads down people's throats (a common trope around these parts), remember that projects like this do exist too!
There are lots of very smart folks working on incredible things, they just aren't as loud.
This isnt the only incredible thing though, AI is being used to make discoveries in the medial field, and even to prevent sepsis related deaths, cutting down on them by detecting sepsis sooner. There was another that discovered the gene for Alzheimers is what activates it not just a sign of it.
There is a large overlap in what we are doing with the medical field as well. A lot of the segmentation methodology and technology we use and adapt originally came out of the medical field for doing things like brain imaging.
When I read translations like these, I always wonder if the tone is translated. Did the writer mean to convey a very formal “to the utmost”, or was it a more casual “to the max”.
How much of the translators bias makes these seem like academic papers instead of social media posts.
It's philosophy, it's probably very dense prose. Formal Greek/Latin writing tended to have very long sentences with a bunch of subordinated clauses. People don't really write like that outside of academia or "highbrow" literature right now.
Casual letters and graffiti would be closer to tweets.
Any useful translation of an ancient text is accompanied by the text in the original language, so that the reader may assess how faithful is the translation.
For anyone who wants to read ancient texts, there are bilingual editions, for example those of the "Loeb library".
The translations that omit the original text are just for the people who want to have some idea about the content, but do not care about the correctness of the translation.
With a bilingual edition, it is easy to understand the original text even with relatively little knowledge about the original language.
The original text is important because frequently the translator is forced to introduce inaccuracies in the translation, because of the absence of exact equivalents in the target language, which would require a long explanation of the original meaning, instead of just a translated sentence.
Especially misleading are translations where several distinct ancient words are translated using the same English word, so some nuances are lost.
Equally confusing are the cases when the translator chooses to translate the same ancient word by different English words, because even if the meaning of a word may depend on the context, many translators fail to judge correctly the context, because they may lack specialized knowledge so their guesses are not necessarily better than of the readers who may be less competent in linguistics, but more competent in the science or technology needed to understand the context. Better translators prefer to use a one-to-one mapping between words, which makes it easier for the readers to discover the meaning intended by the ancient writer, after seeing multiple examples of usage.
This is why I like literal translations & etymological dives, paired with asking what activities would constitute a life in that time. Ie, you may not need to be a competent archer, but it is a little easier to understand someone who used a particular style of bow if you can play around with that type of bow for a bit.
I can actually read Ancient Greek. LLMS are really bad at it.
> * ἐκ- = “out,” “thoroughly,” “to the end”
ἐκ is more motion away from something. It's often an intensifier in verb compounds but not really as a standalone preposition.
Ancient Greek is a very different language from English. I've found people who try to brute force it by looking up individual words without a knowledge of the grammar end up with a worse understanding of a text that someone who just reads in translation.
Sending a tweet is free and takes zero thought to make it (as the vast majority of tweets prove). Writing something on a scroll would take a lot of effort and would not be free. If these were tweet level content in the scrolls, I'd have to totally reevaluate a lot of things to the point I might as well just become MAGA
The person who wrote this was was closer in time to the technology that was able to unwind and read burned fragments of their text, than the technology that build the pyramids. pretty wild to think about.
The stones were cut with enormous precision, at least relative to what we know about the available cutting tools. You cannot still stick a knife between a lot of these stones. Maybe we will learn more about that.
(I know nothing about this subject, feel free to ignore me.)
My dentist is pretty good at doing this too, by putting marking paper between my teeth and having me bite down. I wonder if a similar technique could be used:
Have the blocks close together, constrained to only move on a single axis by rails or whatever. Drape a thin sheet of material over one of the blocks, the non-moving one (perhaps it's an already-placed one?) Maybe it's something that visibly shows when it's crushed, or maybe it's coated with the blood of the powerless. Smash the other block into it. Pull them apart and look where they made contact. If it's mostly everywhere, done. If not, grind down or chip out the parts that touched. Repeat until you run out of innocents.
To do the very last block, you'd have to meld two sides, remove a block, fix up the other side, and then put it back in. Which might make this testable.
But I'm just pulling stuff out of my nether orifice.
I'm pretty sure we've conclusively answered these questions. Hand tools, skill, and absolutely unreasonable amounts of time and patience.
Any master stoneworker from any era should be able to carve stone to that level of precision given enough time and reason. The problem, as always, is that there is usually very little reason to put in that amount of time and effort when you can get 90% as good for 50% the effort.
I imagine it's not the first time, It must've at least been proofread at the time of writing :)
But really impressive stuff! Between this and (a particularly optimistic outlook on) the Linear-A news from the other week this is an exciting time for linguistics.
I found once super old books in our lab (like hundreds of years) and was wondering what they were used for.
Apparently they did CT scans of closed books and read the content.
Polevoy, Dmitry V., et al. "From tomographic reconstruction to automatic text recognition: the next frontier task for the artificial intelligence." Fifteenth International Conference on Machine Vision (ICMV 2022). Vol. 12701. SPIE, 2023. https://iris.unive.it/bitstream/10278/3687069/1/Albertin_et-...
So yeah, but lottery companies probably make it harder by engineering against it.
The tomography was done at a synchrotron (ESRF), and with beamtime being very expensive it would be a net-negative to scan lottery tickets, unfortunately...
Fortunately for anyone wanting to xray lottery tickets, you don't need the IIRC most powerful beamline in the world. A few years ago a Vesuvius Challenge Community member bought a benchtop xray machine for a few grand and scanned pokemon cards and was able to identify them that way.
So far this is some of the best uses of ML I've seen to date! This is one of the few things you can point at and say "AI made the world a better place" IMO (this and medical research).
But wait, the work seems to be from the 2nd century, but it was buried during the Vesuvius eruption in the 1st century?
I love stuff like this because it gives a glimpse into Roman society. To me it seems like they were very similar to us today, forever contemplating learning, existence, gods.
Scrolls from Herculaneum have been read for a very long time. Not disputing the achievement of digitally unrolling one, but the scrolls from the library of have been studied since the 18th century.
I think it's a case of HN once again butchering the title. I submitted it as the exact title from our page on scrollprize.org, "An _Entire_ Herculaneum Scroll Has Been Read For The First Time", which is IIRC true.
They are in a variety of conditions - some of them people were able to "break" open and read. But the vast majority of what remains is too delicate and brittle to risk.
Sure, but its the potential scale that is important. There are also more scrolls still in the ground, which would make sense to dig out if they could be read.
I'm hoping for a complete(ish) Heraclitus. Also Eratosthenes, whose methods have been described but we don't have the original work where we calculated the circumference of the Earth. Also Hipparchus and Thales.
A stable base corpus and some dynamic programming will allow you to clean up the remainder[0].
[0]: http://stackoverflow.com/a/11642687/2449774
Could it be automated to the point where it's faster to scan a book closed than opened?
Ah, the good old bitter lesson strikes again
https://en.wikipedia.org/wiki/Nigel_Richards
Congratulations, and thank-you!
https://en.wikipedia.org/wiki/List_of_lost_literary_works
I was under the impression that there was almost nothing left of that school of thought, and that it’s writings had been destroyed.
What would you like to have instead?
History! That's what intrigues me the most: texts with accounts of events that have otherwise vanished from the historical record.
have any attempts (or just ideas) been made to recreate such charring on known texts?
The team did "the campfire scroll" experiment a few years ago to replicate carbonization, unrolling, and ink detection. That is the only case I am aware of. It proved the method could work but it's not a source of say training data; it varies too much from the real scrolls.
The main limitation is time and cost. We have to scan on what is AFAIK the most powerful x-ray beam line in the world. It is not cheap
In this case the time on the equipment would need to be included, both a portion of the cost of building/maintaining it, and probably the energy needed to run it. Even where the government is providing the grant (likely here), it still needs to be accounted for.
For iron gall ink with high enough iron concentration, the ink stands out in the xray volume through simply masking off low values, such as was shown in our campfire scroll experiment a few years ago. No herculaneum scrolls show similar ink.
I am, though, not a papyrologist, so historical ink making, preparation, and usage are not my field.
It's easy to just read about the breakthrough and see it as one neat, linear line to get there, and hard to comprehend the hours, months and years that so many spent to get there. Big congrats to you, Sean, Nat and the entire team!
Major kudos to all of you on your achievements! This is amazing work for anthropology and for society, and it's greatly appreciated.
You have a potential to rewrite the history of European Antiquity quite substantially. The Herculaneum set of scrolls is enormous and must contain a lot of hitherto unknown.
That comes with a set of peculiar risks. Once your work starts producing something that contradicts previous work of Very Important People, they will lobby to stop you. Be prepared for that.
Science should be neutral and always value new evidence. Scientists as humans are unfortunately subject to all sorts of passions.
So imagine how cool it would be to find a full library with thousand of scrolls across many different topics, that can now be read with this technology.
It's also well known that surviving texts survived because they were copied again and again on costly animal skin during the Middle Ages, by monks who had to make a choice and naturally favored topics that were of most interest to them.
This could quite literally change everything.
[0] https://talesoftimesforgotten.com/2021/09/25/are-there-more-...
Or of technology- steam power, mechanical computation (like the Antikythera mechanism, which is the only known example of such a thing until 1300 years later), mechanized production, mining techniques, etc
Heterodoxy (or really, orthodoxy) wasn't really a thing in 79ad, and you're not likely to find much of it in the private library of a wealthy Roman's vacation home. The only forbidden work you're going to see from that era is stuff critical of the emperor.
The exception though would be Greek literature. Greek literacy collapsed in the early medieval era and a large catalogue was probably just scrapped or discarded before even being collected in Monasteries. Herculaneum could represent a legitimate treasure trove in that regard.
So we can just get ChatGPT to fill in the blanks.
A Post-Great Solar Flare of 2484 Step Brothers DVD Has Been Decoded
There are lots of very smart folks working on incredible things, they just aren't as loud.
I'm kind of obsessed with the ancient world. I dream of being able to read entire pages of new text from ~2,000 years ago.
How much of the translators bias makes these seem like academic papers instead of social media posts.
Casual letters and graffiti would be closer to tweets.
For anyone who wants to read ancient texts, there are bilingual editions, for example those of the "Loeb library".
The translations that omit the original text are just for the people who want to have some idea about the content, but do not care about the correctness of the translation.
With a bilingual edition, it is easy to understand the original text even with relatively little knowledge about the original language.
The original text is important because frequently the translator is forced to introduce inaccuracies in the translation, because of the absence of exact equivalents in the target language, which would require a long explanation of the original meaning, instead of just a translated sentence.
Especially misleading are translations where several distinct ancient words are translated using the same English word, so some nuances are lost.
Equally confusing are the cases when the translator chooses to translate the same ancient word by different English words, because even if the meaning of a word may depend on the context, many translators fail to judge correctly the context, because they may lack specialized knowledge so their guesses are not necessarily better than of the readers who may be less competent in linguistics, but more competent in the science or technology needed to understand the context. Better translators prefer to use a one-to-one mapping between words, which makes it easier for the readers to discover the meaning intended by the ancient writer, after seeing multiple examples of usage.
While I step through the valley of the shadow of death,
I contemplate my life and perceive that nothing remains.
For I have hurled weapons and laughed for so long that
Even to my mother, my mind appears to have departed.
Yet I have deceived no one except him who was worthy of it;
For me to be held as a coward—that indeed is unheard of.
Beware what you speak and where you set out,
Lest you and your companions be outlined in chalk.
Latin is also a very rich language and this is no snippet.
Translation is always hard, especially from a couple thousand years ago BUT this kind of translation comes with a lot of confidence.
* ἐκ- = “out,” “thoroughly,” “to the end”
* πονέω = “to labor,” “to toil,” “to work hard”
> * ἐκ- = “out,” “thoroughly,” “to the end”
ἐκ is more motion away from something. It's often an intensifier in verb compounds but not really as a standalone preposition.
Ancient Greek is a very different language from English. I've found people who try to brute force it by looking up individual words without a knowledge of the grammar end up with a worse understanding of a text that someone who just reads in translation.
You mean ropes and carts?
My dentist is pretty good at doing this too, by putting marking paper between my teeth and having me bite down. I wonder if a similar technique could be used:
Have the blocks close together, constrained to only move on a single axis by rails or whatever. Drape a thin sheet of material over one of the blocks, the non-moving one (perhaps it's an already-placed one?) Maybe it's something that visibly shows when it's crushed, or maybe it's coated with the blood of the powerless. Smash the other block into it. Pull them apart and look where they made contact. If it's mostly everywhere, done. If not, grind down or chip out the parts that touched. Repeat until you run out of innocents.
To do the very last block, you'd have to meld two sides, remove a block, fix up the other side, and then put it back in. Which might make this testable.
But I'm just pulling stuff out of my nether orifice.
Any master stoneworker from any era should be able to carve stone to that level of precision given enough time and reason. The problem, as always, is that there is usually very little reason to put in that amount of time and effort when you can get 90% as good for 50% the effort.
But really impressive stuff! Between this and (a particularly optimistic outlook on) the Linear-A news from the other week this is an exciting time for linguistics.
Beautifully ironic, that we find this message.
A thought: I guess the days of scratch off lottery tickets are numbered?
Apparently they did CT scans of closed books and read the content. Polevoy, Dmitry V., et al. "From tomographic reconstruction to automatic text recognition: the next frontier task for the artificial intelligence." Fifteenth International Conference on Machine Vision (ICMV 2022). Vol. 12701. SPIE, 2023. https://iris.unive.it/bitstream/10278/3687069/1/Albertin_et-...
So yeah, but lottery companies probably make it harder by engineering against it.
I love stuff like this because it gives a glimpse into Roman society. To me it seems like they were very similar to us today, forever contemplating learning, existence, gods.
Emphasis mine.
Amazing!
(Btw, you can use the 'edit' link to fix things like this if the software gets a title change wrong.)
They are in a variety of conditions - some of them people were able to "break" open and read. But the vast majority of what remains is too delicate and brittle to risk.
Fantastic work!