Over recent years we’ve been treated to a series of fascinating advances in the world of x-ray imaging, as researchers have developed their x-ray microtomography techniques and equipment to the point at which they can probe and then computationally reconstruct written material within objects such as letters or scrolls in museum collections whose value or fragility means they can’t be opened and read conventionally. There is more to this challenge than simply extracting the writing though, in addition to detecting the ink the researchers also have to unpick the structure of whatever it was written upon. A particular challenge comes from letterpackets, the art of folding a letter into its own envelope, and a newly-published Nature Communications paper details work from a team of academics in the USA, the UK, and the Netherlands in tackling it.
Letterpackets were more than a practical method of packaging a missive for the mail, they also had a security function often called Letterlocking. A packet would be folded in such a way as to ensure it was impossible to open without tearing or otherwise damaging the paper, and their structure is of especial interest to historians. The researchers had a unique resource with which to work; the Brienne collection is a trunk full of undeliverable mail amassed by a 17th century postmaster couple in Den Haag in the Netherlands, and now in the possession of the Beeld en Geluid museum in that city. In it were a cache of letters including 577 never-opened letterpackets, and the x-ray technique promised a means to analyse these without compromising them.
The researchers have developed an entirely computational technique for the virtual unfolding process. Starting with a 3D volumetric x-ray scan of the unopened packet they then identify the various layers of paper and the bright spots which denote the ink. Their algorithm has to cope with areas in which two or more layers are tightly in contact, for example when multiple levels are folded, and then unpick the resulting 3-dimensional mesh into a 2-dimensional sheet. Their process for mapping the crease pattern involves applying a colour map representing the mean curve radius at a given point. The final section of the paper looks at the multiple different methods of letterlocking, and attempts to categorise them all including a security rating for each. It’s evident that this could be a highly personalised process, indeed they give as an example a letter from Mary Queen of Scots that used an intricate spiral folding technique to identify its sender.
It’s clear that this technique will reveal many more fascinating historical documents as it is both refined and extended across the many more collections of further artefacts that have lain waiting for it. As they say, individual letters do not necessarily contain earth-shattering historical discoveries, but taken together they shed an important light on the social history of past centuries.
One of the names on the paper is [David Mills], whose work has featured here before.
Wonder if the NSA would be able to read our mail using this technology?
Damn cool stuff – If you can read multiple different inks through very dense fold counts with the huge variety of pressures, paper types etc. The pretty consistent paper stock, fold pattern and toner makeup of most paper mail now isn’t likely to to be hard..
Though perhaps toners and modern inks are much more transparent to x-ray, so smaller differential between the paper and ink.. But I’d be surprised if it really matters, probably just changes how long it takes to process…
Don’t worry about the NSA. It will be the post office, selling the information for targeted advertising.
This makes my usual steaming opening letters look positively primitive! Still even though these letters may not reveal any astounding historical discoveries (unless they find Amazon packages) the techniques developed will come in handy if something like the dead scrolls are ever discovered again. (Wonder how it would work on a rolled scroll with palimpsests?)
Someone to read my junk mail for me? Yes please!
Yeah, they used to have lead in inks so that would most likely show up much better on x-ray.
It was the iron in the inks – iron gall ink (also usually contains some copper contamination) that we can see in the X-Rays.
Disclosure, I did the scanning on this project.
(^Those comments were supposed to be replies, idk why it didn’t work.)
When I first looked at this, my thoughts immediately went to the post-processing they probably had to do to discern which lines belonged on which paper. When it was folded over like that, the writing must have scanned in as overlapping.
To see if the government is reading our mail, perhaps we could find a substance that undergoes a chemical change when exited by concentrated x-rays – and then put it on some letters.
Chances are big some letters are xrayed, for example when they come close to ambassies, or contains unknown content. So this would not necessarily show proof of what you’re looking for
I wish I had a photographic memory, there’s something that would be perfect for it, older tech, maybe there’s film of it being used, but for now my memory is a fog. I know, I know, why do I keep posting these undeveloped thoughts?
;-)
Really trying to wrap my head around this and watched the video too. Wouldn’t it be theoretically impossible to fold something up such that it must be damaged to un-open it? In my (admittedly poor) intuition, that doesn’t seem to be how… paper works… for lack of a better term. Even a couple of the comments in the youtube video ask the same question and there doesn’t seem to be a satisfactory answer. And even at the end of the video she just rips it open, instead of showing (somehow?) that it cannot be undone non-destructively.
Some quick google-sleuthing revealed a couple of ways of folding, “letterlocking,” that look like it is folded one way, but is really folded a different way so you do tear the paper when you go to open it if you are uninformed. But that doesn’t mean if you knew the “secret” you couldn’t just un-do it correctly and avoid damage. Right? What am I missing here?
And anyway, if it was really that important to make sure it wasn’t tampered with, wax seals and glue and stuff has been around and in use for this very purpose forever.
Naw. To me and my simple mind, all this is just a way to fold up a paper that is fun, looks cool, and is personalized.
In any case, amazing use of technology to read fragile or otherwise unreadable letters and documents.
I think that the primary issue is the age-related fragility of the paper. Were they to try to unfold it, it is likely that it would be damaged or destroyed in the attempt. It’s far better to fire some x-ray photons through it and have a computer put the puzzle together.
If there is no glue I agree that there must be at least one sequence that takes you back to the flat paper.
Maybe that sequence can be involved and the paper delicate enough that without knowing it you’d likely damage the paper either on trying to undo the folds or on folding it back to cover your tracks?
I know I damaged the paper unfolding some origami to take a look at the crease pattern and then trying to refold (and I knew the folding process in those cases).
Glue is probably the key to secure “letterlocking” as shown in the Mary Queen of Scots video, but I can imagine something glueless yet one-way like an arrowhead-type of flap inserted into a slot might prevent you from undoing it without ripping the flap or the slot.
If you do any complex origami you will find there are many fold situations where just getting it to go where you want in the first place without damage is hard, you can even see that in things like the simple crane if you fold a small one with thicker paper, that top peak of the initial preliminary fold in the centre of its back will almost certainly tear, and getting the tail and neck to fold up properly is getting towards the fold a piece of paper 7 times type issue – the fold is just too thick for the paper to take…
There are also various twisting cleat type folds where you create the entire fold pattern in advance and once you push the paper past its initial resistance while folding every crease at once it really will lock up in a way that’s going to be basically impossible undo – you can’t unfold everything at once – as you can’t actually get to all the layers to pull on them, as some key ones are entirely internal now.. It would probably be like lockpicking – in that with the exact right shape thin tool you can get to the right bits and do what you need, but in practice really damn hard.
Also worth mentioning wet folding – by dampening the right spots of the paper (or even lightly the whole sheet) you can loosen the fibres and get them to hold a new shape as they dry. Be rather hard to do that in reverse as the inner layers are just not going to get properly damp before the outer ones are a soggy mess (easily at least). I did lots of wet folded roses for a friends wedding, despite being relatively thin paper and ending up pretty large – around fist sized that wet folded compound curve shape makes them rather like rocks in the middle of the spiral, with only the outer edges of the petal really feeling like paper – and that much stiffer paper than it really is, as the structure keeps its curved… Wonder if they still exist, I know they kept them for a while at least…
Reminds me of a similar technique that scans and reads ancient papyrus rollls
“Flap and seal” work was highly evolved *very* long ago. This is why sealing wax was widely used. But a sufficiently well funded adversary would simply make a duplicate of the seal. So a scheme of folding that made it extremely difficult to not tear the paper when opening obviously had security value. I’d not heard of this scheme before, but it makes complete sense.
We have family letters written in the 1860’s on super thin paper with the writing on each side of the sheet at 90 degrees to the other side. None of us has been able to decipher the tight cursive hand enough to tell if they are from the husband to the wife or from the wife to the husband. We are fairly certain of the two parties, but that is all.
As for modern mail. All mail in the US is imaged on both sides. Some fraction is x-rayed, but that is generally devoted to bombs, poisons and such. Of course, if “they” are interested in “you” all bets are off.
Ancient inks were commonly based on lamp black or oak gall. I’ve never heard of lead being used. And once long ago I was very interested in secret communications. In any case, an MRI would do a great job whatever the composition.
The basic mathematics for doing this stuff was developed by the oil industry starting in the 1950’s for processing seismic data. Having spent my career in that field it is really fun to see all the things people have come up with that reuses that work. The application changes, but the math is the same. The reason big oil developed it was they were the only ones who could afford it and also live with the limitations of the technology of the time. When they started the Nyquist limit for a 16 channel ADC was 125 Hz and cost a fortune. But it made them money so they funded it, bought the biggest computers available and over time it all became cheap.
In the early 80’s a million dollar state of the art system had 4 MB of RAM, 1.3 GB of disk and could do about 125,000 FLOPS. Go price an off lease HP Z820 workstation on ebay. Twenty+ cores and 512 GB of RAM for under $2000. Add $1000 for a triple parity 20 TB RAID array. Wow!
“The basic mathematics for doing this stuff was developed by the oil industry starting in the 1950’s for processing seismic data.”
In an alternate and politically correct timeline greenies would have developed this tech.
How about around $250,000 for an array of tape drives that could store about 2 gigabytes? Each tape drive the size of a washing machine and the controller as big as two large upright freezers. The tapes were pretty small, single spool, similar to LTO. Saw one of those IBM monstrosities at an auction once. Got no bids.
Forgot to mention that at the time, 2 gigabyte USB drives had just become available. The same amount of data that formerly took a couple thousand (or more) pounds of equipment to store, in a package that might weigh a couple of ounces.
“None of us has been able to decipher the tight cursive hand enough” that sounds like either of my parent’s handwriting! Totally incomprehensible.
AFAIK lead was only used in industrial printing ink.
Can this technology be used for something that is actually useful? I have no idea why a museum is trying to preserve hundreds of unopened letters, lmao. Open and read about 400 of them. You will get the gist of what most of what the correspondence is. Save the remainder for examples of sealed letters and for future researchers to play around with.
I mean when I was 12 years old I would hold envelopes up to a lamp to read the letters inside. Do we really need xrays for this? I think it’s a cool project, but I guess the whole preserving some old unopened mail seems so weird. It seems the real value is in the content. Just open them, scan them, and upload to the internet for everyone to learn from. Then take some high res photos and video of how these folding letters work. Save a few for the museum. Done!
Think 3000 year old papyrus scroll, and there is more value to unlock.