With all the attention on LLMs (Large Language Models) and image generators lately, it’s nice to see some of the more niche and unusual applications of machine learning. GARF (Generalizeable 3D reAssembly for Real-world Fractures) is one such project.
GARF may play fast and loose with acronym formation, but it certainly knows how to be picky when it counts. Its whole job is to look at the pieces of a broken object and accurately figure out how to fit the pieces back together, even if there are some missing bits or the edges aren’t clean.
Efficiently and accurately figuring out how to re-assemble different pieces into a whole is not a trivial task. One may think it can in theory be brute-forced, but the complexity of such a job rapidly becomes immense. That’s where machine learning methods come in, as researchers created a system that can do exactly that. It addresses the challenge of generalizing from a synthetic data set (in which computer-generated objects are broken and analyzed for training) and successfully applying it to the kinds of highly complex breakage patterns that are seen in real-world objects like bones, recovered archaeological artifacts, and more.
The system is essentially a highly adept 3D puzzle solver, but an entirely different beast from something like this jigsaw puzzle solving pick-and-place robot. Instead of working on flat pieces with clean, predictable edges it handles 3D scanned fragments with complex break patterns even if the edges are imperfect, or there are missing pieces.
GARF is exactly the kind of software framework that is worth keeping in the back of one’s mind just in case it comes in handy some day. The GitHub repository contains the code (although at this moment the custom dataset is not yet uploaded) but there is also a demo available for the curious.
Pft! Batman did it back in 2012.
Counter-pfft, Snow Crash did it in 1992
i wonder what would happen if included a piece that wasn’t part of the original object
Probably nothing but increased compute time as that piece would probably not be close enough to fit with any of the other parts, but if it was a good match, maybe even a better match than the original part on say 2 sides of its triangular shape (which could happen if more small bits break off that are not recovered/modelled for the real part) it would probably grind everything to a halt and make the problem functionally a never solve – the real part(s) just doesn’t get up to the same confidence level of fit so every possible iteration that same mistake is made again and again.
Maybe that’s the next step. Give it assorted mixed-up potsherds from a dig and ask it “how many pieces of ceramicware was this, and which pieces go together with which others?” Only about a thousand times more difficult!
Brundel-cup
Seems they missed a golden opportunity to use the acronym “FRAG” instead.
I feel like they purposely made the acronym as the opposite of FRAG, as it’s reversing the damage from a fragmentation.
Defrag?
Oh, it has to be H.U.M.P.T.Y or D.U.M.P.T.Y
A good name would be Jabim—the god of lost and broken things…from Dunsany.
If only there were other unbreakers, like at the end of this SF episode
https://buckrogers.fandom.com/wiki/Shgoratchx!
I genuinly wonder if this is still not some form of bruteforcing with the slight variation of using probability (which will yield higher chamces of being right-er). E.g bruteforce is only true/false on a match, but with AI we get 0-100% match, and the best matches thus are assumed to be correct.
a great application might be to try to stack together the “debri” of old greek temples.
More interesting perhaps is to apply this to my box of lego bricks… make a picture of the bricks and the computer calculates all possible/sensible builds that can be made of it.
That would be great for coarse aligning point clouds from different scans!
Maybe this would improve 2D packing algorithms…
GARF assisted Kintsugi. Sweet.
Now for one that could reassemble my old AMT Enterprise a cousin dashed all to pieces.