Although computers are overwhelmingly digital today, there’s a good point to be made that analog computers are the more efficient approach for specific applications. The authors behind a recent paper in Nature are arguing that inference – essential for LLMs – can be done significantly more efficiently using an analog optical computer (AOC).
As the authors describe it, the function of this AOC is to perform a fixed-point search using only optical and analog electronic components. The optics handle the matrix-vector multiplications, while the analog components handle the non-linear operations, subtractions and annealing. This is performed in 20 ns cycles until noise has been reduced to an acceptable level, considering the analog nature of the computer. A big advantage here is that no analog-digital conversions are required as with other (digital) hybrid systems.
So far a small-scale AOC has been constructed for tasks like image classification and non-linear regression tasks, with the authors claiming the AOC being over a hundred times more efficient than current GPU-derived vector processors.
If I understand correctly, it’s basically a row of LEDs, the light projected into lines onto grayscale TFT, and gathered again into points on photodetector in the perpendicular direction. Fairly simple tech actually.
But I have to wonder, wouldn’t it be a lot more efficient to replace it all with transistors, and stay in the electronic domain? The TFT acts as a not-very-precise multiplier, which can be done by controlling FET gate voltage, and the optics just take an average, which can be done with resistors. If you get fancy with CCDs, the computation can be kept in analog domain through multiple matrix operations.
Optics could theoretically have some speed advantage, but you still need electronics to drive the LEDs and to gather the output. The real bottleneck would probably be update speed of the TFT (assuming you don’t dedicate one computational unit to one set of matrix values).
It’s not that pure analog electronics aren’t possible, they are. But optics helps push the scaling further, reduces some of the electrical overhead (wiring, parasitics, datapath), and enables operations that would be less efficient (or nearly impossible) in purely electronic analog hardware at large size.
Optics + analog electronics give a better “sweet spot” for doing large, dense, linear operations (matrix multiplication, etc) very efficiently in parallel.
Now wondering if it would make sense to make a hybrid of the two. Phototransistors used as photodetectors.
I guess the big question, in my mind at least, is an analog fiber optic system built to run an LLM have any benefits over GPU RAM? Costs aside, how do these 2 systems compare? Is anyone working on analog systems to run AI?
(Not sure where my earlier comment was lost)
I think this could be done faster with analog electronics. There is no reason to believe that driving LEDs and sampling protodiodes would be faster than just doing the approximate multiplication with FETs. Controlling FET resistance through gate voltage is not linear, but neither is TFTs, so both would need some compensation curve.
It is almost trivial to perform real-time 2D Fourier transforms with a lens. And if you can do an FT, you can do all the related transforms plus matrix multiply, which is notoriously time consuming in digital systems.
RE: multiplication with FETs.
I believe that’s what FPAAs where invented for (FPAA = Field Programmable Analog Arrays).
It’s neat and all, but there’s a huge and oft-overlooked reason digital computers won out over analog computers in the 50’s, even the analog computers that were electronic.
With analog computers it’s very hard to make more than a few of something and actually have them work. You’re working with raw values represented by physical quantities, so the physical properties of everything that influences them has to be within a tight spec. Digital circuits? they couldn’t give a toss if your beta value is 50 or 300. That’s a 600% swing instead of say 10%. With that kind of tolerance it’s very easy to make millions of them at a time and have them all work, and for them to STAY working for very long service lives, with no adjustment even as components age. Components hand-trimmed at the factory are already expensive. Just imagine if you needed to hand-tune every logic gate in your computer after 3-5 years too.
Biology works well within those limitations.
Biology is useless for building computers.
Well, one could argue that artificial neural networks are better suited to run on biology, than on computers…
Very cool, can’t wait for 1$ 700B parameter LLM ASIC connected to a speaker telling me sweet lies all day 24/7 and pretending to be my loving wife
Cheapest relationship one ever had.
If I recall correctly, in “Star Trek” (the new generation, obviously) they had this exact thing implemented as etched crystals. (of course, it was fake movie prop, really, a credit-card sized transparent plastic thingie with colored patches, but you get the idea).
Gives me an idea now, off to the drawing board …
Early synthetic aperture RADAR was computed optically in real-time with the RADAR signal scanned to film or a storage display with one scan-line per RADAR pulse as raw data. Satellite versions “dropped” film canisters for re-entry and capture from parachutes(?). The data film was moved through an optical system with output being the processed image. The same optics can do synthetic aperture SONAR. The film step can be skipped for continuously output SAR imaging. https://www.researchgate.net/figure/Components-of-an-optical-SAR-processing-system-a-collimated-beam-illuminates-the-film_fig1_246836455
Absolutely fascinating history, thanks for the link!
Clever lads back then! Digitally this is a very math and array intensive process of deconvolutions and FFT’s on big data. Optically it is practically done before you turn it on. Any thin lens produces a 2D Fourier pattern at the focal point. This is pretty good. Note the collimation lens and the parts to the left also do a FT. The microscope objective plus pinhole is a “spatial filter” and only the center of any diffraction pattern gets through to the collimation lens to get “clean” light. https://www.cis.rit.edu/class/simg455/lab7-20073-fourier-optics.pdf