A common complaint about the rise of commercial AI services is that they are power-hungry and thus damage the environment. If this concerns you then [Squeezlabs] has the solution, in the form of an AI powered by a handcrank.
The guts of the system is a Raspberry Pi 5 running llama.cpp and appropriate speech conversions, but it and the Large Language Model (LLM) side are not the most interesting part of this system. The power comes from a hand crank charger of the type you’ll see for sale on the likes of AliExpress, designed for USB charging. That in itself is not enough to power the Pi though, as upticks in the processing can cause brownouts that crash the machine. Thus there’s a custom-made capacitor board to take up the strain, and even with that the handle resistance varies significantly depending on the computing load.
We can see that this is not the ideal way to experience an LLM, but maybe that’s not the point. It does however point towards a future in which the power demands of processing decrease and less effort is required. Meanwhile, this is by no means the first hand cranked project we’ve seen.
Neat project/installation :)
I guess it’s not also trained using the crank as as energy input?
Of course! Otherwise the adjective “truly” wouldn’t be warranted!
I wouldn’t make bets on computing better becoming maybe more than another 10x or so better. The latest headline figures from TSMC are 1.6x improvement per generation, not 2x, and that number is trending down. And you get 1.6 perf at the same wattage, or 1.6 wattage reduction at the same perf, not both as in the heady days of pre-2005 dennard scaling. A future is visible where doubling happens every 20 years instead of every 2.
We’ve run out of road. We shrank transistors until we couldn’t shrink them anymore. Then we started optimizing their geometry, moving from planar transistors at 28nm to finfets, to forksheets, to stacked single-atom-thick gate-all-around nanosheets in the next generation or two. Then as that ran out we tackled more obscure inefficiencies, implementing tricks like silicon interposer advanced packaging, more radical litho pattern geometry optimizations, backside power delivery, and soon CFETs where we get more areal density by stacking the complementary N-channel and P-channel transistors on top of each other. That last one is the current end of the roadmap, and you’ll notice it gets you more density but doesn’t do as much for power dissipation. The ultimate step is projected to be manufacturing the N and P transistors on different wafers and bonding them together, just to exploit slightly different crystal alignment for each type.
Huawei has been a surprise entrant with layered logic layout and chip stacking, but while this reduces buffers, cuts some power dissipation, and raises clocks, this is another side trick. It wont be long before it’s replicated by other fabs (limiting factor: large-scale software rewrites and precision die placement+bonding), and it isn’t a viable roadmap after we master it in the first couple of generations.
computing becoming*
Oh, and worth mentioning are the litho machines. We’ve taken on EUV litho, then high-numerical-aperture EUV, and ultra-high NA EUV is in the works. But ASML themselves have admitted that pretty soon it’s not going to make sense to keep chasing this further, simply on an economics basis. you’ll never turn a profit with how much the machines will cost and how many wafers you’ll be able to process per minute.