Why LLaMa Is A Big Deal

March 22, 2023 by Matthew Carlson 27 Comments

You might have heard about LLaMa or maybe you haven’t. Either way, what’s the big deal? It’s just some AI thing. In a nutshell, LLaMa is important because it allows you to run large language models (LLM) like GPT-3 on commodity hardware. In many ways, this is a bit like Stable Diffusion, which similarly allowed normal folks to run image generation models on their own hardware with access to the underlying source code. We’ve discussed why Stable Diffusion matters and even talked about how it works.

LLaMa is a transformer language model from Facebook/Meta research, which is a collection of large models from 7 billion to 65 billion parameters trained on publicly available datasets. Their research paper showed that the 13B version outperformed GPT-3 in most benchmarks and LLama-65B is right up there with the best of them. LLaMa was unique as inference could be run on a single GPU due to some optimizations made to the transformer itself and the model being about 10x smaller. While Meta recommended that users have at least 10 GB of VRAM to run inference on the larger models, that’s a huge step from the 80 GB A100 cards that often run these models.

While this was an important step forward for the research community, it became a huge one for the hacker community when [Georgi Gerganov] rolled in. He released llama.cpp on GitHub, which runs the inference of a LLaMa model with 4-bit quantization. His code was focused on running LLaMa-7B on your Macbook, but we’ve seen versions running on smartphones and Raspberry Pis. There’s even a version written in Rust! A rough rule of thumb is anything with more than 4 GB of RAM can run LLaMa. Model weights are available through Meta with some rather strict terms, but they’ve been leaked online and can be found even in a pull request on the GitHub repo itself. Continue reading “Why LLaMa Is A Big Deal” →

A Studio Condenser Microphone For A Constrained Budget

March 22, 2023 by Jenny List 22 Comments

As the Internet has turned so many of us into content creators, we’ve seen the quality of webcams and microphones steadily increase to the point at which even a fairly modestly-equipped YouTuber now captures their wisdom at a quality far exceeding that you might have found in some broadcast studios not so long ago. Still, decent quality costs money, and for that reason [Spirit532] has built his own high quality condenser microphone for less expenditure.

The capsule and body are off-the-shelf items — what he’s produced is the bias voltage supply and preamplifier. In both cases these are the interesting parts of a condenser microphone, so their circuit bears a second look.

The condenser microphone takes a diaphragm and turns it into one side of a capacitor. If you apply a charge to this capacitor, the voltage over it changes minutely with the capacitance as the diaphragm vibrates. Thus to have a usable audio signal level a high-voltage bias supply is required to provide the charge, and a very high impedance preamplifier circuit to catch the signal without draining the capacitor.

His bias supply is a charge pump using a string of diodes and capacitors fed by a chain of CMOS inverters, with an RC filter and resistor chain to provide that super-high impedance. The preamplifier meanwhile is a unity gain high-impedance op-amp with an inverting stage to provide a balanced connection. For good measure the circuit also includes a phantom power supply.

This is an interesting project for anyone with an interest in audio. if you’re further interested in condenser microphones, how about also looking at electret microphones?

Icicle Patterns With Custom Gantry

March 22, 2023 by Bryan Cockfield 2 Comments

[Cranktown City] uses a number of custom-built linear rails used as gantries for various tools in the shop. The first is on a plasma cutter, which is precise but difficult to set up or repair. Another is for mounting a camera, and while it is extremely durable, it’s not the most precise tool in the shop. Hoping to bridge the gap between these two, he’s building another gantry with a custom bearing system, and to test it he’ll be using it to create patterns in icicles hanging from an eave at his shop.

While this isn’t the final destination for this gantry, it is an excellent test of it, having to perform well for a long period of time in an extremely cold environment. The bearing system consists of a piece of square steel tubing turned 45° inside another larger square steel tube and held in place with two sets of three bearings with V-shaped notches. To drive the gantry he is using a motor with a belt drive, and for this test a piece of drip irrigation is mounted to it which lets out a predetermined amount of water on top of the roof to create numerous icicles beneath with various programmed lengths.

After a few test runs the gantry system can create some icicles, although they don’t have the exact sine wave shape that [Cranktown City] programmed into it. They are varying lengths though, and with no more cold days in the forecast he’s called it a success. This isn’t the final destination for this robotic linear gantry, though, but it did help him work out some of the kinks with it beforehand. For other sources of inspiration, take a look at this linear rail system also used for driving various robotic tooling.

Continue reading “Icicle Patterns With Custom Gantry” →

Fresh PCBs For The Quickshot II And II Plus Joysticks

March 21, 2023 by Maya Posch 13 Comments

The Quickshot II was released by Spectravideo in 1983 for the Commodore 64 and compatible systems, with the Quickshot II Plus following the next year. After decades of regular use, it’s quite understandable that these old-timers may be having some functional issues, but as long as the plastic parts are still good, [Stephan Eckweiler]’s replacement PCBs may be just the thing that these joysticks need to revitalize them for another few decades.

What may be a matter of taste is that these replace the nice tactile clicky switches on the QS II Plus with SMD push buttons, but compared to the stamped metal ‘button’ construction of the original QS II, the new board is probably a major improvement. As for the BOM, it features two ICs: a 74LS00 latch and NE555 timer, along with the expected stack of passives and switches, both through-hole and SMD.

The PCB contains break-off boards for the switches within the joystick itself, requiring a bit of wiring to be run to the main PCB before soldering on the DE-9 connector and connecting the joystick for a test run to a Commodore 64. All one needs now is a 3D printable enclosure version to create more QS II joysticks for some multiplayer action.

BitTorrent For Language Models

March 21, 2023 by Al Williams 6 Comments

In the old days of the Internet, FTP was sufficient for downloading the occasional file. But with the widespread use of computer audio and video, it was easy to swamp an FTP server so — eventually — BitTorrent was born. The idea was you would download bits and pieces of a file from different places and, in theory, people would download bits and pieces that you have if they need them. Now Petals wants to use this same method with language models. These AI language models are all the rage, but they take significant computer resources. The idea behind Petals is like BitTorrent. You handle a small part of the model (about 8 gigabytes which is small compared to the 352 gigabytes required), and other people have other parts.

Of course, if you are privacy-minded, that means that some amount of your data is going out to the public, but for your latest chatbot experiments, that might not be a big problem. You can install Petals in an Anaconda environment or run a Docker image if you don’t want to set up anything. If you just want to access the distributed network’s chatbot based on BLOOMZ-176B, you can do that online.

Continue reading “BitTorrent For Language Models” →

Dispense 60 Bolts In 2.3 Seconds

March 21, 2023 by Michael Shaub 26 Comments

We’ve covered a number of projects that assist makers who need to fill orders for their small businesses, or kitting. [Helmke] has sorted thousands of pieces of hardware that they include with 3D printed parts sold online. They have been developing an alternative, a modular system for sorting and packaging specific quantities of parts.

After the break, check out the latest video from their small but growing channel for a very clear walk-through of the counting system they’ve been iterating on. The 2nd video in the series explores solenoids, Geneva drives, and ultimately a sprocket to dispense a variable number of bolts from the sorting machine. The approach gives consistent results, easily to vary quantities, and is fast! These videos are also rich with lots of small details you might want to explore on your own like magnetic part feeding, discussions of different sensors for detecting and counting parts, 3D printed gear box designs, and we love the use of stackable crates for project enclosures.

We hope to see more videos from [Helmke] in the series as the project matures for deeper dives into the existing mechanisms and new features they develop next. Hungry for more? We’ve brought you everything from cutting and stripping wire, to SMD tape, to resistors, to laser-cut parts. Continue reading “Dispense 60 Bolts In 2.3 Seconds” →

3D-Printed Parts Don’t Slow Down This Speedy Printer

March 21, 2023 by Dan Maloney 7 Comments

Truth be told, we generally find speed sports to be a little boring. Whether it’s cars going around in circles for hours on end or swimmers competing to be a few milliseconds faster than everyone else, we just don’t feel the need for speed. Unless, of course, you’re talking about speedy 3D printers like “The 100”, which claims to produce high-quality prints in a tenth the time of an ordinary printer. In that case, you’ve got our full attention.

What makes [Matt the Printing Nerd]’s high-speed printer interesting isn’t the fact that it can do a “Speedboat Run” — printing a standard Benchy model — in less than six minutes. Plenty of printers can do the same thing much, much faster. The impressive part is that The 100 does it with a 3D-printed frame. In fact, most of the printer’s parts are 3d printed, a significant departure from most speed printer builds, which generally shy away from printed structural elements. [Matt]’s design also aims to keep the center of gravity of all the printer’s components within a very small area, which helps manage frame vibrations that limit print quality. The result is that the CoreXY gantry is capable of a speed of 400 mm/s and an eye-popping 100,000 mm/s² acceleration. What also sets [Matt]’s printer apart is that The 100 is designed to be a daily driver. It has a generous 165 mm x 165 mm print bed, which is far more useful than a bed that’s barely bigger than a standard Benchy.

The video below has much more details on the open-source build, plus some nice footage of some speed runs. The quality of the prints, even done at speed, is pretty impressive. Perhaps there is a point to speed sports after all.

Continue reading “3D-Printed Parts Don’t Slow Down This Speedy Printer” →