Twitter RNG is Powered By Memes

Twitter is kind of a crazy place. World leaders doing verbal battle, hashtags that rise and fall along with the social climate, and a never ending barrage of cat pictures all make for a tumultuous stream of consciousness that runs 24/7. What exactly we’re supposed to do with this information is still up to debate, as Twitter has yet to turn it into a profitable service after over a decade of operation. Still, it’s a grand experiment that offers a rare glimpse into the human hive-mind for anyone brave enough to dive in.

One such explorer is a security researcher who goes by the handle [x0rz]. He’s recently unveiled an experimental new piece of software that grabs Tweets and uses them as a “noise” to mix in with the Linux urandom entropy pool. The end result is a relatively unpredictable and difficult to influence source of random data. While he cautions his software is merely a proof of concept and not meant for high security applications, it’s certainly an interesting approach to introducing humanity-derived chaos into the normally orderly world of your computer’s operating system.

Noise sampling before and after being merged with urandom

This hack is made possible by the fact that Twitter offers a “sample” function in their API, which effectively throws a randomized collection of Tweets at anyone who requests it. There are some caveats here, such as the fact that if multiple clients request a sample at the same time they will both receive the same Tweets. It’s also worth mentioning that some characters are unusually likely to make an appearance due to the nature of Twitter (emoticons, octothorps pound signs, etc), but generally speaking it’s not a terrible way to get some chaotic data on demand.

On its own, [x0rz] found this data to be a good but not great source of entropy. After pulling a 500KB sample, he found it had an entropy of 6.5519 bits per byte (random would be 8). While the Tweets weren’t great on their own, combining the data with the kernel’s entropy pool at /dev/urandom provided something that looked a lot less predictable.

The greatest weakness of using Twitter as a source of entropy is, of course, the nature of Twitter itself. A sufficiently popular hashtag on the rise might be just enough to sink your entropy. It’s even possible (though admittedly unlikely) that enough Twitter spam bots could ruin the sample. But if you’re at the point where you think hinging your entropy pool on a digital fire hose of memes and cat pictures is sufficient, you’re probably not securing any national secrets anyway.

(Editor’s note: The way the Linux entropy pool mixes it together, additional sources can only help, assuming they can’t see the current state of your entropy pool, which Twitter cats most certainly can’t. See article below. Also, this is hilarious.)

We’ve covered some fantastic examples of true random number generators here at Hackaday, and if you’re looking for a good primer for the Kingdom of the Chaotic, check out the piece by our own [Elliot Williams].

[Thanks to DarkSim905 for the tip]

What is Entropy and How Do I Get More of It?

Let’s start off with one of my favorite quotes from John von Neumann: “Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin. For, as has been pointed out several times, there is no such thing as a random number — there are only methods to produce random numbers, and a strict arithmetic procedure of course is not such a method.”

What von Neumann is getting at is that the “pseudo” in pseudorandom number generator (PRNG) is really a synonym for “not at all”. Granted, if you come in the middle of a good PRNG sequence, guessing the next number is nearly impossible. But if you know, or can guess, the seed that started the PRNG off, you know all past and future values nearly instantly; it’s a purely deterministic mathematical function. This shouldn’t be taken as a rant against PRNGs, but merely as a reminder that when you use one, the un-guessability of the numbers that it spits out is only as un-guessable as the seed. And while “un-guessability” isn’t a well-defined mathematical concept, or even a real word, entropy is.

That’s why entropy matters to you. Almost anything that your computer wants to keep secret will require the generation of a secret random number at some point, and any series of “random” numbers that a computer generates will have only as much entropy, and thus un-guessability, as the seed used. So how does a computer, a deterministic machine, harvest entropy for that seed in the first place? And how can you make sure you’ve got enough? And did you know that your Raspberry Pi can be turned into a heavy-duty source of entropy? Read on!

Continue reading “What is Entropy and How Do I Get More of It?”

Dual-mode Avalanche and RF Random Number Generator

[Paul] designed a new open-hardware RNG (random number generator) that includes two sources of entropy in a small package. The first source of entropy is a typical avalanche diode circuit, which is formed by a pair of transistors. This circuit creates high-speed random pulses which are sampled by the onboard microcontroller.

What makes this design unique is a second entropy source: a CC2531 RF receiver. The RF receiver continuously skips around channels in the 2.5Ghz band and measures the RF signal level. The least-significant bit of the signal level is captured and used as a source of entropy. The firmware can be configured to use either source of entropy individually, or to combine both. The firmware also supports optionally whitening the entropy byte stream, which evens out the number of 1’s and 0’s without reducing entropy.

The OneRNG uses the USB-CDC profile, so it shows up as a virtual serial port in most modern operating systems. With the rngd daemon and a bit of configuration, the OneRNG can feed the system entropy source in Linux. [Paul] also has a good writeup about the theory behind the entropy generator which includes images of his schematic. Firmware, drivers, and hardware design files are open-source and are available for download.

The Development Of A Hardware Random Number Generator

rng

[Ian] had a need for a lot of random numbers. There are dozens of commercial offerings when it comes to RNGs, but there are also hundreds of different ways for an electronics hobbyist to shoot random bits at a serial port. One of these methods is an RNG based on the avalanche breakdown noise in a PN junction. As with any circuit in hobbyist electronics, there are dozens of prototypes floating out there on the web, but not too many finished projects. [Ian] decided he would build one of these RNGs as a prototype and bring it to something resembling a finished project.

An avalanche noise RNG takes advantage of the fact that a strongly reverse-biased PN junction, like one found in a transistor, will create a condition where one electron knocks another electron out of place, leading to a sustained chain reaction. It’s quantum, it’s chaotic, it makes for a great source for a random number generator, and there are already dozens of prototype circuits around the Internet.

[Ian] took one of these circuit designs by [Will Ware] and started the iterations that would lead to a finished design. Round one was a simple PCB with the basic circuit and a power supply. Just a few transistors, resistors, and a DC/DC boost converter. Confirming the circuit was generating noise, the next iteration brought in an ADC and an ARM micro with a USB interface. Iterating over this again with an improved ADC – 20 megasamples per second – the design finally reached a point where a final PCB could be designed.

In the end, [Ian] turned a simple circuit that could have been built on a breadboard into a USB device that throws 9kB/s of random data into a computer. The data are actually good, too: the project passed most of the Dieharder test suite, making it very useful for whatever crypto or gaming application [Ian] has in mind.

Improved hourglass entropy

improved-hour-glass-entropy

[Wardy] built himself a high quality entropy source with parts he had lying around. It’s based on the hourglass entropy project we saw in a links post earlier this month. Just like that project, he is bouncing a laser off of the falling sand and reading the result. But he brings a few innovations to the party, and has test results to back up his work.

The first change is an obvious one; motorize the hourglass so that you don’t need to flip it by hand. We thought this might mess with the laser alignment but the clip after the break proved us wrong. He changed up the sensor, using an LED connected to the base of an NPN transistor. The next change was to mount the light sensor at an angle to the laser rather than straight on. This picks up reflections of the laser and not the direct beam itself, resulting in a wider range of readings.

He used an Ethernet shield to get the system on the network. It’s pushing 420k random numbers per second and was tested with the DieHarder suite. It didn’t get a very high score, but it did pass the test.

Continue reading “Improved hourglass entropy”