Statistics and Hacking: An Introduction to Hypothesis Testing

In the early 20th century, Guinness breweries in Dublin had a policy of hiring the best graduates from Oxford and Cambridge to improve their industrial processes. At the time, it was considered a trade secret that they were using statistical methods to improve their process and product.

One problem they were having was that the z-test (a commonly used test at the time) required large sample sizes, and sufficient data was often unavailable. By studying the properties of small sample sizes, William Sealy Gosset developed a statistical test that required fewer samples to produce a reasonable result. As the story goes though, chemists at Guinness were forbidden from publishing their findings.

So he did what many of us would do: realizing the finding was important to disseminate, he adopted a pseudonym (‘Student’) and published it. Even though we now know who developed the test, it’s still called “Student’s t-test” and it remains widely used across scientific disciplines.

It’s a cute little story of math, anonymity, and beer… but what can we do with it? As it turns out, it’s something we could probably all be using more often, given the number of Internet-connected sensors we’ve been playing with. Today our goal is to cover hypothesis testing and the basic z-test, as these are fundamental to understanding how the t-test works. We’ll return to the t-test soon — with real data. Continue reading “Statistics and Hacking: An Introduction to Hypothesis Testing”

We Need To Have A Chat About Something Important

Yes, I really did print this the day before the story broke.
Yes, I really did print this the day before the story broke.

With hindsight, I picked the wrong day to 3D print a Cap’n Crunch whistle downloaded from Thingiverse. I was covering the hackspace textile evening, so I set the Ultimaker going and headed off to spend my evening making a laptop pouch. My whistle, a reasonable reproduction of the famous cereal packet novelty whose 2600 Hz tone allowed special access to American telephone networks, was ready for me to take away as I headed home.

The next day, there it was. The legendary phreaker [John Draper], also known as [Captain Crunch] after his use of that free whistle, was exposed as having a history of inappropriate conduct towards teenage boys and young men who he encountered in his tours of the hacker community as a celebrity speaker.

My whistle will no longer go on a lanyard as a piece of cool ephemera, it’s sitting forlornly on my bench. The constant procession of harassment allegations that have been in the news of late have arrived at our doorstep. Continue reading “We Need To Have A Chat About Something Important”

Mike Ossmann and Dominic Spill: IR, Pirates!

Mike Ossmann and Dominic Spill have been at the forefront of the recent wave of software-defined radio (SDR) hacking. Mike is the hardware guy, and his radio designs helped bring Bluetooth and ISM-band to the masses. Dominic is the software guy who makes sure that all this gear is actually usable. The HackRF SDR is still one of the best cheap choices if you need an SDR that can transmit and receive.

So what are these two doing on stage giving a talk about IR communication? Can you really turn traffic lights green by blinking lights? And can you spoof a TV remote with a cardboard cutout, a bicycle wheel, and a sparkler? What does IR have to do with pirates, and why are these two dressed up as buccaneers? Watch our video interview and find out, or watch the full talk for all of the juicy details.

Continue reading “Mike Ossmann and Dominic Spill: IR, Pirates!”

The Flight That Made The Calculator And Changed The World

It was the fall of 1965 and Jack Kilby and Patrick Haggerty of Texas Instruments sat on a flight as Haggerty explained his idea for a calculator that could fit in the palm of a hand. This was a huge challenge since at that time calculators were the size of typewriters and plugged into wall sockets for their power. Kilby, who’d co-invented the integrated circuit just seven years earlier while at TI, lived to solve problems.

Fig. 2 from US 3,819,921 Miniature electronic calculator
Fig. 2 from US 3,819,921 Miniature electronic calculator

By the time they landed, Kilby had decided they should come up with a calculator that could fit in your pocket, cost less than $100, and could add, subtract, multiply, divide and maybe do square roots. He chose the code name, Project Cal Tech, for this endeavor, which seemed logical as TI had previously had a Project MIT.

Rather than study how existing calculators worked, they decided to start from scratch. The task was broken into five parts: the keyboard, the memory, the processor, the power supply, and some form of output. The processing portion came down to a four-chip design, one more than was initially hoped for. The output was also tricky for the time. CRTs were out of the question, neon lights required too high a voltage and LEDs were still not bright enough. In the end, they developed a thermal printer that burned images into heat-sensitive paper.

Just over twelve months later, with the parts all spread out on a table, it quietly spat out correct answers. A patent application was filed resulting in US patent 3,819,921, Miniature electronic calculator, which outlined the basic design for all the calculators to follow. This, idea borne of a discussion on an airplane, was a pivotal moment that changed the way we teach every student, and brought the power of solid-state computing technology into everyday life.

Continue reading “The Flight That Made The Calculator And Changed The World”

Can Commodity RC Controllers Stay Relevant?

Visualize some radio controlled airplane fanatic of yesteryear, with the requisite giant controller hanging from a strap, neck craned to see the buzzing dot silhouetted against the sky. It’s kind of a stereotype, isn’t it? Those big transmitters were heavy, expensive, and hard to modify, but that was just part of the challenge. Additionally, the form factor has to a degree remained rigid: the box with gimbals — or for the 3-channel controller, the pistol-grip with the big pot that looks like a cheesy race car wheel.

With so much changing in RC capabilities, and the rise of custom electronics across so many different applications, can commodity RC controllers stay relevant? We’re facing an age where the people who invest most heavily in RC equipment are also the ones most likely to want, and know how to work with customization for their rapidly evolving gear. It only makes sense that someone will rise up to satisfy that need.

Continue reading “Can Commodity RC Controllers Stay Relevant?”

The Database of the Time Lords

Time zones have been a necessity since humans could travel faster than a horse, but with computers, interconnected over a vast hive of information, a larger problem has emerged. How do you keep track of time zones? Moreover, how do you keep track of time zones throughout history?

Quick question. If it’s noon in Boston, what time is it in Phoenix? Well, Boston is in the Eastern time zone, there’s the Central time zone, and Phoenix is in the Mountain time zone; noon, eleven, ten. If it’s noon in Boston, it’s ten o’clock AM in Phoenix. Here’s a slightly harder question: if it’s noon in Boston, what time is it in Phoenix during Daylight Savings Time? Most of Arizona doesn’t observe Daylight Savings Time, so if it’s noon in Boston, it’s 9 AM in Phoenix. What about the Navajo Nation in the northwestern part of Arizona? Here, Daylight Savings Time is observed. You can’t even make a rule that all of Arizona is always on Mountain Standard Time.

Indiana is another example of bizarre time zones. For most of the 20th century, Indiana was firmly in the Central time zone. Starting in the 1960s, the line between Eastern and Central time slowly moved west from the Ohio border. Some countries opted not to observe Daylight Savings Time. In 2006, the entire state started to observe DST, but the northwest and southwest corners of the state remained firmly in the Central time zone. The odd geographic boundaries of time zones aren’t limited to the United States, either; Broken Hill, New South Wales, Australia is thirty minutes behind the rest of New South Wales.

Working out reliable answers to all of these questions is the domain of the Time Zone Database, a catalog of every time zone, time zone change, and every strange time-related political argument. It records Alaska’s transition from the Julian to the Gregorian calendar. It describes an argument in a small Michigan town in 1900. It’s used in Java, nearly every kind of Linux, hundreds of software packages, and at least a dozen of the servers and routers you’re using to read this right now.

Continue reading “The Database of the Time Lords”

Organizing Teams With Collective Fictions

There is often an observable difference between what is considered the right thing to do, and what actually is being done.

Terry Pratchett said it best when he made Death declare mercy and justice nonexistent: “TAKE THE UNIVERSE AND GRIND IT DOWN TO THE FINEST POWDER AND SIEVE IT THROUGH THE FINEST SIEVE AND THEN SHOW ME ONE ATOM OF JUSTICE, ONE MOLECULE OF MERCY.” (Note that Death is not shouting, he simply speaks upper case.)

We can’t measure justice and mercy. These are collective fictions — things we agree to believe to enable us to get along — and finding consensus on the immeasurable extends to political systems, religion, and most of economics. In a recent article [zwischenzugs] makes the point that methodologies in software development fall into the same category. Like collective societal fictions, methodologies tend to elicit strong emotional responses among those dealing with them.

A software development methodology is a playbook for getting from nothing to something. It’s a control system for how people working on the project spend their time. And there are a lot of these prescribed methods, from Agile to Waterfall, and any combination of letters is likely to turn an abbreviation for a methodology. An interesting game when hanging out in groups of software engineers is to start the “Have you ever tried the…” conversation. Just don’t expect to move to another topic anytime soon.

One disheartening aspect of methodologies is their resistance to scientific scrutiny. Two samples of development teams will differ wildly in so many characteristics that a meaningful comparison of the way they organize their work is not possible. Which will leaves us with anecdotes and opinions when discussing these things.

Current opinions regarding the impact of methodologies on the success of a project range from ‘marginal’ to ‘essential’. The latter position is mainly propagated by consultants selling agile certifications, so you may want to take it with a grain of salt. Whether a team adheres strongly to the methodology or adopts it in name only, it’s obvious they serve a purpose — but that purpose may not match the face value of the method.

Continue reading “Organizing Teams With Collective Fictions”