Build an In Line Network Bandwidth Monitor

netTap1

[Kurt] likes to know what’s going on with his network. He already uses bandwidth checking software on his DD-WRT capable router, but he wanted a second opinion. So he built his own network monitor. [Kurt] started by building a passive Ethernet tap. He then needed a network interface chip that would serve his purposes. The common Wiznet chips used with Arduinos didn’t allow enough manipulation of raw packet data, so he switched to a Microchip ENC624J600 (PDF). The Microchip controller allowed him to count the bytes in the raw Ethernet packets.

With the Ethernet interface complete, [Kurt] turned his attention to a microcontroller to run the show. He started with an Arduino, but the lack of debugging quickly sent him to an Atmega128 in Atmel Studio. After getting the basic circuit working, [Kurt] switched over to a PIC24F chip. With data finally coming out of the circuit, he was able to tell that his original back-of-the-napkin calculations for bandwidth were wrong. [Kurt] created a PCB to hold the microcontroller, then wrote a Python program to plot the data output from his circuit. The bandwidth plot matched up well with the plot from DD-WRT. Now he just needs a giant LED matrix to show off his current network stats!

How Hacker News Page Rankings Really Work

hacker-news-page-rankings

Page rankings are the secret sauce of websites that automatically aggregate user submissions. The basic formula used by Hacker News was published a few years back. But there are several pieces of the puzzle that are missing from that specification. [Ken Shirriff] recently published an analysis that digs deeper to expose the article penalization system used by Hacker News’ ranking engine.

One might assume that the user up and down votes are what determine a page’s lifespan on the front page. But it turns out that a complex penalization system makes a huge difference. It takes into account keywords, and domain names but also weighs controversy. It’s a bit amusing to note that this article on the topic was itself penalized, knocking it off of the front page.

You can get the full details of the system from his post, but we found his investigation methods to be equally interesting. He scraped two pages of the news feed every minute using Python and the Beautiful Soup package (a pretty common scraping practice). This data set allowed him to compare the known algorithm with actual results. What was left were a set of anomalies that contained enough sense for him to reverse engineer the unpublished formulas being used.

Can An 8 Node Raspberry Pi Cluster Web Server Survive Hackaday?

8-pi-cluster-3

Plenty of folks have used their Raspberry Pi as a web server. [Steve] however is the first 8 node load balanced pi cluster server we’ve run into.  While we have seen pi clusters before, they’ve never been pressed into service as a public facing web server. [Steve] has created a really nice informative website about the Raspberry Pi, and Linux in general. As his page views have increased, he’s had to add nodes to the server. Currently [Steve] sees about 45,000 page views per month.

At first glance it would seem that the load balance system would be the weak link in the chain. However, [Steve] did realize that he needed more than an Pi to handle this task. He built the load balancer using an old PC with 512MB of RAM and a 2.7GHz x86 CPU. The most important thing about the balancer is dual network interfaces, one side facing the internet, the other facing the Pi cluster. The balancer isn’t a router though. Only HTTP requests are forwarded. The Pi nodes themselves live on their own sub net. Steve has run some basic testing with siege, however nothing beats a real world test. We figured a couple of links in from Hackaday would be enough to acid test the system.

Monitor GitHub Activity with an RGB LED Matrix

tim-display

Ever wonder who is forking your code? [Jack] did, so he built a real time GitHub activity display for his company’s repositories. The display is based a Wyolum The Intelligent Matrix (TiM) board. The TiM is an 8 x 16 matrix of the ubiquitous WS2811/Smart Pixel/NeoPixel RGB LEDs with built-in controller. We’re seeing more and more of these serial LEDs as they drop in price. Solder jumpers allow the TiM to be used as 8 parallel rows of LEDs (for higher refresh rates), or connected into one long serial chain.

[Jack] wasn’t worried about speed, so he configured his board into a single serial string of LEDs. An Arduino drives the entire matrix with a single pin. Rather than reinvent the wheel, [Jack] used Adafruit’s NeoMatrix library to drive his display. Since the TiM uses the same LEDs as the Adafruit NeoPixel Matrix, the library will work. Chalk up another victory for open source hardware and software!

An Electric Imp retrieves Github data via WiFi and passes it on to the Arduino. This is a good use of a microcontroller such as the AVR on the Arduino. [Jack’s] display has a scrolling username. Every step in the scroll animation requires all the pixel data be clocked out to the TiM board. The Arduino can handle this while the IMP takes care of higher level duties.

[Read more...]

Browsing the web one step at a time

After modifying his new manual treadmill to fit under his standing desk, [Brian Peiris] found a way to let him stroll all over the internet.

After removing the treadmill’s original time/distance display, [Peiris] reverse engineered the speed sensor to send data to an Arduino and his PC.  We’ve seen a number of projects that interface treadmills with virtual worlds, but what really makes this project stand out is a simple script using the Throxy Python library which allows the treadmill to throttle his machine’s internet connection.

The end result is a browsing experience that reacts to how fast the user runs.  In the demonstration video, you can see Peiris tiptoe through images or jog through YouTube videos.  A minimum bandwidth setting keeps the connection live, so if you can’t make it all the way through that HD Netflix movie, taking a breather won’t time out the connection.

It’s certainly a great way to get in shape, or at the very least, it’ll make your ISP’s bandwidth cap feel a lot bigger.

Video after the jump.

[Read more...]

Free yourself from social media with classical conditioning

pavlov

While [Robert] and [Dan] should be working on their dissertation, they found they actually spend a whole lot of time whiling away their days on Facebook and other social media sites. Taking inspiration from a Skinner box, they rigged up their computer to shock them every time they surfed on over to Facebook.

Their build uses the UI inspector in OS X and a Python script to activate an Arduino connected to one of those trick ‘shocking chewing gum’ pranks. The contacts for this shocker are attached to a keyboard wrist rest, providing a wonderful tingling sensation whenever the guys surf on over to Facebook.

Because shocks just aren’t extreme enough, [Robert] and [Dan] took their build one step further by invoking the wrath of Mechanical Turk users. They wrote a Python script to look at their UI inspector and submit a job to Mechanical Turk whenever they logged on to Facebook. The result is a random person being paid $1.40 to yell at [Robert] or [Dan] over the phone for wasting time on Facebook.

Video below, and be sure to like this post on Facebook.

[Read more...]

Defeating reddit’s CAPTCHA

cap

Here’s something we’re sure SEO specialists, PR reps, and other marketeers already know: how to write a script to game reddit.

The course of upvotes and downvotes controls which submission makes it to the front page of reddit. These submissions are voted on by users, and new accounts must log in and complete a CAPTCHA to vote. [Ian] discovered that reddit’s CAPTCHA is not really state-of-the-art, and figured out how to get a bot to solve it

The method exploits the 8-bit nature of the distorted grid in the CAPTCHA. Because this grid isn’t pure black or pure white, it’s at a lower intensity than the letters in the CAPTCHA. Putting the CAPTCHA through a threshold filter, deleting any blocks of pixels smaller than 20 pixels, and running it through a classifier (PDF there), a bot can guess what the letters of the CAPTCHA should be.

Out of the 489 CAPTCHAs [Ian] fed into his algorithm, only 28 – or 5.73% – were guessed correctly. However, because he knows which CAPTCHAs had failed segmentation, ignoring those can increase the success rate to 10%. Theoretically, by requesting new CAPTCHAs, [Ian] can get the accuracy of his CAPTCHA bot up to about 30%.

Combine this with a brilliant auto voting script that only requires someone to enter CAPTCHAs, and you’ve got the recipe for getting anything you want directly to the front page of reddit. Of course you could do the same with a few memes and pictures of cats, but you knew that already.

Follow

Get every new post delivered to your Inbox.

Join 96,408 other followers