Data scraping and visualization with Python


[Greg] built himself a small indicator dial with his laser cutter, and wanted to use it for visualizing server performance and load information. Before he started using it for server monitoring however, he thought he should test out his data parsing skills on a simpler data set.

Pachube has a wealth of information that can be freely used for whatever project you might have in mind, so [Greg] started looking around for something interesting to track. Eventually he located the data feed for a tanker ship and wired his dial to display the ship’s speed. He uses a Python script to interface with the Pachube API, which is fed to his Netduino board. A servo motor then changes the position of the dial based on the feed’s data. Since large tankers don’t change speed often, the experiment was a bit of a letdown. He searched for a bit and tuned into another feed that tracked wind speed in New Zealand, getting much better results.

His future plans include hooking it directly to his network and eventually using it to monitor his servers…at least once the novelty of tracking random data feeds wears off.

All of his code is available on GitHub, and he is happy to make a gauge for anyone who is interested, though he doesn’t currently list a price.

Data plotting for the visually impaired

This setup helps to represent data in a meaningful way to for visually impaired people. It uses a combination of physical objects to represent data clusters, and audio feedback when manipulating those objects. In the video after the break you’ll see that the cubes can orient themselves to represent data clusters. The table top acts as a graphing field, with a textured border as a reference for the user. A camera mounted below the clear surface allows image processing software to calculate the locations for the cubes. Each cube is motorized and contains an Arduino and ZigBee module, listening for positioning information from the computer that is doing the video processing. Once in position, the user can move the cubes, with modulated noise as a measure of how near they are to the heart of each data cluster.

The team plans to conduct further study on the usefulness of this interactive data object. We certainly see potential for hacking as this uses off-the-shelf components that are both inexpensive, and easy to find. It certainly reminds us of a multitouch display with added physical tokens.

Continue reading “Data plotting for the visually impaired”

Easy data input for LabVIEW

Props go to [Michael Nash] for establishing an interface between National Instrument’s labVIEW and an Arduino (an example video using a potentiometer is above). Personally, from the one time we were forced to use labVIEW, we hated every second of it.

One reason it’s so terrible, is the Data Acquisition Modules cost well into the hundreds of dollars, yet the documentation and help resources are very scarce. By using an Arduino instead of the modules, the price and difficulty decrease a considerable amount. Which begs the question why has it taken so long to get a decent (and so simple) of a setup working?

Keyboard input for PlayStation

Anyone who has tried their hand at RPG Maker 1 (or any text input with a controller) knows how difficult it can be typing long paragraphs into the console. [Thutmose] is here to save the day with Kupid 1.0 (2.0 in production). A PICAXE takes ps/2 keyboard input and converts it to a series of d-pad button presses for PS1 and PS2 controllers, providing quick data entry compared to the previously monotonous task.

We’re happy to learn that the source code and hardware is released, meaning it has the potential to be easily adapted to any controller/console.

1-Wire LAN with an LCD

[Viktor], one of our favorite avid hackers, has been playing around with 1-wire systems all this month. What started out as a MicroLAN Fonera has turned into an iButton interface, to a 1-wire powered hub, and finally a 1-wire character driven LCD. Anyone looking at 1-wire systems or OWFS could surely benefit from his testing.

However, if you still haven’t gotten your fill of 1-wire goodness, let us remind you of the 1-wire HVAC and IPv6 to 1-wire protocol translator.

[Thanks Juan]

Homing pigeon email


Yes, you read that correctly: electronic mail carried by birds. [Ferdinand] tipped us off to this story, which involves combining new and old methods in transferring data. The Unlimited Group, a firm in a remote section of South Africa, transfers loads of encrypted documents to a second office 50 miles away. A pricey broadband connection would take between 6 hours and two days to transfer a standard load (4GB) of data between these locations. On the other hand, Winston (seen above) can complete an equivalent flight within 45 minutes. A memory card is strapped to his leg, and using his wit and instinct, Winston finds his way home. For those without their calculators on hand, Winston’s bandwidth is between 7x and 63x faster than what they had before. If his flash card were to be upgraded to 16GB, that would be an instant fourfold increase on top of current gains. As [Mark] pointed out on the Daily Mail website, homing pigeons still need to be taken back to their departure point.

This solution still has its advantages over a courier: they are lower in cost, they work over longer hours, and have potentially faster delivery speeds. Multiple pigeons can be transported back at once, and released with data as needed.

Recovering photos with PhotoRec


A coworker approached us today with a corrupted SD card. It was out of her digital camera, and when plugged in, it wasn’t recognized. This looked like the perfect opportunity to try out [Christophe Grenier]’s PhotoRec. PhotoRec is designed to recover lost files from many different types of storage media. We used it from the command line on OSX, but it works on many different platforms.

It’s a fairly simple program to use. We plugged in the card and launched PhotoRec. We were prompted to select which volume we wanted to recover. We selected “Intel” as the partition table. PhotoRec didn’t find any partitions, so we opted to search the “Whole disk”. We kept the default filetypes. It then asked for filesystem type where we chose “Other” because flash is formatted FAT by default. We then chose a directory for the recovered files and started the process. PhotoRec scans the entire disk looking for known file headers. It uses these to find the lost image data. The 1GB card took approximately 15 minutes to scan and recovered all photos. This is really a great piece of free software, but hopefully you’ll never have to use it.