Data Scraping And Visualization With Python


[Greg] built himself a small indicator dial with his laser cutter, and wanted to use it for visualizing server performance and load information. Before he started using it for server monitoring however, he thought he should test out his data parsing skills on a simpler data set.

Pachube has a wealth of information that can be freely used for whatever project you might have in mind, so [Greg] started looking around for something interesting to track. Eventually he located the data feed for a tanker ship and wired his dial to display the ship’s speed. He uses a Python script to interface with the Pachube API, which is fed to his Netduino board. A servo motor then changes the position of the dial based on the feed’s data. Since large tankers don’t change speed often, the experiment was a bit of a letdown. He searched for a bit and tuned into another feed that tracked wind speed in New Zealand, getting much better results.

His future plans include hooking it directly to his network and eventually using it to monitor his servers…at least once the novelty of tracking random data feeds wears off.

All of his code is available on GitHub, and he is happy to make a gauge for anyone who is interested, though he doesn’t currently list a price.

8 thoughts on “Data Scraping And Visualization With Python

  1. Sorry for OT but I found the same Pocket Oscilloscope on ebay yesterday, and i was wondering if any of you hackaday’ers have had any experiences with it?

    Its dirt cheap, and maybe it is hackable too?

  2. hack of the century
    but seriously, is a rabbit and a snail the best way to describe server load?
    a slow to respond server would indicate its having to work pretty fast to keep up with demand
    i would go with an ice cream for no load and a turd for full load.

  3. @Johannes i have the DSO nano v2, it is very hackable. it is open hardware and comes with a miniature screwdriver to take it apart. Same goes with firmware. i like it so much i will get the dso quad next month.

    things to know on the nano v2:
    -very easy to mod
    -transparent lcd screen cover is too soft.
    -the stereo plug for the probe; if it is out by 3mm IT WILL short the probes.
    -probe ground is same ground as usb.
    -you cant live without it after a while.

    nice hack, the tanker ship display thing gave me flashbacks from Hackers :P

  4. @HAD group:
    The fellow is not running a Netduino, (although I wish he was.) He is indeed running a regular Arduino. But he also had a Netduino around as well, so that’s next up he says.

  5. On behalf of Turtles. They are persistent and dogged at getting things done. Search YouTube for fast turtle you won’t believe the warp factor. Rabbits are like cable internet, stop and go this way and that way. If they aren’t laying eggs or making more. Snail-mail.
    A bank of these gauges are power output of the Krell machine (Forbidden Planet) look out!

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.