Conquering The Earth With Cron

The GOES-R series of Earth observation satellites are the latest and greatest NASA has to offer. As you might expect, part of the GOES-R job description is imaging Earth at high-resolution, but they also feature real-time lighting monitoring as well as enhanced solar flare and space weather capabilities. Four of these brand new birds will be helping us keep an eye on our planet’s condition into the 2030s. Not a bad way to spend around 11 billion bucks.

To encourage innovation, NASA is making the images collected by the GOES-R satellites available to the public through a collaboration with Google Cloud Platform. [Ben Nitkin] decided to play around with this data, and came up with an interactive website that let’s you visualize the Earth from the perspective of GOES-R. But don’t let those slick visuals fool you, the site is powered by a couple cron jobs and some static HTML. Just as Sir Tim Berners-Lee intended it.

But it’s not quite as easy as scheduling a wget command; the images GOES-R collects are separated into different wavelengths and need to be combined to create a false-color image. A cron job fires off every five minutes which downloads and merges the raw GOES-R images, and then another cron job starts a Python script that creates WebM time-lapse videos out of the images using ffmpeg. All of the Python scripts and the crontab file are available on GitHub.

Finally, with the images merged and the videos created, the static HTML website is served out to the world courtesy of a quick and dirty Python web server. The site could be served via something more conventional, but [Ben] likes to keep overhead as low as possible.

If you want to take the more direct route, we’ve covered plenty of projects focused on pulling down images from weather satellites; from using old-school “rabbit ears” to decoding the latest Russian Meteor-M N2 downlink.

15 thoughts on “Conquering The Earth With Cron

  1. GOES-R the Goserian, good evening. As a duly designated representative of the city, county, and state of New York, I order you to cease any and all supernatural activity and return forthwith to your place of origin or to the nearest convenient parallel dimension. (Ray Stantz – GhostBusters)

  2. I have to say, this is pretty neat. It definitely sounds like it’s higher resolution than the existing Landsat imagery. I work in an industry that consumes data like this, so to hear that imagery like this is being given away for free is pretty awesome (if you are the one using it, or pretty bad, if you are another imagery provider).

    1. Sub-kilometer resolution of cloudtops from GEO is quite an accomplishment for this former weatherman, but as a current employee of “another imagery provider” that works in the less-than-100-meter resolution range from LEO, I’m not so worried.

  3. Nice work on this write up!

    On the topic of cron, I had an idea for a useful series of articles that I’d like to see. Like it or not, most linux distributions come with systemd these days. Even if you’re anti-systemd, a lot of us will eventually encounter a system that uses it. One of the quirk about it is that now not every linux distribution comes with cron. Arch Linux comes to mind especially.

    It’d be great to have a good resource that coaches us through using systemd to do tasks like this! How would someone implement this project using systemd timers instead of cron, for example? Just a thought; the Hackaday crew can take it for what it’s worth!

      1. The lazy way is the right way! sudo pacman -Su cronie. Done.

        And while you’re at it, install ‘at’. Sometimes you just want to schedule a one-off quick and easy. No sense in reinventing the wheel, bundling it up with a “wheel service” and creating a silly proliferation of configuration files. (Yup, systemd, I’m talking to you!)

  4. The original stuff is even higher resolution!
    The sources are all about 10,000×10,000. This downsamples to 2000×2000 for processing, then to 1000×1000 for the video. (the latest hi-res view shows the 2k x 2k image)
    (The original data is around 1km resolution – you could always grab the Python and tweak it to give better results).

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.