Apple II Weather Display (part 1)

Due to computer issues I had to rob some parts from my “electronics” computer, which wasn’t bad, since I was not working on anything at the time and I felt a software project itch. I also wanted to do something with my Apple //c, which resides on my computer desk, so this ghetto brute force “solution” to use the 25 year old computer as a weather display came about.

In a nutshell there is the Apple II, a serial cable, and a PC running linux mint 10 and a handful of command line utilities. My specific Apple is the fist revision of the //c which means its got a buggy rom and the serial port(s) can be troublesome, the best speed I was able to get was 600 baud with just basic, though every other model could probably go a little faster.

On the linux side, wget downloads html and the radar image from Weather Underground’s mobile site, which is not a perfect source, but its easy. A lua script phrases text and graphics into string patterns that the Apple II can handle as keyboard input, and its sent down a serial cable where it is drawn on screen in basic.

Yea its pretty darn slow … it typically takes about eight to twelve minutes to redraw the screen, which is not all that horrid (imo) considering what is going on, but anyone with a more serious take on this could find numerous ways to optimize it, I just wanted to see what it would look like.

Join us after the break for a short video and to read all the details about how this all works!

Stuff you need:

*nix PC with a serial port and something with the following software

GNU Wget

GNU core utilities

(both are  probably already installed )

Lua 5.1 (I used sudo apt-get install lua5.1)

ImageMagick (again sudo apt-get install imagemagick)

Lynx (I am just using it to strip html tags and didnt feel like using anything “proper”)

Sjinn, which is a nifty little command line program that lets you deal with the serial port without having to manually set it up and fuss with redirecting i/o in the terminal.

And my project folder.

On the Apple II side, you need a 128k IIe or newer, disk drive/controller, serial card, blank disk,  and high resolution graphics. For software you will need ADT Pro,  and the disk image I made. ADT Pro is the currently developed Apple Disk Transfer utility that allows you to copy disk images to and from the Apple II, and the disk image is contained in the zip file above.

Transfer the disk image to the Apple and reboot with that disk in the drive, once the screen is at the white box, you are free to run “lua a2weather.lua” on the PC. The lua script fetches data, pipes it to the Apple, and then goes to sleep for 45 min using the gnu sleep() command.

Since lua is doing all the lifting, I guess that’s a good place to start. Also, let’s get this out of the way in case you missed it, I am not a programmer, much like my writing, be prepared to see some massive sins!

First up is the main a2weather.lua script  (and I see wordpress is going to murder my formating)

-- Weather Underground to Apple //
-- 2011 Kevin Dady
--
-- main script:
-- read data off of the http://www.wunderground.com/ mobile site
-- phrase it
-- process text & graphics
-- send
-- sleep
-- loop
--
-- This software is provided 'as-is', without any express or implied
-- warranty.  In no event will the authors be held liable for any damages
-- arising from the use of this software.
--
-- Permission is granted to anyone to use this software for any purpose,
-- including commercial applications, and to alter it and redistribute it
-- freely, subject to the following restrictions:
--
-- 1. The origin of this software must not be misrepresented; you must not
--    claim that you wrote the original software. If you use this software
--    in a product, an acknowledgment in the product documentation would be
--    appreciated but is not required.
-- 2. Altered source versions must be plainly marked as such, and must not be
--    misrepresented as being the original software.
-- 3. This notice may not be removed or altered from any source distribution.
require("req/web")
require("req/text")
require("req/radar")
require("req/os_commands")

graphicsKey = "123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"

cmd.serialPort = "/dev/ttyS0"
cmd.webURL = "http://m.wund.com/US/FL/Orlando.html"
--cmd.webURL = "http://m.wund.com/cgi-bin/findweather/getForecast?brand=mobile&query=West+Yellowstone%2C+MT"

while true do
 web.get()
 web.sortTXT()
 web.cleanTXT()

 text.createIMG()
 text.convertIMG()
 text.sortIMG()
 text.packageIMG()
 text.sendIMG()

 radar.convertIMG()
 radar.sortIMG()
 radar.packageIMG()
 radar.send()

 -- clean up data
 radar.data.input  = {}
 radar.data.packed = {}
 radar.data.apl2 = {{},{},{},{},{},{}}
 text.data.input  = {}
 text.data.packed = {{},{},{},{}}
 text.data.apl2   = {{},{},{}}

 cmd.sleep("45m")
 -- after sleep do tell the apple and do it again
 cmd.sjinn("update")
end

… pretty straight forward, some comments, zlib license, Ah, required files! These are just other lua scripts with functions and variables in them, though there is no reason all of those functions could not be in this script, I just find it easier to deal with.

web.lua downloads html and the radar image from weather underground, strips HTML tags via lynx, searches for keywords, and packs those keywords into a table (tables in lua are like arrays, but you can put pretty much anything you want in them, even functions)

text.lua takes the text data gathered in web.lua and passes it to ImageMagick to make simple text images, then phrases those images into data strings and transfers that data to the Apple.

radar.lua dithers the radar image with ImageMagick, phrases the that file, builds data strings, and  transfers that image data to the Apple.

os_commands.lua contains functions to run command line programs like wget, lynx, etc.

The graphics key needs to remain constant, I will go into how that works in a little while, but each character represents a numeric value within a 35 pixel column of the screen, there are 280 horizontal pixels in Apple // high res graphics. Its ripped from base36, missing the zero value and you dont add the numbers up, each is its own.

Theres also a string to define your serial port, and one to put the url for the area you want to see, go to Weather Underground’s mobile site search for wherever (I dont know how well it works outside North America)  and copy/paste the resulting page url.

Next is a infinite while loop and the various functions contained within the above required scripts. Then, a little blurb where I define a bunch of empty tables (thats just me being re-assured tables start fresh and I didn’t goof up).

Next is web.lua

-- Weather Underground to Apple //
-- 2011 Kevin Dady
--
-- Web Processing:
-- get text data
-- get radar image
-- phrase text for apple //

web = {}

web.data = {}
web.keywords =
{
"National Weather Service:",
"Updated","Windchill","Temperature","Humidity",
"Conditions","Dew Point","animated radar image"}

web.get = function()
-- download html from website
cmd.wget(cmd.webURL, "text.html")
-- look for the radar image url
local file = io.open("temp/text.html")
local image = ""
for line in file:lines() do
if string.find(line, "jpg") then
image = line
break
end
end
file:close()
-- remove the html tags from image url
local firstQuote = string.find(image, "\"") + 1
local secondQuote = string.find(image, "\"", firstQuote) - 1
image = string.sub(image, firstQuote, secondQuote)
-- download image
cmd.wget(image, "radar.jpg")
end

web.sortTXT = function()
local fileLines = {}
-- use lynx to strip html tags
cmd.lynx("temp/text.html")
-- dump the lynx output file into a table
local file = io.open("temp/text.txt")
for line in file:lines() do
table.insert(fileLines, line)
end
-- look for keywords and scrape data
for y = 1, #fileLines do
local sub = ""
-- storm advisory
if string.find(fileLines[y], web.keywords[1]) ~= nil and web.data[1] == nil then
sub = string.gsub(fileLines[y + 1], "%d", "!")
sub = string.gsub(sub, " , ", " ")
table.insert(web.data, 1, sub)
-- last updated date and time
elseif string.find(fileLines[y], web.keywords[2]) ~= nil then
table.insert(web.data, 2, fileLines[y])
-- windchill
elseif string.find(fileLines[y], web.keywords[3]) ~= nil then
sub = string.gsub(fileLines[y], web.keywords[3], "")
table.insert(web.data, 3, sub)
-- everything but forecast
else
for keyWord = 4, (#web.keywords - 1) do
if string.find(fileLines[y], web.keywords[keyWord]) ~= nil and web.data[keyWord] == nil then
sub = string.gsub(fileLines[y], web.keywords[keyWord], "")
table.insert(web.data, keyWord, sub)
end
end
end
end
end

web.cleanTXT = function()
for y = 1, #web.data do
if web.data[y] ~= nil then
-- remove extra spaces from start of each string
while (string.sub(web.data[y], 1, 1) == " ") do
web.data[y] = string.sub(web.data[y], 2, -1)
end
-- remove degrees character
web.data[y] = string.gsub(web.data[y], "°", "")
end
end
-- Add keywords back in (skipping nill values, and tempature)
for keyWord = 3, (#web.keywords - 1) do
if web.data[keyWord] ~= nil then
if keyWord ~= 4 then
web.data[keyWord] = web.keywords[keyWord] .. ": " ..web.data[keyWord]
end
end
end
-- remove / from Temperature
web.data[4] = string.gsub(web.data[4], " / ", " ")
end

Yea I know its a mess, WordPress is ignoring tabs and it is only 40 columns if I had to guess. At the top there is a function called web.get(), this function uses wget to download the html page from weather underground, it then opens that file and scans through it for a JPG image, as luck would have it, the only jpg on the entire page is the radar. Once it has the URL for the radar it strips the HTML tags and uses wget again to download just the radar image.

The next function is web.sortTXT(), this sends the downloaded HTML file to lynx where it strips out the tags and spits out a plain text file. The function then reads each line looking for keywords. Some keywords require special action … like if it finds a storm advisory, it then knows to skip to the next line, because that line just says “National Weather Service:” and change the link number inserted by lynx to a” !”.  At the end of the function there is a little loop that looks for the rest of the keywords and then removes the actual keyword, this is needed to remove unwanted spaces between the keyword and data.

The final function is web.cleanTXT(), this goes through all the lines of data we have and removes all the leading spaces. It then adds back keywords to the appropriate line, while it ignores nil values as they were not present (like advisories or windchill). It also ignores the Temperature keyword as I don’t need the word Temperature in the display. Lastly I remove the “/” in my temperature data as it makes the graphics format funny, and its just to separate F and C values in the original string.

So we now have the software installed and running, data and radar downloaded from weather underground, text data scrapped, sorted and cleaned and we are now ready to make some graphics!

Join me in part 2 where I explain how graphics are reduced, encoded and drawn on the Apple II computer.


Comments

  1. aztraph says:

    Wow. Lots of work in that. Great exorcise.

  2. FaultyWarrior says:

    Awesome project! Can’t wait to see the second part!

    Great music on the video too!

  3. The Cageybee says:

    What horrible music….Love it!

    Needs Apple II with “high resolution graphics”..LOL

  4. fartface says:

    I remember when the internet was that slow!

    300 baud acoustical coupler modem and a “liberated” account at the local college to get online.

  5. BLuRry says:

    Nice. You can actually get the //c rom 0 port up to 115200 baud with some undocumented modes as long as you have a good serial cable. The trouble is you have to make one. I was able to cobble together one a few years ago out by ripping apart the 5 pin din connector off an old keyboard.

    I’m very curious to know how you did the graphics conversion, that darn hi-bit is a big pain in the neck. ;-) I have some graphics (also double-hires and a pesudo-131 color mode) conversion classes that can be found here: http://sourceforge.net/projects/a2gameserver/

  6. BLuRry says:

    http://a2gameserver.svn.sourceforge.net/viewvc/a2gameserver/a2gameserver/src/ags/ui/graphics/DHGR2Image.java?revision=162&view=markup

    Specifically, this is the class that renders two individual images, which can be returned with the getAppleImage(0) for aux and getAppleImage(1) for the main mem part. Flip between the two images every VBL and you can get a flickery version of the image with a higher perceptual number of colors.

  7. Scott says:

    Nice experiment.

    Feedback:
    – Use the Weather Underground API. It’s more robust. Otherwise the scrape method you use now will (someday) break.

    – Get a fixed ROM for your Apple. Or ditch the serial method and get a network card for it… you can do that you know.. :-)

  8. BLuRry says:

    Scott, there are no expansion card slots in the original apple //c. There are _some_ options that involve inserting a riser between the CPU and the mobo, but nothing for networking AFAIK. And even in a bigger // with an uthernet card, you still have to provide your own TCP/IP stack or code your whole thing inside Contiki. My take on this sort of thing was to write a little assembler routine to kick the port into the undocumented 115200 mode and add some checksum to the transfer routine to re-send blocks if they don’t transfer correctly.

  9. Scott says:

    @BLuRry – Interesting. Pardon me for the Ethernet comment. I was under the impression this was more trivial or better supported (by virtue of the large base of people still hacking the 8-bit Apples)

    If this were a //e, you would have room inside to stash a dd-wrt router and feed “serial data” to the computer (either serial as you do now, or tcp/ip).

    But I shouldn’t talk… I have an Atari 520ST and a (bondi blue original) iMac. I’m space limited, but if I could get them upgraded and wirelessly networked it would be easier to justify unboxing them. :-)

  10. BLuRry says:

    @Scott: Serial is easiest bc. the firmware on the serial card (in a //e or //gs) or in the rom of the //c has a cool feature: You type one command from basic “in#2″ and the basic interpreter takes input from the serial port as if a human were typing on keys. So it’s easy to bootstrap the computer by having the other side “type” in a program. The best part is no hardware mod necessary to use it, so protects the computer from getting fried accidentally. Interesting idea about DD-WRT, but wouldn’t you also need a level shifter to convert between 3.3v and 5v between the two? I though about something similar using a dockstar…

  11. Scott says:

    @BLuRry – Yes you use the dd-wrt serial method with either 3.3 or 5v.

    Any extra steps (like 5v to 3v conversion) depend on which router you used. Some routers have proper serial ports on the motherboard. Some only have 3.3v TTY serial. Your mileage will vary.

    Sparkfun sell a 35v serial port adapter for about $20. If inclined you can build one for less (the 232 chip can even be had for free as a mfg. “sample”).

    See also: http://todbot.com/blog/2010/12/16/wifi-for-arduino-with-asus-wl-520gu/comment-page-1/

  12. Meeza says:

    Pretty cool – refresh rate reminds me of Prodigy back in the day.

  13. mjmahon says:

    The serial communication is not the speed bottleneck–it’s the graphic rendering.

    If you converted your block fonts into shape tables, Applesoft BASIC could “DRAW” or “XDRAW”
    (to erase) whole characters quite rapidly.

    -michael

  14. BLuRry says:

    It could be faster.

    Fast-forward to 3:22… the display is being sent over from the PC directly every time there’s a keystroke, and if you notice there are nearly full-size screenshots being sent over in less than a half-second each in some cases. It’s not going to be a movie player, but it is decent enough for a simple user interface.

  15. strider_mt2k says:

    That is so totally useful!
    That’s the kind of thing that would make an interesting display.

    Well done sir!

  16. me says:

    Cool but huuuuuuuuuuge waste of energy to leave that thing running

  17. Scott says:

    @me – You’re correct about the power draw for the monitor… probably 150-200 watts (Blu do you have a Kill-a-Watt?)

    Power draw of a Apple //c or most any classic 8-bit is appx 5 watts. Varies a little if the disk is active.

  18. iamdigitalman says:

    I need some help. everything was great until I get to run the lua script. I am using a USB serial adapter, so I set that in the script to /dev/ttyUSB0. When I run the lua program, it seems to hang at that point, but only when the adapter is plugged in. If I unplug it, I get nonstop cannot find /dev/ttyUSB0 messages. I have the rxtx library installed to run ADTPro, and it runs fabulously with my adapter.

    Any thoughts?

  19. Auxking says:

    This is such a waste of time. Why is this a main feature of the website with such a large link? Is it because one of the webmasters did it?

  20. Paul P says:

    Sadly an Apple IIc is too ‘new’ for me.

  21. hammy says:

    Paul P simply rereads a weather report from 1974 through an oboe

  22. hammy says:

    Or does the act of mentioning this publicly make it passe?

  23. echodelta says:

    I picked up last week at our Ham Fest a CGA analogue and 9 pin dsub jack transparency flat screen for projection. I also have a Apple 11e with a bad mem chip. I want to get rid of them. echodelta9somewhereyahoodotcom

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 93,583 other followers