Quick, what do “cloud compute engines” and goofy Raspberry Pi Internet of Things hacks have in common? Aside from all being parody-worthy buzzword-fests, they all involve administering remote headless (Linux) installations. It’s for exactly that reason that a new Ubuntu distribution flavor, Ubuntu (Snappy) Core, targets both the multi-bazillion-dollar Amazon Elastic Compute Cloud and the $55 BeagleBone Black.
If that combination seems unlikely to you, you’re not alone. But read on as we hope to make a little more sense of it all.
Hacker Hardware History
When rumors of the Raspberry Pi first hit the scene in 2011, it was marketed as the solution to the world’s computer literacy woes — at $25 per computer every child in the world could have one at their fingertips. (Nevermind the price of a keyboard, monitor, HDMI cable, mouse, power supply…) Nobody was thinking that we were in for an army of headless Linux servers, but that’s exactly what happened.
There were other small computers around; at the time, the BeagleBoard was the hotness on the single-board computer (SBC) front, but it was expensive enough that only the committed and nerdy were using them. We had one hooked up to a hard drive and an external DAC as the center of our stereo system at the time. The system was a little bit clumsy in that we occasionally had to haul out a screen and keyboard to perform maintenance on the thing, but overall it worked just fine as a multimedia hub.
Then the Raspberry Pi came out at around a fourth of the price of the BeagleBoard, and something funny happened. Instead of being the solution to the world’s computer needs, the Raspberry Pi ended up working its way into the same project-space that the Arduino had inhabited. Maybe it was the price point and form factor.
For example, one can’t deny that a Twittering Toilet is a necessity of the modern era. (Don’t blame us! It was the Zeitgeist.) On the other hand, getting your Arduino connected directly to the Internet was fairly difficult at the time — and so it’s no surprise that the Arduino senses flushes and communicates with a real computer that takes care of the Twittering.
The Raspberry Pi changed that. They’re cheap enough that you can dedicate one to the toilet without only minor pangs of guilt. So not long afterward, we have Raspberry Pis in toilets and chicken coops rather than in grade-school curricula.
These type of projects don’t use the device as a “computer” at all. Indeed, of the bazillions of Raspberry Pi projects we see here at Hackaday, how many of them have attached screens, keyboards, and mice? Very few, aside from the MAME builds and emulated Amigas stuffed into their own floppy drives.
Or have a look at the Beagle family. The BeagleBoard (2010) was a full-fledged single-board computer (SBC); you could plug in a monitor and keyboard. Then came the bare-bones (and slightly less expensive) version: the BeagleBone (2012). Signalling that it was meant for embedded applications rather than being a standalone computer, it had no video output and was a bit cheaper. It was a success.
Even less expensive, the BeagleBone Black (2013) again picked up an HDMI port, because, heck, why not? And as if to answer that question, the newest BeagleBone Green replaces the video out with some I2C peripheral connectors, reaffirming the device’s intent as an embedded computer rather than a mini-desktop.
So which do you want for your projects? A recent survey at Linux Gizmos asking their readership to rate their favorite single-board computers turned up no surprises on the popular-brand fronts: the Raspberry Pi is most popular, followed by the BeagleBone Black, followed by the Odroid offerings.
More interestingly, they also asked folks what applications they’re putting their SBCs toward. The results, in order:
- home automation
- special function servers
- home multimedia
- education
- robotics / vehicles
- data acquisition / control
- HMI / industrial
- “other”
- kiosks
This matches our gut feeling about what the average Hackaday hacker is interested in as well, so we’ll buy it.
Looking over the application list, how many of the project applications require a keyboard, monitor, and mouse? How many are traditional “computer” applications? Let’s say some fraction of “education” and probably all of “kiosks”. The rest of the applications are either standalone, run over the network, or are probably best served by a small LCD at most. In short, they’re embedded projects that benefit from the connectivity and ease of development that a (tiny) real computer brings to the table.
So What?
The point of this recent-history lesson is that if you make a computer cheap enough, it becomes an embedded device because nobody can turn down cheap Internet or USB connectivity, or a pleasant development environment, or even just interpreted programming languages with broad-ranging library support. If web-scraping is part of the device’s functionality, you’re not going to want to code that up on an 8-bit microcontroller. And so formerly-Arduino applications get amped up with Raspberry Pis, and our comments section overflows with people decrying “overkill”.
But more to the point, we’re seeing embedded Linux systems inside projects where it’s difficult to get at the HDMI port, or where you just don’t want to schlep a monitor. These are headless Linux systems that aren’t sitting in racks in some server room, but instead under the nightstand in your bedroom. And the humble hardware-hacker is looking at the remote administration of a headless networked server, which sounds like an entirely different job description.
Running Headless: The Software Catches Up
So if the Internet of Things is going to be an internet of headless Linux boxes, isn’t it time our software / operating systems caught up? No more of this GUI configuration menu crap — that’s for desktops that have the luxury of monitors. What you need, once you’ve got your Raspberry Pi or BeagleBone sitting deep inside some box somewhere, is quick and easy deployment and network-based remote administration of the tiny headless Linux server that lies within. And until you’ve got its web server or VNC up and running, that means spending some quality time with the console.
Queue the parallel development in the “cloud” world. The need to quickly spin up machines on servers has put a premium on ease of installation and updating of complete systems using simple and powerful command-line tools. In particular, the ability to create a system, save the configuration and installation details, and then replay them back into another instance has become a lot easier in the last five years. See containers and Docker and all that jazz.
Back in the embedded world, the Ubuntu Core documentation goes through an example of setting up a webcam on a BeagleBone Black and then replaying that configuration back, potentially to a completely new installation. If sharing your embedded appliance with the world means conveying all sorts of system and application configuration details, it’s great to have tools to make this easier.
Finally, installing security patches is the hot-topic bugbear of the IoT world right now. Nobody wants network-connected devices sitting around in their home network with buggy security implementations, yet so many IoT devices are flashed with firmware that’s inflexible and/or difficult to update. A major advantage of the convergence of “cloud” with embedded tech is that the devices get the benefit of easier, hands-off security update mechanisms.
Conclusion
The unlikely marriage of software tech from “cloud appliances” with our hacked (non-metaphorical) appliances is just starting, and we think it’s actually going to benefit us hardware types. Granted, it’s partly semantic, but we think that recognizing your Twittering Toilet as a headless Linux server could bring increases in reproducibility and security, as well as convenience.
What do you think? Was the Raspberry-Pi-as-miniature-desktop-computer idea doomed from the very start? Would increased ease of installation and administration push some of you microcontroller die-hards into the microcomputer camp? Are you a distributed Linux sysadmin because you have four BeagleBones in your home automation setup? Or have we been philosophizing too much in our beer?
[Photos of boards courtesy: BeagleBoard Foundation and Lucasbosch]
one thing that you touched on was that the low-cost rasp pi started to overlap the arduino space in terms of cost and in some ways, even size. why use an arduino and its ethernet/wireless/ip stuff when you can use a native, real tcp/ip stack with all that it includes.
that’s something that deserves more discussion. in particular, the attractiveness of one-chip ip solutions (like the $5 ESP wifi module that is now all the rage, and the wiznet ip-on-a-chip module) makes people jump to them and think that they have a nice neat little remote ip-reachable solution.
my problem with those on-chip ip solutions is: what’s the update ‘story’? how can these be patched if there are flaws or updates needed to things IP? similarly, is there a firewall or anything at all that lets you have more control over remote access. the small modules also rarely have any ssl or encryption ability on them, so security is not very easy, if even possible, at all.
otoh, the linux boards such as beaglebone and rasp pi take a pretty normal and up to date linux install, which includes kernel, user and network ‘worlds’. as long as you have something that traces up to, say, debian, you can get security patches and updates even if your direct upstream vendor goes away or stops doing support. being software-based lets you do that and being opensource lets you do that. the closed-source chip-based solutions can’t (usually) take updates and the vendors probably don’t care or are unable to.
look at cell phones! perfect example of a network-reachable embedded system that regularly gets abandoned by its vendor and quickly enough becomes a walking security violation. I have a phone that can’t get updates, the vendor (google) abandoned and it has huge numbers of holes that need fixing, but I understand how the phone market is and my phone is now abandoned. use at my own risk, only. and I hate that!
so, my point is about being able to use and design for/with components that have a good ‘update story’ in terms of support, upgradability (ro vs rw) and if there is an upstream that you can get access to, in case your direct parent ‘goes away’ for one reason or another. I shy away from the arduino IP solutions because I feel uneasy about the kinds of quality of IP I see in arduino land. otoh, I pretty much know and trust the linux world when it comes to things IP and I know my options are as full as they can be, if I pick a linux based IP endpoint instead of a controller-based IP endpoint.
I’ll use controllers across a private link that connect to a central IP node. that IP node will run linux. the remote sensors can run arduino-like controller setups. I could have chosen to make everything IP, but I don’t want to push the burden of managing so many instances (installs) of IP, and certainly not one per every sensor or data-source in an IoT home domain. yes, part of wanting to avoid having so many instances of IP is that you now have to keep them ALL updated and current. even if you can do that, technically, its a hassle and its more complication that may not really be necessary.
I’m playing with ESP8266’s on a pretty hardcore basis, and they do support remote updating of firmware :) there’s an example showing how to use it in the latest Arduino Core release
Aren’t there also compilers available for programming them and sdk’s available? I’m sure there are for the ESP.
yupsl, the easiest way to jump in would be to get the arduino 1.6.5+ ide, then find the esp8266 core for arduino online and add their board manager url to arduino’s board manager. then its just click and go! tons of examples too
Raspberry-pi’s data transfers rates are just too slow if you need to transfer even only moderate amount of data
The idea of “Raspberry-Pi-as-miniature-desktop-computer idea…” ran aground as soon as I realized I couldn’t run Eclipse on it.
The article mentioned Snappy in passing. Going through those features alone could be a series of articles.
The blog post at http://yrarc-splatter.blogspot.com/2015/07/eclipse-on-raspberry-pi-2.html seems to indicate it’s possible to run Eclipse on the Raspberry Pi 2.
Why does this article have a image of a IBM Blue Gene Q system? Unfortunately does not cover any of your buzz words. (Actually worked on the hardware and still have part of one on my desk as a memento)
@Peter, I’m sure it’s a generic datacenter pic to represent cloud computing…
I’m pretty sure it’s a BG/P system, btw.
I never believed the Raspberry Pi as a mini desktop really died, it just takes way longer for any changes to be made in a scholastic curriculum than the pace we’re used to in the hacker world. NYC just announced ambitious plans for teaching more comp sci in public schools, and I’ll bet with the right push we could see a lot more Pis in the hands of some really smart kids within the next generation.
Would increased ease of installation and administration push some of you microcontroller die-hards into the microcomputer camp? – I am slowly coming around that way, recently having replaced my goto “slap an atmega328 or an attiny85 into it” approach with a “slap an esp8266 into it” approach. The next step I feel will be microcomputers, but so far I don’t have any projects which _need_ Linux (choose the tool for the project, not the project for the tool).
Though, one thing I’ve always wanted to build is a distributed backup system (three separate low-power headless servers, each with 2-3 disks, all continuously monitoring each other for changes and mirroring with a 2 out of 3 vote for conflicts), so I’m definitely keeping an eye out for low-cost contenders with SATA ports (looking at you, banana/orange pi), $40 and a 3TB drive each for keeping three backups in three locations is a pretty good backup strategy for larger files imo.
I like having IP connectivity but what’s really happening underneath on that ESP module? we have api access to a layer and below that, nothing. that is kind of a show-stopper for me. I can hack with it, enjoy doing POC one-offs with it, but I would never use it in a shipping product.
a full software based ip stack is much more auditable and supportable.
for balance, again, I try to shoot for one instance of an IP station and the rest of the network of embedded domain devices are now proxied to, using something much less heavy than IP. controllers are the leaves and the IP linux box is the hub of that connected network. you can run all sorts of ip rules on that linux box and have full control over what goes where. and there is just 1 software-based system to manage; the rest are controllers. works for me.
If I recall correctly, there are official versions of the ESP8266 SDK with a fully open-source TCP/IP stack, though the layers underneath it are still closed.
It’s an interesting topology. I’m building three separate ones that could function independently of any others though (each with basic Linux OS), so they’re more like ‘disaster capsules’ than a real-world backup scenario :)
I was thinking about doing this, but ended up using https://github.com/yadayada/acd_cli . $60/year for unlimited storage is pretty hard to beat, I just use a local 5tb drive for fast access, and rsync to keep cloud drive in sync and have remote access. 1/10th the effort, and more reliable to boot.
My use case scenario is having a ‘doomsday box’ that has everything I need and a working OS to get up and running quickly :) no internet or anything
Pi as educational computer is marketing. It’s way to expensive. They say it is $35, but can you spend 350 and have 10 Rpi computers up and running? No, because of all the stuff they require along the Pi.
For education you are much better at buying a tablet which comes with integrated screen and keyboard for the money it takes to get a basic pi running(35+VAT + 10 for card, 10 for power supply). Or, for the price of 2, buy a cheap small laptop.
Not that I am complaining, i like my Pi’s for other stuff around the house.
one thing that the embedded PIs and bones missed was a simple, *standard* onboard display. I’m talking craft interface style display, like a hd44780 or the new, cooler oled 128×64’s that are now all the rage.
the thing is, when you boot up, you need to know the ip addr, maybe change the ip (if you are moving the device). you might need some other functions that a local lcd and some buttons might be able to do.
going truly headless is complex and hard. going fully headed is easy but clumsy and stupid for small embedded systems. there is a need for a middle ground where even a one line or two line text display and a small arrow pad would have done WONDERS to enable the small boards to be remotely deployed and even upgraded.
I hope the next single board system that catches on has at least some kind of standard (ie, everyone gets it, so you can always count on having it) display and some minimal button input. hell, even an IR sensor would do; and you could have a few keymaps for some standard ir remotes. I’ve done that in the past as a way to have a ‘buttonless’ system that still needs some kind of config ability.
I think your point is considered in the educational side. Essentially, the teaching seems to be similar to that of imaging servers: Set 1 up headed and configure it properly, then image and deploy. In an actual embedded environment, a business would be more likely to swap out SD cards running the system or to remote into a system.
Frankly, the only troubl I have with my headless Pi is when the power goes out and I forget to rest my ireless bridge (had one laying around so $0 was cheaper than adding wi-fi to the pi. Also, I have Tomato firmware on my router and it’s easy as pie (pun intended) to always have DHCP hand out the same IP address to the RPi via DHCP. That way, I could still take it to another network and not have to connect a keyboard/mouse/monitor just to reconfigure network settings.
The one button I would like to have is a “kill” switch that triggers the OS to shut down for those instances when I need to power down and I’m too lazy to even remote in and issue the commands (even though I have ServerAuditor right on my phone).
this is one of the primary motivators for Shields/Hats/etc for each dev board. You can make a display/input board to add if you want. My RPi is tucked in by the Apple TV and Blueray player, so I can always snag an HDMI from one of them if I need to go headed.
I suppose a display and some buttons would be nice, but the Pi foundation tries to keep things at the $35 price point. To get those extras on, you’ll have to cut corners elsewhere to keep at the price point. That or raise the price. There will always be folks like me that want a truly embedded system with no display, but with an option to add something on as needed. I’ll put my $$ in that hardware and add the extras on when I need them (I already have an array of displays, buttons and even an Ir receiver and remote from various projects/kits/recovery from broken stuff).
DISCLAIMER: I have worked for years supporting servers at our clients that are usually just another 1U somewhere in the rack (or more frequently are a VM in one of the clusters). WQorking remotely with windows systems no less is easy for me (and arguably, Linux is more more remote/embedded friendly than say server 2000 or even server 2003).
I’d rather see the Pi software have the same capabilities as webcams, where you can run a simple program and it will find all the Pi’s on your network, and return their MAC and IP addresses. Simply plug it in and run the software on another machine to find it’s address. Double click to connect with a ssh client would be nice too.
well, i made this which can be made for the pi as well
http://www.electrobob.com/ser-show/
The lcd is about $5-6 on ebay and you need another 2 for the micro. This is to skip altering the PI in any way, otherwise the LCD can go directly on the SPI, but you need new kernel i guess.
“Are you a distributed Linux sysadmin because you have four BeagleBones in your home automation setup?”
Are they distributed and treated as a linked system of Linux servers and devices? Say, a gateway+firewall combo, a webserver, a database, and let’s throw the last one to doing backup and maybe IDS. In that case, yeah, you are a sysadmin. Distributed? Depends on how you log in to each system; ftp, remote shell, or other plain text would get a no from me, but good ssl/vpn practices with public key on the remote devices to ensure that you could only log in as a user and never as root then yes you are doing distributed right.
Are the Pis useful as headless devices? Yeah, same as those of us learning Linux in the old days used old x86 PCs as various headless devices. Sure, those old PCs might have built in video, or even a video card; we just ignored that part. And at the price point, there is little to beat them. Sure, one could use a Pi Compute and a usb controller and an ethernet jack (which would probably end up costing more).
As for Pi replacing ‘Duinos . . . yeah, it’s overkill. For a dev-board price of $25, though, would you like to add ethernet to an Arduino or just use a Raspberry Pi? For the new developers, which offers the better price point?
There is also the Scaleway ARM cloud provider, which is based on Marvell ARMv7 cores.
Its interesting reading the comments, I tend to use micro-controllers due to the plethora of hardware IO’s available for me to use especially when some of my projects use more IO’s than an Uno [equivalent] can handle. I then link all these via IP on a wired ethernet to do basic reporting back to a central node that then collates the data and makes other things happen based upon what it knows. I also have a tendency to power my devices using [true, 48V 802.11af] PoE to power the devices to centralize things as much as possible, although I do keep them on their own VLAN for good measure. Honestly I go with what @Jarek said, use the right tool for the job, which at this point I have no real need for RPi’s, mainly due to the lack of IO on it, the BeagleBones and other however do not have this issue and I am intrigued by them. I am tempted to play with XBEE or ZigBee systems and then link them back to something akin to the BeagleBone, but IP is just so common place these days, that using it for reporting, monitoring and management makes more sense to me, then using a central management node. But as has been pointed out it makes it hard for OTA updates, which SSH makes quite easy on the micro-computers
Having said that I tend only to build things where there is no commercial product and it is not linked directly to the property, as I do not want to have to replace things or remove things if/when I move. I will use commercial things for the “built-in” stuff and only go homebrew when I can pick it up and take it with me.
I’ve grown to like Linux terminal through both Rpi2 and cloud VMs. On my Pi I’ve stayed away from Snappy as I understand it doesn’t run software as easily as the trusty edition, the problem being trusting software sources. I was just thinking similar thoughts on my latest project, http://deepspace.xyz/IOTSolarGarden/
for remote admin of headless things running Linux, install WebAdmin. It allows you to do command line type things like install packages, edit Cron jobs, run updates, monitor tasks and memory and more through a web browser GUI with no setup on the headless side. I’ve used it before on PogoPlugs and RasPi’s.
The Tweeting toilet didn’t go through an intermediate computer; it used a WIZnet (hurrrrr) module to talk directly to the internet. The picture of the computer there was because I was trying to, uh, flush out some bugs.
Been using two Arduino megas for data collection on my home automation system which is then polled by a Debian machine for storing in MySQL and doing all the heavy lifting and have started looking into the ESP modules for additional sensors, etc.
Got four Raspberry Pi’s as well.
2x OpenELEC XBMC machines
1x Whole house audio control with the cirrus logic sound card
1x for doorbell camera http://nl.com.au/wordpress/?p=225
I am in embedded linux guy. I fell in love with the Raspberry pi when it first hit the market. I was on the first mailing list. I recently worked in a job where I got to use AWS on a daily basis. I think my on the ground experience with headless linux boxes in the home world helped me grasp the cloud platform quite well. I think all embedded guys have a great advantage in the cloud computing.
Having been using the Beaglebone black since day one release to the public. I’d have to say that I do( sort of ) agree with much of the article. Some of the comments I’ve read I’d also at least understand, but do not necessarily agree with. In the context of security for example. The system is only as secure as the user using the system. This applies to any system. Be it bare metal, running C or an RTOS. Or an embedded Linux system such as the Beaglebone black.
Another thing that I have not seen discussed yet however. Embedded Linux SBCs such as these can, and often do offer an advantage in some, or perhaps many situations. Indeed, I’ve built a CANBUS monitoring device, that was fairly simple to code using socketCAN – With a Beaglebone black + Logic supply serial / can cape. What’s more, this same board was used as a websocket server, to put this data out over our network to a web browser. For me, this was a very cool project, and an excellent learning experience. However, there are some limitations when using an embedded Linux system as such. Largely I’m talking about application latency due to using an OS in the first place. Secondly, we need to keep in mind that the Cortex A8 processor, is a general purpose application processor( in many ways similar to an x86 / x86-64 processor – in this context – only better ;) ). Which means that the Arduino for specific tasks, might actually be faster. Please, do not get me wrong, I am by far *not* an Arduino groupie. In fact, I tend to lean towards TI, or Freescale for my bare metal needs. But I feel that this is an important point.
In the case of the Beaglebone SBC’s this is mostly a moot point. As the AM335x processors have 2 on die PRU’s( Programmable Real-time Unit ), that is roughly an equivalent to a Cortex M3 processor @200Mhz. Which also shares fabric with the main processor core. But can run completely separate from the main core(bare metal). So effectively, you’re getting a rPI( with perhaps less capable graphics ), plus two on die Arduino’s++.
So more succinct and In other words, Apples, Oranges, and Lemons . . .