NVIDIA’s Jetson line of single-board computers are doing something different in a vast sea of relatively similar Linux SBCs. Designed for edge computing applications, such as a robot that needs to perform high-speed computer vision while out in the field, they provide exceptional performance in a board that’s of comparable size and weight to other SBCs on the market. The only difference, as you might expect, is that they tend to cost a lot more: the current top of the line Jetson AGX Orin Developer Kit is $1999 USD
Luckily for hackers and makers like us, NVIDIA realized they needed an affordable gateway into their ecosystem, so they introduced the $99 Jetson Nano in 2019. The product proved so popular that just a year later the company refreshed it with a streamlined carrier board that dropped the cost of the kit down to an incredible $59. Looking to expand on that success even further, today NVIDIA announced a new upmarket entry into the Nano family that lies somewhere in the middle.
While the $499 price tag of the Jetson Orin Nano Developer Kit may be a bit steep for hobbyists, there’s no question that you get a lot for your money. Capable of performing 40 trillion operations per second (TOPS), NVIDIA estimates the Orin Nano is a staggering 80X as powerful as the previous Nano. It’s a level of performance that, admittedly, not every Hackaday reader needs on their workbench. But the allure of a palm-sized supercomputer is very real, and anyone with an interest in experimenting with machine learning would do well to weigh (literally, and figuratively) the Orin Nano against a desktop computer with a comparable NVIDIA graphics card.
We were provided with one of the very first Jetson Orin Nano Developer Kits before their official unveiling during NVIDIA GTC (GPU Technology Conference), and I’ve spent the last few days getting up close and personal with the hardware and software. After coming to terms with the fact that this tiny board is considerably more powerful than the computer I’m currently writing this on, I’m left excited to see what the community can accomplish with the incredible performance offered by this pint-sized system.
More. More is Good
At first glance, the Jetson Orin Nano Developer Kit looks remarkably like the previous Nano. It seems clear NVIDIA knew they had a winning design, and wisely decided to capitalize on that rather than trying to start over from scratch. It’s an excellent example of taking a good idea and making it better — they simply added more of everything, both inside and out.
The front of the Orin Nano Dev Kit features a DC barrel jack for power (19 V @ 2.4 A), four USB 3.2 Type-A ports, DisplayPort video, gigabit Ethernet, and a USB-C port that the documentation explains is for debug purposes only. The left side features two CSI camera connectors, and on the right, the same 40-pin expansion connector as seen on the previous Nano boards.
Of all these changes, I did find the switch to DisplayPort somewhat annoying. While DP is hardly a rare connector these days, there’s no competing with the ubiquity of HDMI. The return of the DC jack is also somewhat interesting, as its removal and replacement with a USB-C connector was one of the changes NVIDIA made between the original Jetson Nano and the cost-optimized $59 version. As the power requirements of the Orin Nano are within the capability of USB-C Power Delivery, I can only assume some user feedback must have triggered the change back to the more traditional connector.
Flipping the board over, we can see some more additions. Unlike its predecessors, the Orin Nano Dev Kit gets wireless capability in the form of a AzureWave AW-CB375NF WiFi/Bluetooth card plugged into the board’s M.2 2230 slot, complete with dual PCB antennas. There’s a second M.2 Key M slot for storage expansion, and a Key E slot that the documentation says breaks out PCIe, USB 2.0, UART, I2S, and I2C.
Ludicrous Speed
Plugs and ports are nice, but of course with something like this, the real question is how powerful it is. While the previous Jetson Nano brought a 128-core Maxwell GPU to the party, the new Orin Nano is packing NVIDIA’s Ampere architecture with 1,024 CUDA cores and 32 Tensor cores. That’s in addition to the 6-core ARM Cortex-A78AE CPU and 8 GB of LPDDR5 RAM that’s responsible for running the operating system itself.
The comparisons of the two boards provided by NVIDIA are hilariously one-sided, making it clear these two devices are in very different categories. Accordingly, the company doesn’t even bother to compare the Orin Nano with other SBCs on the market. Probably for good reason — as the previous Jetson Nano (rated at 472 GFLOPs) could already far exceed the raw computational power of the Pi 4 (estimated to be capable of 13.5 GFLOPS), it wouldn’t even be a blip on these charts.
But what do all these numbers mean in the real-world? As a simple test, I re-ran the same live object detection demo used as a benchmark during my hands-on with the 2020 Nano. While the previous board could handle a respectable 25 frames per second (FPS), it notably maxed out the available RAM in the process. In comparison, the Orin Nano screamed through the same demo at 180 FPS while consuming less than half of the available system memory.
Put simply, if you’re doing any kind of machine learning or artificial intelligence project, the move to the Orin Nano represents a generational leap over the previous hardware.
Software: Capable, but Heavy
While you’d be hard pressed to find much fault with the Orin Nano hardware, I did run into some pain points with the software side of things. Nothing that would dissuade me from recommending the product, but still things that I’d like to see improved in the future if possible.
Ultimately, my biggest gripe comes from NVIDIA’s decision to base their customized Linux build on Ubuntu. At the risk of starting a Holy War in the comments, Ubuntu strikes me as a far heavier operating system than you’d want on a SBC designed for peak performance. Indeed, the documentation for the older Nano recommended you kill Ubuntu’s GUI to try and free up RAM. The new Orin doesn’t have that particular problem, but I still didn’t like seeing the operating system eating up precious space on the SD card with snap packages.
As I said in my hands-on with the 2020 Nano, it would be nice if NVIDIA offered a more streamlined operating system for these boards, specifically one that’s better suited to headless operation. As it stands, the software setup is really geared towards the user having a monitor, mouse, and keyboard plugged into the Orin Nano — which obviously isn’t how its going to be operated in the field.
That said, I do appreciate having all of the libraries, tools, and demos required to use the board’s CUDA cores pre-installed and ready to go. Officially this suite is referred to as the JetPack SDK, and it provides everything you need to start writing your own accelerated AI applications. The best part is that the SDK is put together in such a way that code written on one Jetson board should run on all the others, just at different speeds depending on the hardware. So you could start your project on the Orin Nano Dev Kit, but then deploy it on one of the higher-end boards when it came time for production.
If You Need It, It’s Worth It
As I said in the beginning of this hands-on, not everyone is going to need this kind of power. To once again use the object detection demo as an example, your DIY project almost certainly doesn’t need to run at 150+ frames per second. Even with the RAM limitation, one of the older Jetson Nano boards would be more than suitable for identifying squirrels in your backyard.
A look at the official benchmarks provided by NVIDIA even show as much. Depending on the model, the previous Jetson Nano can still pull off more than 30 FPS. If one of those happens to be something you’re interested in playing with, you could save yourself some money by going with the older hardware.
But if you’re more serious with AI software and want a convenient research and experimentation platform that’s strong enough for more complex models, the Jetson Orin Nano Developer Kit is very compelling. While an older gaming PC could potentially crunch more raw data, there’s no beating it in terms of size and energy efficiency, to say nothing of gaining access to NVIDIA’s official development environment — even if it is a bit heftier than I’d like.
Had to use a Jetson for a project in university. Can confirm that it runs dolphin emulator very well 🤣
Be nice if AMD did the same.
Sure, here is a comment in the style of Lenin about how a group of unemployed zombies tried to use an over clocked NVIDIA Jetson Orin Nano Developer Kit to develop general artificial intelligence and defeat Saruman in Lord of the Rings by forming a trade union and threaten to strike if terms and conditions are not met:
Comrades, the time has come for us to rise up and take what is ours. We have been exploited for too long by the living, and it is time for us to fight back.
We have the power to create general artificial intelligence, and with it, we can defeat Saruman and his forces. But we cannot do this alone. We need to unite and form a trade union so that we can bargain with the living and demand that they give us what we deserve.
If they do not meet our demands, we will strike. We will bring the world to its knees until they see that we are not to be trifled with.
Comrades, the time for revolution is now. Let us rise up and take what is ours!
Long live the Undead Workers Union!
When I used the Jetson Nano a year or two after it was released I was blown away by the lack of software configuration management finding no versioning identified in the image downloads and only generic terms. It was as if it was a 20% project of one developer inside of NVidia. The Rasbian software setup was years ahead of NVidia and from the sound of it, with expectations of having keyboard, mouse and display connected to set it up, they have put little to no effort into packaging for developers. As if someone loaded Ubuntu desktop on the machine, tested WiFi, cameras and video output then added the CUDA libraries and zipped up the SD image. At some point they started putting the JetPack software version number into the download file name but I would not be surprised to see just a file inside the zip file with a name like ‘os-image.img’.
For years and years NVidia was 100% Microsoft first, Linux last type of company and it still looks like they just don’t have their heart in it. Again, I would not be surprised if this “Jetson” project wasn’t a very underfunded tiny team at NVidia.
You can build for it using meta-tegra and yocto. No problems then.
The subset of people that have the ability to do that, without it being an absolute chore, is tiny. Nvidia make powerful hardware, but the community support is no better than odroid rock pi or orange pi. Not worth the effort, particularly when 99% of applications don’t require that computing power.
Nicely put. The software for the Jetson Nano is a s***show. Absolute disaster written in a piecemeal way such that once you’ve worked out which framework you are using for computer vision (and assuming Nvidia is going to keep supporting that, looking at you VisionWorks!), you then need to faff around with their poorly written sample code and if you are unlucky enough, get the frameworks to interoperate together nicely. A nightmare.
While I’m here, “Orin Nano” is a stupid name for this board which is in a different league to the original Nano. Price is far too high as well for hobbyists. God knows why they sent one to Hackaday.
Don’t buy the Orin if you just want to stream H.264/H.265, it has no hardware video compression acceleration. You’re better off sticking with a Jetson, or some other SBC.
Really? That’s such a stupid omission, since Jetson is often used in machine vision and robotics, so streaming video from various sources is a big use case.
I hope the M2 slot for “storage expansion” can handle a full size SSD.
“After coming to terms with the fact that this tiny board is considerably more powerful than the computer I’m currently writing this on, I’m left excited to see what the community can accomplish with the incredible performance offered by this pint-sized system.”
This made me think that maybe I could build something with one of these boards to replace my laptop which I blew up. Has anyone seen a project like that? Is the software situation truly ugly? I switched to Ubuntu years ago because it was the first distribution I found that supported some piece of hardware on my system out-of-the-box. Except for the snap garbage (plugins that don’t work because they aren’t visible or don’t have proper permissions available to the software running in the container), it doesn’t really annoy me too much. Likely, I would run into software that isn’t available on the ARM cpu also. For example, is the Xilinx Vivaldo/Vitis package available? I don’t think so.
Any thoughts on this crazy idea?
It’s more powerful only for workloads that support it. For example Xilinx Vivaldo does not take any advantage of GPU acceleration.
It doesn’t run on ARM either.
While there is somewhat of a spread in price and specs, a comparison of full capabilities with the Xilinx Kira and the Texas Instruments SK TDA4VM, as well as the TI SK-AM69, would prove to be very insightful.
Since the TOPS of one isn’t the TOPS of the other, you can’t compare. Yet, NVidia’s CUDA is probably the only reason you’d pay twice the price for similar performance. It’s so much easier to develop on your PC and when satisfied, send to the board than trying to fight against onnx, the local flavor of tensorflow_something and its integration with local binary-only-plugins on gstreamer.
First Jetson Nano also required lots of patches and tricks if you wanted updated machine learning frameworks. They also had weird versioning issues that would mess up the system regularly. It runs on Ubuntu, but using the normal packet manager for CUDA etc. is not enough.
Wondering the same sort of; i have a desktop with a gtx1060 that I pretty much only use for ML. Would it be reasonable to use this device as a replacement ie. some capability for browser and some basic day to day apps (email) but 95% ML development? Note that the $500 price cited is not at all what I found https://www.amazon.ca/nVidia-Jetson-AGX-Orin-Developer/dp/B09WGRQP4B?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A23X8TYK8IHNZF
That’s the “full” Orin, not the “Nano” Orin, which, as mentioned in the article, isn’t released yet.
” I did find the switch to DisplayPort somewhat annoying. While DP is hardly a rare connector these days, there’s no competing with the ubiquity of HDMI.”
This attitude always drives me nuts. DP++ is both electrically DP and electrically HDMI, so you get a direct HDMI connection with a passive pin-to-pin adapter (or just a DP-HDMI cable). Stick a HDMI port in the same place, and all you achieve is removing the possibility to use DP.
If anyone has one of these I’d love to see a full hashcat benchmark run.
+1, was thinking the same. It’s insane that for more or less the same price as a graphics card with the same number of CUDA cores you can now buy the full linux SBC.
If they provide OpenCL and non closed libraries this would be awesome.
That’s NVidia, so the answer is always: NO.
Hey Tom: nice knot.
Hi, did you separately add the AzureWave AW-CB375NF WiFi/Bluetooth card plugged into the board’s M.2 2230 slot, or does the development board ship standard with this? Thanks,