It wasn’t too long ago that one could conjecture that most hackers are not avid video game players. We spend most of our free time taking things apart, tinkering with microcontrollers and reading the latest [Jenny List] article on Hackaday.com. When we do think of video games, our neurons generally fire in the direction of emulating a console on a single board computer, such as a Raspberry Pi or a Beaglebone. Or even emulating the actual console processor on an FPGA. Rarely do we venture off into 3D programs meant to make modern video games. If we can’t export an .STL with it, we’re not interested. It’s just not our bag.
Oculus Rift changed this. The VR headset was originally invented for 3D video games, but quickly became a darling to hackers the world over. Virtual Reality technology is far bigger than just video games, and brings opportunity to many fields such as real estate, construction, product visualization, education, social interaction… the list goes on and on.
The Oculus team got together with the folks over at Unity in the early days to make it easy for video game makers to make content for the Rift. Unity is a game engine designed with a shallow learning curve and is available for free for non-commercial use. The Oculus Rift can be integrated into a Unity environment with the check of a setting and importing a small package, available on the Oculus site. This makes it easy for anyone interested in VR technology to get a Rift and start pumping out content.
Hackers have taken things a step further and have written scripts that allow Unity to communicate with an Arduino. VR is fun. But VR plus physical reality is just down right exciting! In this article, we’re going to walk you through setting up your Oculus Rift and Unity game engine to communicate with the outside world via an Arduino.
Off the Shelf Options
If you head over to the Unity Asset page and run a search for Arduino, you get a few options. Sadly, searches for Raspberry Pi do not yield any fruit. There are a few generic serial communication options such as Simple Serial and SD Serial, but these options are not free and do not, at face value, appear to be well supported. Unidino looks promising, but it’s thirty bucks and there’s not much activity on the forum. The obvious choice to play around with on a rainy day is ARDunity. There’s a free version that still has plenty of capability to experiment with, and it’s well supported and documented. It’s written in more of a WYSIWYG style that can be off-putting to coders, but it will have to suffice until someone bangs out more advanced version.
Getting Started
We’re assuming that you already have Arduino and Oculus setup on your PC. If you don’t have an Oculus Rift, we recommend to go to the Oculus site and install the software anyway. It will allow you to test the Unity/Arduino communication through the Oculus run-time even if you don’t have the hardware. This way when you do get a headset, you won’t have to do anything. Just execute the .exe and you’ll see your work in VR. Note that they used to prevent the software from installing on computers that did not meet the minimum requirements. Oculus has since toned down this nuisance, and now allows the software to be installed on most computers, including laptops!
Setting up Unity is straight forward – simply go to the site and download the installer. The latest version at the 
time of this article is 5.6.1. You’ll want to grab the personal version – it’s about five Gigabytes, so give it some time to download. Once installed, head over to the Oculus site and grab the Unity tools import package.
Open Unity and open a new project. Then head back to the Unity Asset store and install the ARDunity Basic Import package. This will open Unity, and you should see a tab called Asset Store. Click on the tab and then import the package. Then go to Edit–>Project Settings–>Player and set the API compatibility level to .NET 2.0. This will clear the error you see in the bottom of the screen. Then under Other Settings, be sure to select the Virtual Reality Supported option.
Now import the Oculus package by going to Asset–>Import Package–>Custom Package and point towards the Oculus package you downloaded earlier. Restart the Unity program if needed.
Putting It All Together
At this point, everything should be set to get Unity talking to your Arduino through the Oculus Rift. Put an LED on D2 and note your comm port. From the Project tab (lower left), expand the folders ARDunity–>Examples–>LED–>Digital. Double click the last example – ReactingTrigger(DigitalLED).unity.
That will load the example. You should now see two cubes in the view window, and a new folder called ARDunity under the hierarchy tab on the top left. Click on ARDunity and you’ll see the inspector tab (far right) change to include a button that says “Export Sketch”. Hit the button and it will export the Arduino sketch to a place of
your choosing. The .ccp and .h files will be generated automatically, along with some other dependencies. I shouldn’t have to tell you to have a look around, but don’t forget to compile and upload it after you’ve had your fill.
Just below the Export Sketch button will be the Comm Serial (Script) window. Search for and select the comm port for you Arduino. After you find your comm port, hit the little play button on top of the main scene window. Once you do this, you’ll see a “Connect” button appear where the Export Sketch was. Go ahead and connect. Now click on the Scene tab above the main viewing window. This will allow you to manipulate the cubes. Go ahead and grab the white cube and run it into the green one. If you did everything right, you’ll see your LED light when the two cubes collide.
The gears in your noggin should be turning right about now… if you can toggle IO from within a game engine; there is some seriously cool stuff you can do! But we’ve only scratched the surface. Let’s get this working in Virtual Reality!
Enter the Rift
In order to enter the virtual world, you need a couple of things. One is a character controller and the other is something to walk on. Hit the play button again if you haven’t already. This will disconnect everything and allow you to edit the world. Go to Game Object–>3D Object–>Plane and adjust the plane so the two cubes are hovering above it. Then un-collapse the ARDuino and expand the OVR folder to OVR–>Prefabs. Select the OVRPlayerController.prefab and drag it into the main hierarchy in the upper left. You’ll see the player controller appear in the main scene window. Drag it away from the cubes, and then under the inspector tab on the right hand side, de-select the Use Profile Data option in the OVR Player Controller (Script) section.
Now, select ARDuino under the hierarchy tab, press play and then connect. If you have an Oculus Rift connected, you should be able to put it on and run into the green cube and light the LED on the Arduino. If you don’t have one, just use the arrow keys to do the same.
Conclusion
So where can you go with this? The biggest thing that jumps out to us is haptic feedback. Imagine instead of a cube, you have a wall or table or something of that nature. Have the Arduino trigger some type of feedback when you touch or bump into the object.
Dig into the code and let us know of any bugs or improvements. Be sure to follow some other examples as well, and check out the video below for a demonstration. If you’ve done anything cool with Unity, show us in the comments.
“The gears in your noggin should be turning right about now… if you can toggle IO from within a game engine; there is some seriously cool stuff you can do!”
I can RULE THE WORLD! *mad cackling*
But seriously this is nice. You may even be able to do design and simulation in VR in the future.
I don’t see why you can’t do it now.
Lots of modem games use complex physics to figure out reflections and shadow. Accurate newtonian physics has been around for at least a decade. Your hardware may not have the chops to render it in real time, or every frame but i don’t see why it can’t be done now.
You’ll probably have to roll your own FEM for the engine but everything else is basically there.
How about Blender instead? Its free, its open!
Blender is better suited to be 3D modeling software. And Blender Foundation should focus on that instead of adding different, not very useful features. For example I’d rather cut and glue film stock by hand than use Blender as video editor…
Besides, how many games on Steam use Blender?
I think the concept of Blender Game is interesting, IMO they should do something to make it better as well.
blender isn’t a game engine, you’d use blender to make 3d assets that can then be exported to unity. also unity is free.
No, its not, its closed source.It is a game engine, it has “Blender Engine” built in.
It’s been a long time since I ran into someone pretending “free as in freedom” is the default meaning of the word when it comes to software.
Its definitely been decades since I encountered somebody that didn’t know free software is a thing. Nobody says, “oh you mean like shareware?” anymore.
It may be not for you. It is for me. Your defaults aren’t my defaults and vice versa.
Peace and respect. You exist. I exist.
I use Blender, but seldom. I’ve been using Unity for years, and it’s my go-to program for a lot of projects. Hands down, Unity is well worth learning. Everything from reasonably small apps to AR to demonstrations of a concept all the way to massive online multi-player games, it’ll do it.
Blender just needs a little more work and you’ll be able to 3D model while in VR. With Unity you need to create the VR environment first and then use it. Why not create and use the environment at the same time? Just alt tab into reality to hook stuff up to a python script. I’d rather spend time improving Blender than learning Unity (something I can’t modify) so I might be bias.
Meh…
https://wiki.unrealengine.com/Building_On_Linux
https://memecrunch.com/meme/99OTE/meh/image.jpg?w=625&c=1
FWIW, the Unity editor for Linux is available here (though you don’t get to compile it yourself):
https://forum.unity3d.com/threads/unity-on-linux-release-notes-and-known-issues.350256/
Unity looks great, right? So many features, just download it and use the simple UI and start making games in 3D. I did. Then thought why not get this upgrade package for $35 per month that has all these things you can download as needed? signed up. Then find my day job changes, and there is no time for using Unity or any 3D programming for now. Hi Unity you know my subscription? lets cancel it. Hi user, yes you can cancel after you pay for the full 12 months as you agreed. thank you. No, you don’t understand, I said cancel it now, you charged me 2 months and I didn’t even login or use it, cancel it. Yes hi user, as you were told in the last email, you can cancel after you have paid the full 12 months. it is the law and it cannot be changed, it is forbidden to get out of this contract for any reason. me: Well, I can call the bank and tell them to cancel it? No user, you are liable for paying the full 12 months it is a contract you are bound to and there is no way you can change it.
I said this is the US, not Europe, your socialist rules don’t apply here. Hello user, after reviewing your case you will need to show us your full, original bank statements with account numbers, all income records with account #s and other original financial statements (copies not accepted) proving your severe financial hardship and we will review and possibly make a special exception. However as stated you will not get any refund, we do not issue refunds under any circumstances.
Its still being debated. I was able to make a 3D program(s) that run on my phone, but now wouldn’t touch Unity, don’t know who the competition is even. But when I go back to 3D coding, it will be with not(Unity)
Unreal is the competition, and I believe they have VR support.
> I said this is the US, not Europe, your socialist rules don’t apply here.
that might be the reason: The stuff you’re sniffing has put you into a different reality. Try reading what you subscribe to next time? And: McCarthy is dead, today even capitalists can be assholes.
Is this kind of reply necessary?
Yes. Calling out stupidity is necessary. Read the f**king contract before you sign it.
You sign up for 12 months, you pay for 12 months. The payments being monthly is not relevant.
Actually, no, if I’m paying for a monthly service I’ll cancel it at any point, and if you think I violated a contract you can sue me. If you simply refuse to stop charging the card, I’ll have the bank take that money back. And they will.
And, you don’t normally “sign a contract” in a way that lets them add random conditions. You normally simply click a checkbox, and all that does is establish that they notified you of various “terms of service” that give them reasons to cancel your service. It isn’t an “express agreement” that they can hold you to, it just covers their ass.
And even if you sign a contract, you can still cancel when it is a monthly paid service because if you stop using it, you’re not getting anything in return for your money, and even with a contract that isn’t allowed.
So, no, no, and also no.
Iirc Oculus Rift DK2s are quite cheap and still work with a little fiddling. By cheap I mean £100 second hand instead of £500 for the CV1 new.
MQTT, openhab, Gear vr. remote control of home automation devices as well as an interface that could be themed. Add viewing of home camera system and then figure out a way to add the setup to the Vive home app as an environment. I would really like to be able to do what I normally have to take my headset off for. Have a screen pop up when someone rings the doorbell, or motion is detected on the cameras. I already modeled my home and added the geo data from google. Now I can add my physical lights to my digital render. At the very least, I can test the layout of possible interactive points in my home for better physical UI.
Hi,
I would like to know: is this article sponsored in any way by Unity ?
Thanks for letting me know.
No. We don’t do sponsored content. When we do hardware reviews, we clearly state the review hardware was provided.
Going further, I’d like to extend my appreciation to you for this comment. It is impossible to prove anything is not sponsored in any way, and this comment gives us the opportunity to clarify that we do not publish sponsored content in any way, shape or form.
Ok thanks for this clear answer.
Free and open source…..http://torque3d.org/
Have you used Torque? Because, uh…
Is there even a SINGLE Unity game on the market without performance issues? Im not even going to mention the maturity of the tools and codebase.
Infinifactory starts up pretty fast.
Please define “without performance issues.”
GC resulting in shuttering for example
Ooh, you liars.
Judging by the comments, I’m NOT “the only one not playing with Unity”.
what little i know about unity i know from ksp modding. but when i want to do some game dev i dont pick up the first rent an engine that i see. i just go work on my own.
Is your aim to make games or to /do game dev/ itself? Because rolling your own everything sucks when you just want to express something. Unity is impressive for its usability.
I say that as someone who has spent a long time writing game-specific opengl engines; you get full control and you get to stretch your legs programming, but it’s a crap way to make a typical game.
There is however peace to be found in writing your own system for everything, because you know how it will behave and you will have a good idea how to achieve everything that your engine can do.
Godot is really neat for prototyping. It’s under pretty active development and just gets better and better.
Exactly. If you aren’t out there mining the metals to forge your computer yourself, you aren’t a real programmer. Work on your own stuff people!
I tried many versions os Unity Editor on Linux (Ubuntu 16.04 LTS). All tries pull one core to 100% and transform my notebook in a toster. The only workaround was using cpulimit, but had severe implications: too slow/chop or still hot. I didn’t find a good threshold.
Maybe here is the best place to find some HACK to fix it. Can anyone throw a tip?
If you really want to do VR or AR application with support for Arduino, Occulus, Leap Motion, Kinect, and many more thing using a free (as in freedom and as in free beer) software, you should definitly look at OpenSpace3D : http://www.openspace3d.com/
There is also a lot of videos on their youtube showing exemple of how to move a robotic arm with an arduino and a Myo from OS3D, how to use marker recognition to quickly make an AR application, …
The langage use by OS3D (Scol) is a bit weird but you can use a block-programming interface to quickly prototype your App.
At $35pm, “I’m” probably not the only one not using Unity…. That’s a bit of a bigoted statement…
As has been said here, there are plenty of other options, most being free, which is the real deal for a tinkerer / hacker / maker, on a limited budget.
I’m not sure why everyone seems to think Unity is not free. It is, unless you are making a bunch of money with it. You don’t need the things you get by paying for it, the free version is just fine. There are also a lot of third party stuff for it, path finding and stuff like that, that you can use for free.
I like Unity. A simple AR app that uses markers is maybe two hours work, start to finish. A really simple AR app takes maybe ten minutes. A more polished one done to a ‘public release’ standard will take a bit longer, of course, but that’s going to be true for the ‘free’ alternatives too. Want to write a massive online game with easy networking, not a problem.
Mix AR with an Ardunio? No problem, add another few minutes. Need to compile for OS, Windows and Android with the same code? Done. Need to run it on Linux? Done, same code base.
I’ve also never run into a big performance issue. We always want more performance, of course, that’s just natural. But it’s never even been a factor for me with what I’ve done with Unity.
I’m not trying to talk anyone into using it, use what you’re comfortable with. I just don’t want anyone just getting started to get turned off by all the ‘not free’ talk and not even give it a look.
I’m lazy, I want to get the job done as fast and easy and cheaply as I can. For me, that means Unity, but of course your mileage may vary.
Also, if you don’t mind doing a pinch of extra work, I really like using OSC to communicate between Unity and Arduino. It’s very flexible and gives you lots of freedom if you want your Arduino to be doing other things at the same time.
I also agree that Unity isn’t the best or most stable game engine, especially now that Unreal is free to use, but it’s SO EASY (compared to other options), has pretty good documentation, and lots of community support.
It wasn’t too long ago that one could conjecture that most hackers are not avid video game players. We spend most of our free time taking things apart, tinkering with microcontrollers and reading the latest [Jenny List] article on Hackaday.com. When we do think of video games, our neurons generally fire in the direction of emulating a console on a single board computer, such as a Raspberry Pi or a Beaglebone. Or even emulating the actual console processor on an FPGA. Rarely do we venture off into 3D programs meant to make modern video games. If we can’t export an .STL with it, we’re not interested. It’s just not our bag.
https://www.youtube.com/watch?v=MZD8CVy7zZ0