Have you ever wanted to make your own augmented reality app, but don’t know where to start? Or, are you just interested in AR technology more broadly?
In this series, we’ll demonstrate how to use Unity’s game engine to make a small AR app that can demonstrate important principles of the technology. To do so, we’ll dive deep in to Unity’s AR Foundation framework.
AR Foundation allows you to use ARCore from Google and ARKit from Apple at the same time (the primary frameworks for building mobile AR experiences). Think of it as an interface that will use ARCore and ARKit in the background.
By the end of this series, you’ll have an app similar to IKEA’s AR app:
We’ll start with the basics: setting up Unity, setting up Unity for Android and iOS, and an overview of how augmented reality works.
What is AR?
Augmented reality (AR) uses computer graphics or video composited on top of a live video feed to augment what a user sees—essentially creating an interaction between real and virtual objects.
Augmented reality can be used for a wide range of purposes, including in games, for medical training, in retail, design, and more.
How it works
All these examples listed above (games, design) require understanding the physical world from a camera feed—i.e. the AR system must understand what is in the world and where in the world before adding digital objects to a scene. This is achieved using computer vision. The process itself is complicated and could fill an entire topics on its own, but essentially we use AR frameworks to detect plane , distances, and other key 3D calculations.
To get started, download Unity. You have to sign-up and accept their license too download. Installation is pretty straightforward—just make sure to check these options so we can work across platforms:
I am using Unity 2019.4.9f1 (LTS) on a Linux (Kubuntu) machine. You can download the same Unity version OR can install any latest one. The steps are similar regardless of the operating system.
Create a new project (select 3D project) and name it ARWorld:
Understanding Unity’s Interface
Here’s a quick look at some of the core pieces of Unity’s primary interface:
- Project: The Project section holds your project files and structure.
- Console: The Console is where you can see the logs, errors, and console output.
- Hierarchy: The Hierarchy is where you can see the game components that are currently in your scene (game-level).
- Scene: The Scene (a.k.a viewport) is where you can see your game level and how it looks.
- Inspector: The Inspector is where you can set the properties of your gameobjects (components that you drop in your game/scene).
Setting Up Unity
First, we need to setup Unity for the specifics of our project. We’ll start with Android. Following the gif above, go to File → Build Settings and change the platform to Android. Click on switch platforms.
Next click on Window → Package Manager:
Install the following packages: AR Foundation, ARCore (if using Android), and ARKit (if using iOS):
Next, we’ll change our Project Settings (found under the Edit menu). First, go to XR plug-in management and select Android, choosing ARCore. For iOS, it will be ARKit in the iOS section. From here, for clarity, we’ll assume you’re working with Android and ARCore.
Next go to Player Settings → Other Settings. From here, make sure the Auto Graphics API is set to Android version 7.0 (minimum)—this is the Android version in which ARCore was introduced. Uncheck multi-threaded rendering (it won’t work with AR).
Next, head over to the XR section and save these settings. Click on Create, keeping the default name:
Setting Up Our Device
In order to test our application and to further develop it, we need to run it on our device. There are certain steps that are needed to allow our device to install our app.
This can be done by enabling developer settings. Developer settings(access) allows us to do perform certain operations like install app via USB, debugging that are not allowed for normal users.
This step varies from vendor to vendor, so I suggest look up the information specific for your device. But generally speaking, you need to go into your device settings, find build info, and tap 7 times on the build number or kernel . For more detail, you can look at the Unity docs (Android linked here):
Once the developer options are enabled, make sure both “Install via USB” and “USB Debugging” are checked:
Go back to the Unity interface and open up the Build Settings (File → Build Settings). Connect your device and click the refresh button. It should be listed as follows:
With all these prerequisites setup, we now need to test them, just to make sure we’re ready to go. To do this, we’ll create a very small test app.
Right-click on the Hierarchy section and choose XR → AR session.
We also need to add origin point in our app . Origin is (0,0,0) coordinates. Right click on the Hierarchy secition and choose XR → AR session origin.
The AR Session object provides a lot of built-in setup that our scene needs in order to work as an AR scene. It gives us a way to query the physical world around us, and also a way to manipulate the scale of our virtual world.
Notice that our AR Session Origin already has a camera—as such, we don’t need the default camera (main camera) anymore. Select it and delete it.
Next, we’ll set out AR camera as the main camera. Head over to the inspection section and select “Untagged” to main camera. Tags help you identify GameObjects for scripting purposes. They ensure you don’t need to manually add GameObjects to a script’s exposed properties using drag and drop, thereby saving time when you are using the same script code in multiple GameObjects.
Although this step is not needed but it’s a good practice as recommended by Unity. To learn more about good practices you can refer : https://docs.unity3d.com/Manual/BestPracticeGuides.html
Next, we’ll create a test object( cube) and place it in our scene. Right click and select 3D Object → Cube.
Change the coordinates of the Cube, as shown below:
Next, go to File → Build and Run. It will ask for the filename—you’ll need to provide the application’s apk file (ARWorld.apk).
Once the build is finished, the app will launch on your device. Try to move your camera a little, and you should see the cube that we created in our scene. With this, we’ve set up our project, hooked it up to Android and ARCore, and implemented our first (very simple) AR object!
In the introduction to this series on augmented reality and Unity’s AR Foundation framework, we learned some basics about AR, Unity’s core interface, how to setup our device, and deploy our very basic test application on Android.
In the next part of this series, we’ll actually start working on our IKEA clone app. We’ll learn about plane detection and placing objects through user interaction with our app.