Rapid prototyping in AR and VR

Introduction

The use of augmented and virtual reality applications has been steadily increasing over the last few years, with manufacturing companies implementing the technology within their organisations and the ever growing catalogue of consumer apps driving interest in augmented, virtual and mixed reality. According to the market research report published by P&S Market Research, the global augmented reality and virtual reality market is expected to reach $94.4 billion by 2023. Smartphone capabilities are continuing to improve, granting AR and VR capabilities to millions of users and the introduction of software solutions such as Apples ‘AR Kit’ means it is now easier than ever before to push an augmented reality application out to a market that has the ability to use it.

The growth in popularity of AR and VR means that digital agencies are beginning to explore the field more and so the demand for tools to aid the design and development of such apps is growing. Here at Komodo we have engaged in a number of small R&D projects to test our abilities and nail down a workflow for designing AR and VR applications. Whilst doing this, we have discovered a few tools that have greatly sped up the design process and allowed us to rapidly prototype the UI’s that we are creating. This short article will discuss our thoughts surrounding these tools and how they can be used prototype and test AR and VR applications.

Sketch to VR

Sketch to VR is a sketch plugin that has been around for a few years now, however, the speed at which it allows you to iterate concepts is what makes it so impressive. The plugin transforms a small sketch file into an HTML file borrowing off of Mozilla’s Aframe JS framework. This can then be quickly viewed on a mobile device using a local https server and in turn viewed using any VR headset, such as Google Cardboard. We have mostly used this plugin at Komodo to quickly iterate ideas before transferring them into a more high-fidelity prototype. Being able to quickly save out a new UI layer and refreshing the prototype in seconds allows us to quickly mock up and test scenarios with little effort.

Sketch to VR – https://github.com/auxdesigner/Sketch-to-VR

The sketch file that accompanies the plugin includes a ‘background’ and ‘ui’ artboard, allowing you to use any 360 degree image to visualise the UI you have created against. This can be useful, as testing the UI on a background that will closely mimic the actual environment that the app will be used in will help to iron out any major issues with the legibility of text etc.

The only downside to using this plugin is the restrictions that it places upon the user. You are constrained to a cylindrical strip running 360 degrees around you to place your UI within, this does throw up some issues with the warping of images and UI elements, however, in its simplest form, this plugin allows you to quickly iterate and test UI elements within a VR environment.

Download Sketch to VR for yourself at https://github.com/auxdesigner/Sketch-to-VR with a ~5 minute set up time; it is a must for designers looking to quickly iterate VR UI ideas.

Torch

A new addition to the app store is “Torch”, an AR app lead by CEO Paul Reynolds and Josh Faust, who previously led Magic Leap’s software development kit and applications efforts. Torch allows the user to explore a range of features including the ability to place flat images and 3D models in 3D space. Possibly the most impressive thing about the app is it’s performance – running the app on an iPhone 6s provides no issues to the Apple A9 chip, however previous models may struggle. After a number of days using the app to create some small prototypes, we never experienced a crash and performance was always smooth.

The main draw to the Torch app for prototyping is the ability to create a very basic UI flow within the app. This is achieved by creating a number of ‘Scenes’ and linking them together using tap interactions. Within each scene, you are able to place a number of  UI elements (currently the application only supports static image files, however, they are expanding to include other files such as video and .gifs). Every element within a scene can be enhanced with an interaction. These can range from transforming the object or linking to another ‘scene’ with the next screen within a UI flow. The only downside to this being the inability to create hotspots to click through, instead, the scene change is triggered by tapping anywhere on the image you have placed within the scene. This limits the complexity of the prototype you are able to create, but the ability to create even just a simple interactive AR prototype to visualise a user flow can be very valuable for user testing and presentation purposes.

Another benefit of this application is the apps integration with Dropbox and google drive, no more exporting images and having to transfer them manually to your device. This speeds up the workflow of creating a prototype greatly. Furthermore, the ability to import Sketchfab and Poly 3D models opens more doors to explore within the AR prototyping world. This is something we are yet to explore fully, but initial tests with 3D objects pre-loaded into the Torch app are very impressive.

Torch app integration with Sketchfab: https://blog.sketchfab.com/torch-launches-intuitive-ar-development-tool-with-sketchfab-support/

Check out Torch for yourself at https://www.torch.app

These two examples provide a strong starting point to begin to prototype AR and VR designs at a basic level. They enable us to test initial design ideas and carry out small user tests within 3D environments. This quick workflow gives us the capacity to quickly iterate concepts and ensure that designs are fit for purpose. Once we have iterated our concepts a number of times we move to higher fidelity programs, which allow us to create a more detailed and complete prototype for further testing. We will continue to keep our attention on the ever-changing landscape of the AR and VR market, with new apps and tools being released regularly, there will inevitably be more ways to prototype virtual experiences as the sector grows. Watch this space.