AR App Development Manual: Tools, Steps, and Best Practices

It’s still a long way off from Iron Man’s fictional HUD, but augmented reality is well and truly here — and it’s getting better at merging the physical and digital. 

Once confined to sci-fi, AR is now used in all sorts of real-world applications such as police training, professional training, and surgery simulations.

Augmented reality app development is becoming more lucrative, too. The AR/VR market is expected to earn $38.6 billion in 2024 as it enters the mainstream.

Gaming (not surprisingly) leads the charge. From Niantic (the developer of Pokémon GO) to Sony, everyone’s investing in AR game development, with Meta’s Quest 3 headset release and Apple’s Vision Pro heading worldwide adding fuel to the excitement.

All this momentum may reward early adopters who are not afraid to innovate. Interested in this promising new space? In this blog, we break down how AR works and its exciting potential.

In this article:

• What is an AR application?
  • How do AR apps work?
• Types of AR applications

What is an AR application?

AR technology layers digital elements onto your real-world environment through smartphone apps and dedicated devices (such as head-mounted displays).

How do AR apps work?

Simply put, they rely on either physical markers or your phone’s built-in sensors to understand where you are and what you’re looking at.

These apps then overlay 3D models, animations, text, and sounds onto your real-world view. In more interactive apps, you can even move these objects and change their appearance.

Trivia: The concept of AR is far from brand new. In fact, the first-ever AR machine was invented in 1957. Called Sensorama, it immersed viewers in multisensory theater experiences complete with smell and sound simulations. But this technology, along with other early AR, didn’t get enough financial backing to succeed, perhaps because it was too strange for its time.

Some 33 years later in 1990, a researcher named Thomas Caudell first used the term “augmented reality” to describe special glasses that helped electricians assemble complex wiring in the Boeing 777. AR was used mostly in hangars and factories during this time.

Fast forward to today and AR is far more mass-market. Smartphones and AR glasses are putting this tech in everyone’s hands. And with so much interest and investment, new AR developments should roll out faster than ever.

Types of AR applications

Several different types of AR applications use different technology to get AR features working. Here are the main AR application types.

Marker-based

These apps rely on physical markers — typically QR codes or unique images — to trigger the AR experience. The app scans the marker with your device’s camera and then overlays digital content on top.

Marker-based AR was a big leap forward when it first came out. Remember those Nintendo 3DS AR cards from 10 years ago? They relied on the user’s 3DS camera to scan the unique image on the card (the marker), which then caused dragons to appear. That was considered groundbreaking at the time (and we have to admit, it’s still pretty cool)!

But compared to how we use AR today, marker-based can feel a bit primitive. You’d have to download a specific app for each marker and then scan the image to activate the AR effect.  That’s like needing a separate app to unlock every single face filter and background effect on social media.

We expect smoother and more immediate experiences these days — which is why most AR app development projects use markerless tracking. That said, we owe a lot to marker-based AR. It demonstrated the tech’s potential and established a baseline user experience for AR interactions.

Markerless AR

These apps don’t require physical markers. Instead, they use your device’s sensors, location software, accelerometer, and computer vision to understand your environment. They then place digital elements on top of surfaces and recognize real-world objects to overlay information that’s visible from any viewpoint.

Have you ever used an app that lets you try out furniture in your room to see how it would look? Or social media filters that add funny effects to your face? Those are all markerless!

There are two kinds of markerless AR:

• Location-based – These apps access your device’s GPS to determine where you are and then overlay AR content specific to that location. Pokemon GO is a great example — it uses your location to spawn virtual Pokemon in your real environment — as is Google Maps, which superimposes directions onto your real-world view as you walk.
  
• Projection-based AR – This type of AR uses projectors to overlay digital images onto physical objects or surfaces. It’s often used for larger-scale experiences. If you’ve ever seen museum exhibits that bring artifacts to life, you’ve experienced projection-based AR!

6 Examples of popular AR apps in the market

Now that we know the different types of AR apps, here are a few you may have heard of or used that are available in app stores today.

Pokémon GO 

This popular game (downloaded over 1 billion times!) blends the real world with the Pokémon universe. You walk around your real surroundings, encountering and catching virtual Pokémon that appear on your screen.

Google Lens  

This built-in Android phone feature uses your phone’s camera to recognize objects in the real world. Simply point it at a landmark to get information, at foreign words to translate them, or at a product to shop for similar items. Too tired to read a printed document? Give your eyes a break and ask Google Lens to read it for you.

Porsche AR Visualizer 

Dreaming of a Porsche? This app brings it to your driveway (or anywhere else you want to put it). You can customize a virtual supercar in your environment — choose the color, rims, and other features, and then walk around it and even peek inside!

Wanna Kicks 

Don’t you wish you could virtually try on sneakers before you buy them? Now you can. Wanna Kicks projects a 3D model of the shoes onto your feet using your phone’s camera, so you can see how they look and fit from different angles.

Quiver 

Looking for new ways to entertain the kids (or yourself)? Turn coloring pages into interactive AR experiences! Color a picture on a special sheet of paper, then scan it to animate the characters in 3D. 

Polycam 

This Photogrammetry app turns your phone into a 3D scanner. Capture real-world objects in stunning 3D detail, then print them out, share them online, or use them in other AR adventures. The possibilities are endless! Polycam is just one of many apps of its kind.

Different AR application use cases

While AR is great for gaming and mobile phone apps, it’s also proving to be incredibly useful across a wide range of industries. Here are some different use cases for AR applications.

Healthcare and medical

Robotic-assisted surgery has long been saving lives and money, but AR is taking it to the next level. Here’s how it’s making medical care more effective and affordable:

AR augmented surgery 

AR allows surgeons to see data and 3D visualizations overlaid onto the patient in real-time so they can make better decisions and minimize errors. This translates to faster recovery times and better outcomes, especially in fields like robotic liver surgery and reconstructions.

Medical Education 

The looming physician shortage demands new methods of training doctors faster and more effectively. AR-powered virtual reality apps now make it possible for medical students to explore complex anatomy in 3D detail, preparing them for real-world scenarios. Institutions like Case Western Reserve University are already starting to use AR headsets for anatomy studies.

Wound care management 

Traditional wound care relies on subjective assessments that can make treatments less precise and potentially uncomfortable for patients. AR steps in to generate 3D models of wounds and project key data points. Doctors and nurses can use this information to tailor treatment and dressing techniques.

Physical therapy and rehabilitation

AR can guide injured patients through movements in a safe virtual environment and even make exercises more enjoyable by introducing rewards and storytelling. AR vision and motion tracking advancements may soon allow therapists to deliver remote care solutions to improve access and lower costs.

Retail and e-commerce

With 61% of shoppers preferring to buy from brands that offer AR experiences, no wonder retailers are investing in this technology. Here’s how AR is changing how we shop:

Enhanced in-store experiences 

AR-powered product demos are making shopping trips much more personal and convenient. Check out how Marks & Spencer’s List & Go speeds up shopping with AR! The app lets you build a shopping list on your phone. Then, as you walk around the store, AR overlays on your screen point you right to those items so you don’t waste time wandering the aisles.

Gamification

Retailers are creating interactive challenges, quizzes, and rewards systems that turn shopping into mini adventures. For example, UK supermarket chain Tesco routinely gamifies grocery runs with their app’s AR features. Customers can scan products to unlock discounts and share their achievements on social media, turning pantry-stocking missions into a friendly competition.

Virtual try-ons  

Looking to buy a jacket but not sure how it will look on your frame? Simply hold up your phone and AR’s magic can overlay a realistic 3D model of the garment on your reflection. Twirl around to see how it drapes and get a general sense of the look — all from the comfort of your living room (or wherever you do your online shopping).

Mid-range brand Warby Parker was among the first to embrace AR with an app that lets customers try on eyeglasses online. This tech is also making a splash in the luxury market. High-end fashion houses like Prada and Burberry are joining the trend.

Beyond fashion, virtual try-ons are useful in furniture shopping, too. Now you can decorate your home with 3D furniture before you buy. Just point your phone and poof — sofas, tables, lamps. See them all in your space, lifesize. No more wondering if they will fit!  

Case study: Cheesecake Labs AR furniture store experience

We’ve created our own version of a virtual try-on app by developing an AR furniture store experience. This app helps you see exactly how that couch or lamp would look in your living room, all from the comfort of your couch (pun intended!).

Here’s how it works:

• Aim your smartphone or tablet where you want to place the furniture.
• Browse our diverse selection and tap to see it come to life in your space.
• Once you’ve found your dream furniture arrangement, capture photos and videos to share with friends and family (and maybe get some decorating advice!).

An app like this has plenty of commercial potential. It can help improve sales and decrease returns because customers can virtually try before they buy.

Retailers can also use in-app data to retarget their marketing campaigns, suggest complementary pieces, or highlight popular trends. Plus, the app’s social sharing feature can help generate buzz and attract new customers.

Sports and entertainment

AR isn’t just keeping sports fans more interested and watching for longer — it’s also changing how athletes train and how we consume theater.

Improve sports broadcast viewer experience 

Did you know that incorporating AR into major sporting broadcasts boosts viewer engagement by 15% and retention by 20%? No wonder ESPN now uses AR overlays to turn sports broadcasts into interactive experiences for viewers at home.

Thanks to this technology, player statistics and bios are displayed directly on the screen. For example, a batter’s average or a quarterback’s completion percentage hovers beside them during a game!

Sports training  

AR can simulate a wide range of realistic scenarios to make sports training sessions much more effective. And because AR devices can display real-time data (like angles, distances, and speed), they support ultra-precise training for professional players.

And no, AR is not just for elite athletes. Anyone can use it to play better. You can download an app that uses your smartphone camera to analyze your soccer kicking technique and then overlays a virtual model with proper form on top of your own so you can compare and adjust. You can even use AR apps to track your golf swing and learn how to drive like a champion.

Theater experiences 

Beyond sports, AR is also enriching theatrical performances. The immersive entertainment industry was valued at over $60 billion in 2019, with theater contributing over $28 million.

The National Theatre in London was one of the pioneers in combining digital with live action. In 2016, they staged an AR-VR musical adaptation and launched a program to support artists who want to blend digital and live theater. More recently, their 2019 AR-VR production of All Kinds of Limbo even scored a Sundance nomination.

Case study: Cheesecake Labs AR car customization

Ever wanted to customize a vehicle and see how it looks on your driveway using just your smartphone or tablet? We had the same thought, and that’s why we built an AR car customization app. 

Here’s how it helps you design your dream car:

• Point your device to wherever you want to place your car and tap the screen.
• Customize the car to your heart’s desire. Experiment with paint jobs, rims, or decals. You can even upload a picture to extract a custom color palette.
• Share your vision! Take pictures and record videos of your car to share with friends and family.

Aside from being a lot of fun, this AR experience can be a high-conversion marketing tool. It allows users to see what a vehicle would look like in their own garage so they can decide on a model, color scheme, and add-ons before they visit a dealership or auto body shop. Car companies can also use the data to see what trends are popular and adjust their marketing/product development accordingly.

Plus, an app like this could potentially integrate with targeted advertising platforms to show users ads for customization options they’ve experimented with. And as users share their customized creations on social media, they entice other car enthusiasts to get in on the action.

Tourism

The global virtual tourism market is expected to more than double by 2028, reaching over $847.95 billion. Here’s how AR is injecting a dose of excitement — and a whole lot of convenience — into travel:

Virtual and augmented museum experiences

These days, you can virtually try on historic costumes at de Young Museum in San Francisco and come face-to-face with extinct species at the National Museum of Natural History in Paris. AR can overlay historical artifacts with detailed information, recreate lost structures in 3D, and even let visitors interact with virtual exhibits!

Immersive navigation

Pretty soon, it will be almost impossible to get lost in a new city. AR is increasingly better at overlaying real-time directions and highlighting points of interest in our surroundings.

Google Maps pioneered this feature when it launched Live View in 2022. Simply point your phone’s camera at any spot on the map, and the app instantly overlays details about that location on your screen — even if it’s behind you or around a corner. Live View also provides access to useful information like wheelchair accessibility, EV charging stations, business hours, and price ranges.

AR travel guides

Are you still using generic guidebooks? It’s time to give AR apps a go. They can personalize your travel experience, suggest hidden gems, and even translate street signs and menus in real time. The Singapore government is now using Google’s ARCore Geospatial AI to help inbound travelers discover the country’s best cultural and culinary offerings.

Language Translation

See a restaurant menu or shop sign you can’t understand? With AR-powered apps, you can just point your phone and get an instant translation. Some of them work offline, so you don’t have to worry about Wi-Fi access or data charges.

Education

Augmented reality is changing the classroom with immersive, hands-on instruction:

Interactive learning

AR overlays transform textbooks into interactive platforms to make complex concepts easier to understand. Snapchat recently teamed up with the virtual learning platform Inspirit to create an AR-based STEM curriculum of immersive simulations and 3D models.

AR vocational training

Learning by doing takes on a whole new dimension with AR simulations. They can provide trainees with safe but realistic practice environments for everything — from welding to pipefitting — while reducing training costs.

Lincoln Electric RealWeld and Columbus State Community College have partnered to integrate AR into welder training programs. Some countries, like Nigeria, are also investing in AR training to quickly equip their workforce with practical skills.

AR for professional development

AR can be used to create immersive simulations for professional training. For example, several aviation companies now use AR to train aircraft mechanics. These training platforms can superimpose information and instructions directly onto real-world engines, allowing mechanics to see step-by-step procedures and identify specific parts.

The state of the AR market today

 Thanks to a recent wave of consumer-friendly devices, AR is becoming a more mainstream (and more practical) reality.

Apple Vision Pro’s spatial computing

Even though some critics call it a head-scratcher, there’s no denying that Apple Vision Pro is a powerful augmented reality development tool for creating inventive experiences.

This $3,499 headset packs a punch with impressive AR tech, but Apple refuses to call it an AR or VR device. Instead, they’ve dubbed it a spatial computer.

This hints at a broader vision — they’re leaving it open for developers to unleash its true potential (which is likely why they unveiled it at their worldwide developers conference). The strategy seems to be working. Developers have used Apple’s AR development kit to create an ecosystem of more than 1,000 apps and counting. 

Want to learn more about the Vision Pro’s potential? Check out our post about creating cross-platform apps for the Vision Pro.

Meta’s AR smart glasses

Meta’s smart glasses were launched as the more affordable and approachable alternative to Vision Pro. These stylish AR Ray-Bans promise to integrate seamlessly into everyday life.

You can take calls, get info about your surroundings through a built-in AI assistant, or capture photos and videos, all hands-free! And at $299, they’re a glimpse into a future where AR is as familiar and accessible as smartphones.

While they were well-received by the tech community, these glasses don’t live up to the sci-fi dream (at least not yet). Their battery life is average at best, and the AI features are a bit too slow.

Still, they’re bringing AR to the masses and are likely to inspire a wave of similar offerings from other companies, making AR more accessible to everyone.

AR adoption and user engagement

While widespread consumer adoption is still on the horizon, AR is one of the fastest-growing segments in tech. Here are the numbers:

• By 2028, a staggering 3.674 billion users are expected to use AR/VR in some shape or form. That’s over half the world’s population.
• Each user will spend $6.20 on AR/VR products and services, on average.
• The AR/VR market should reach $38.6 billion in 2024 and grow at over 10% annually to $58.1 billion by 2028.
• AR/VR software is the largest market segment, with projected sales of $13 billion in 2024.

There’s clearly massive potential waiting to be explored. Companies that invest in augmented reality app development now will be at the forefront of this exciting new frontier.

How much does it cost to develop an AR app?

Like always in software development, the answer to this question is, “It depends.” Creating an AR application involves many variables, including the number of 3D models, rigging of each 3D model, what kinds of interactions and animations will affect the objects, what kind of device will run the experience (i.e., handheld mobile devices or stationary tablets), and much more.

Creating a simple AR app that plots a single object on the scene using tools like 8thWall can be relatively simple. If the 3D model is ready at the beginning of the project, creating an app like this could take around one month and cost less than $30,000.

Consider your app’s complexity and feature count

Timescales and costs increase when the app includes more features. Let’s say you want to build an AR app that plots a lifesize object that can be manipulated in position and size.

Plus, the user can play around with animated textures that need to be applied on top of the object mesh. Let’s also add in some recording and sharing capabilities for good measure. A more complex project like this could take around three months to develop and cost around $100,000. 

Of course, the more variables you add, the longer development will take and the more the project will cost. For example, when you add multiple objects to the application, you also need perfect performance.

If meshes need to attach to physical objects with extreme precision, that can also be a challenge. If the user needs to be able to interact with the 3D object’s rigging, that’s yet another level of complexity. 

All of these things inform whether an AR application project will cost $20,000 or $200,000.

How to develop an AR Application: 6 essential steps

1. Define your target and mission

What problem will your AR app solve? What experience will it create? Clearly define the core functionality you want to offer and discuss it with your AR app development company.

Will your app thrive on mobile devices (iOS or Android) or be a web-based experience accessible through any browser? Consider your target audience and the type of experience you want to build. AR mobile apps are more immersive and location-specific, while WebAR is easier to access. 

2. Choose your AR development approach

How do you want your AR app to interact with the real world? Each development approach shapes your app’s design and functionality in different ways. In general: 

• Marker-based is best for apps that require precise object recognition, such as educational apps with interactive elements that are triggered upon recognizing a specific marker.
  
• Markerless is ideal for situations where users don’t need a physical marker to activate the AR experience, such as apps that let users measure objects in their real-world environment.
  
  • Location-based AR is perfect for apps that tie the AR experience to a specific geographic location. This could be a city guide app that overlays information on points of interest as users walk around or a game that uses real environments as playing fields.
    
  • Projection-based AR is often the best choice for large-scale interactive experiences or public installations. It uses projectors to create AR elements on surfaces, walls, or even people. This type of AR is less common but can be very impactful.

3. Select your AR development platform

Choose an AR software development kit that suits your project’s needs.

• Do you want a native app? ARKit is specifically designed for Apple devices while ARCore is for Android development.
  
• Do you want a cross-platform app? Try combining Unity3D with Vuforia. It’s a potent duo: Unity3D offers a feature-rich development environment for building AR apps that run on multiple devices using a single codebase, and Vuforia enhances it with AR capabilities like image and object recognition, surface detection, and multi-user support.
  
  However, both Unity and Vuforia require a learning curve. Complex AR experiences also demand a lot from mobile devices. Think about your project’s goal before proceeding.
  
• Do you want to build a WebAR experience? AR.js is a great choice for web-based apps (no download required). It’s especially useful for building quick prototypes, educational projects, or easily shareable AR filters.
  
  However, keep in mind that AR.js might have limitations compared to native apps. Processing power and interactivity can be slightly restricted, making it less suitable for complex AR experiences with heavy 3D graphics or demanding real-time interactions.

4. Streamline your development workflow with low-code AR platforms

Consider using a specialized, low-code platform to build your AR application. They can provide pre-built features, simplify complex tasks, and allow for faster prototyping to save time and resources. We’re big fans of 8th Wall, a great low-code platform for WebAR. It’s a great way to accelerate the development process and improve efficiency. 

5. Create mockups

Before coding begins, create mockups to visualize your augmented reality app’s interface and user flow. Make sure that everything is intuitive and easy to navigate. 

6. Carry out rigorous QA testing

Extensive QA testing ensures that your augmented reality app functions flawlessly across different devices and platforms. Always test for usability, performance, and stability. +

Work with a top-notch AR app development company

Have an idea for Apple Vision Pro app development? Need help starting your Android AR development project? 

Cheesecake Labs offers all the AR development solutions you need. We have the team, tools, and expertise you need to launch a next-level AR experience. Let’s chat!