AR, MR, VR, XR: these are abbreviations without which the second half of the 2020s cannot be imagined. Today, technological advances add newer and newer abbreviations and technical terms to our languages; in some cases, these are worth learning even for those who do not work in the field of IT.

In connection with these ‘four Rs’, the UX-UI profession is particularly affected since its primary aim is to support interaction with objects and software. The ‘R’ stands for Reality, and a shared feature of all of them is that they enrich our usual environment by displaying (visual) information.

The ‘four Rs’ turn the market upside down just as much as the computer or the smartphone did – at least this is what investors hope for.

Instead of hopes, let’s look at facts, including what has come true and what has not, and let’s take a short trip into the future and look at what we can expect in the coming years.

UX is further developed with the help of new technologies

The UX profession has already taken advantage of innovative solutions, such as voice commands and the touchscreen. Things that were at first only novelties for tech gurus and ‘nerds’, quickly became accessible to a wider audience, which created a demand for simple and efficient handling.

To the computer, it is not important whether it receives the instructions and data on a punch card, floppy disk, entered through a keyboard, or in the form of voice commands, to the users, it is the exact opposite.

Augmented reality solutions provide more and more opportunities

Ergomania, one of the most significant UX-UI companies in Europe, is continuously monitoring the development of technologies that make the lives of users easier and attempts to take advantage of the opportunities within these for the benefit of its customers.

Virtual reality (VR) has already been available for at least thirty years, it was high time it started to develop! It is important to note that these new technologies are arriving from different places and are attempting to fulfil different demands. In spite of this, there are significant overlaps between them.

What does Extended Reality (XR) mean and how does it relate to VR, AR, and MR?

These days, new and emerging technologies insist on using abbreviations and virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR) are no exceptions either.

The ‘four Rs’ can be abbreviated as 4R, or in Hungarian, 4V; however, the latter might rather be read as Volts or perhaps mistaken for the Visegrad Four’s abbreviation, so let’s just stick to Rs, even in Hungarian. Fans of Star Wars may call it Forar too, seeing how R2 has become Artoo.

Following the ‘forays into the nomenclature of 4R’, let’s look at each of the four expressions in alphabetical order, noting that the hierarchical order would be different, just as the chronological one: VR would be followed by augmented reality (AR) and mixed reality (MR). Today, we can also talk about extended reality (XR).

Augmented Reality (AR)

Right from the start, it is worth taking into account that the translation of these terms into Hungarian is a source of some confusion. The English word ‘augmented’ was originally translated as ‘kiterjesztett’ (extended) but the new term ‘extended reality’ is the one that actually pertains to extension.

‘Augmentation’ is more correctly translated as a word referring to enhancement and expansion. It can be used to imply extension but it would be more correct to use ‘Bővített Valóság’ (literal translation of Augmented Reality); however, the term ‘Kiterjesztett Valóság’ (literal translation of Extended Reality) has already taken root as the translation of AR.

Mixed Reality (MR)

MR falls somewhere between AR and VR, as it unites the real and the virtual worlds.

Mixed Reality (MR), sometimes called ‘Hybrid Reality’, is the amalgamation of the real and the virtual worlds in order to create new environments and visualisations, where physical and digital objects exist alongside each other in real-time and they are able to interact with one another.

What are the advantages of MR?

For example, Snapchat filters represent an AR function, while the ability to assign 3D content to an object in an application supporting repair activities of machinery can be thought of as an MR function.

In colloquial language, MR combines the physical and digital worlds, primarily attempting to achieve actually useable functionality, as opposed to AR, which is for entertainment above all else.

Therefore, MR can bring significant changes in the dominant trends of a variety of industries, including manufacturing, design, construction, medicine, education, and research.

Virtual Reality (VR)

VR is short for Virtual Reality and generally refers to real, immersive technologies that place the user in an imagined environment that mimics the real world.

Virtual Reality entered mainstream conversation a few years ago and the industry has been developing quickly ever since. Users always use a headset to see visual images and hear what is happening in the virtual world.

Extended Reality (XR)

Extended Reality is a collective term. It applies to all real and virtual environments generated by computer graphics and wearable devices.

This is often achieved by placing computerised text or graphics into a real or virtual environment, or even a combination of the two. XR solutions typically enhance functions already present in VR and AR platforms.

The hierarchical order is different among ‘realities’

With the appearance of MR, the creation of a collective term, XR, became necessary. XR includes augmented reality (AR), virtual reality (VR), and mixed reality (MR) as well.

Although all three ‘realities’ have shared and overlapping characteristics and requirements, each of them has different goals and uses different underlying technologies.

Furthermore, augmented reality (AR) and mixed reality (MR) are not interchangeable terms. The general distinction is as follows: all of MR is AR but not all of AR is MR. MR is interactive.

Why are ‘additional realities’ needed in the field of UX-UI?

The answer is simple: because the ‘traditional’ reality often proves to be insufficient. Let’s look at an example. You want to travel by car from A to B but you do not know the way. You are travelling alone. What do you do?

1. You memorise the route, all junctions, and forks in the road so that you will find your way even in rain or at night?
2. You look through a printed map after every few turns?
3. You enter your destination into the navigation application of your mobile phone or onboard system and let the machine intelligence guide you?

Since the majority of people do not fall into the first category and since poring over paper maps is often either difficult or can lead to an accident, the majority of drivers will choose option 3.

By doing so, they are already using an ‘augmented reality user interface’. This can sound incredible at first since the term ‘XR’, that is Extended Reality, has become synonymous with VR headsets and solutions such as Oculus or Google Glass.

While we do not have special devices called ‘XR-ready laptops’ or ‘XR-enabled telephones’. This is because all devices that have AR, MR, or VR technology built in can already be classified as an ‘XR device’.

One of the key features of UX-UI is to ensure easier and better useability

Our UX-UI experts at Ergomania work on designing the user experience. Their task is to make the practical experience of users the best possible, thus reinforcing their engagement with the product in question or even making it possible to view a museum’s collection virtually.

XR also provides opportunities for designer creativity in a variety of areas. Let’s take the Cleveland Museum of Art as an example, where additional information about certain exhibited items is displayed with the help of an AR application.

This fulfils multiple purposes at once:

1. It provides visitors with an enhanced experience (resulting in extra income, better reviews, and a better financial year),
2. It communicates extra information without the need for extra space (the latter being severely limited in a museum),
3. It makes it possible to experience the collection more vividly (this is also important to museologists from a professional point of view).

Therefore, a true win-win situation is created, in which UX-UI professionals play a big part, even if the majority of people do not think too much about this.

The ‘additional realities’ would have been needed in the past too

Thus it can be said that there would have been a demand for XR even in the past. To be fair, if we interpret the term as widely as possible, even visual arts can be thought of as XR since they add a new layer to otherwise everyday objects (walls, pieces of wood, canvas, paper, ceramics, metals, etc.) that carry additional meaning, which was not naturally present in these objects.

Artists extend their own subjective perspective on reality to the material, thus applying an additional dimension of reality to it. It is not likely to be a coincidence that in the case of both AR and VR, artistic and visual content is what makes up the majority of the additional reality.

UX-UI opportunities in ‘additional realities’

It is typically true that people are taking advantage of the opportunities provided by AR, MR, and VR technologies more and more. Navigation software, financial applications, and museum exhibit apps are all excellent examples.

Additional realities may also be becoming more and more popular because they each have their own strengths.

We love AR – but it requires powerful devices

In AR, we primarily experience an imagined reality projected onto our physical reality, where the two realities appear to be interacting with each other.

Its advantages are its relatively easy implementation and gamification; however, its disadvantage is that it requires a sufficiently powerful device and relies too heavily on how the device is handled.

AR apps that take up too much of a device’s resources create a negative customer experience: the device will overheat, slow down, and the battery charge evaporates like people’s zest for life after an hour-long presentation.

VR has been trying to take over the markets for decades

The key advantage of VR is that with the appropriate equipment, it is able to make our brains believe that it truly is in another reality. The VR headsets, which everyone is familiar with, and the associated software have undergone enormous changes over the past approximately thirty years and had the potential to revolutionise the entire gaming industry, 3D design, and even medicine.

The problem is that it is cumbersome, difficult, and causes nausea or even sickness in the case of some users. The reason for this is simple: the information arriving from the eyes is not aligned with the information arriving from the other senses of the body, such as our sense of balance.

If you are sitting on your couch while also experiencing a rollercoaster ride, the brain has difficulty deciding whether to believe the eyes or the other senses.

It appears that MR will finally bring the much-awaited breakthrough

MR combines the advantages of AR and VR while getting rid of as many disadvantages as possible. The essence of Mixed Reality is to display additional virtual information in a way that allows us to connect with it and interact with it.

Of course, this brings up the legitimate argument that this was already possible ten years ago in AR apps and it is possible to handle machinery with a VR headset too if someone takes the time to program it.

The difference is best understood in the following way:

1. In the case of Arm a device generates the actual environment with the help of its cameras, on top of which it overlays the additional reality (games, museum exhibitions, etc.).
2. VR provides a view fully independent of the physical environment, on which additional information can also be displayed.
3. In the case of MR, the physical environment is visible in itself and the elements of the additional reality are projected by the software onto a transparent surface.

There have so far been few examples for the latter, most of the ones available are smart glasses. There are several concepts (such as smart contact lens); however, the one that has already been implemented in practice and represents the state of the art is HoloLens by Microsoft.

XR in Fintech and business life

As usual, the world of finance and companies quickly discovered how to make forms of augmented reality truly profitable.

An example of this is when we can upload an image of our room to a furniture store’s website to check what the selected couch, armchair, or Gjöttenbrk aquarium stand would look like in it or when we can put filters on selfies, such as an animal head, which then mimics the facial expressions of the user.

However, Ergománia is committed to trying out innovative solutions in addition to using the tested methods and techniques, provided that these innovations make the daily lives of our customers easier or result in a more efficient user interface, such as one controlled with voice commands.

From the following good practices, it is not out of the question that some will soon emerge on the Hungarian market as well.

VR and Fintech

DBS Bank of Singapore and Century 21 Hong Kong, a residential property lettings agency, offer a service that helps future apartment buyers in Hong Kong purchase their property and obtain financing.

With the help of DBS Home360, the smartphone app developed by DBS, buyers can browse the property listings of Century 21 and view 360° VR images of the apartments they are interested in.

In Japan, GMO Click Securities launched GMO-FX VR Trade, a smartphone application that can be used in conjunction with a VR headset at the end of January 2017. The application simulates a virtual trading floor, which users can navigate with the help of their headsets.

Users can zoom on the forex graphs, select currency pairs, and submit buy and sell orders with the help of an eye-tracking interface.

We have to add, that the whole idea became practically a moot point due to the pandemic since everyone was holding their meetings via zoom calls and in virtual spaces anyways.

AR and Fintech

It was a car sales company that began using AR in the most imaginative way. Capital One published its own car loan application, Auto Navigator. First, users undergo a preliminary assessment for the car loan.

After this, they can begin browsing the inventory of car dealerships near their place of residence. When users visit the dealership in person to view the vehicles, they can scan the QR-codes of the cars with their smartphone’s camera and Auto Navigator creates a bespoke payment plan for the specific vehicle in question, including the sums of any advance payments and monthly instalments.

MR and Fintech

Citigroup and 8ninths, a US-based startup, jointly developed a holographic workstation, which can be used with Microsoft’s HoloLens MR device (we will talk about this gadget more later on). The workstation is created for financial traders, who have to monitor large quantities of market data.

The financial data are displayed as a combination of 2D and 3D holograms, making it possible for traders to gather information about the market environment at a single glance, without having to glean it from multiple screens, thus improving their efficiency.

In Japan, Gaitame.com, a retail forex broker developed a highly similar software called MarketMR. This is also a solution for assisting in trading, which is also used with the help of HoloLens.

First published in July 2017, MarketMR displays currency exchange rates, graphs, and economic news in the form of holograms placed inside a virtual space.

Do we really have to say goodbye to the user interfaces we are used to?

It is important to clarify what the metaverse is not going to be: it is not going to be the own product/service of a single company and it is not going to replace reality.

The metaverse has existed for a long time; however, other than a handful of fans, people have so far only heard about it from sci-fi movies such as Ready Player One.

Since 28 October 2021, Facebook has been operating under the name ‘Meta’, suggesting that it is one and the same with the metaverse; however, this is only a marketing trick. The building of the metaverse will require extraordinary levels of cooperation and collaboration between several of the largest organisations around the globe.

What is the metaverse actually?

A metaverse is a futuristic place where every attendant is present in a virtual form. The term itself is based on a 1992 novel by Neal Stephenson titled Snow Crash.

The figures called avatars are controlled by living people, who can be anywhere in the physical world. The first truly successful metaverse, which is still in existence today, is Second Life, which has been live since 2003.

We could also put all open-world massively multiplayer online (MMO) games in this category, where players control avatars and they are able to interact with each other, chat, and live a virtual life in general.

Among these, Roblox is really popular among young people but even Minecraft could be considered to belong to this category as it has servers that fulfil the requirements for the metaverse. Not incidentally, these requirements are the reason why the metaverse is a market where entry as a large player requires billions. USD, rather than HUF of course.

The requirements for the metaverse are the following:

• Massively scaled,
• Continuously operating,
• Data continuity is ensured,
• Unlimited number of users,
• Use through an individual presence (avatar),
• Interoperability or multiplatform use,
• Real-time rendering,
• 3D virtual worlds,
• Synchronisation between devices.

Facebook has also entered the metaverse market

Thus, Facebook entered the metaverse market as the umpteenth player but one with significant resources when it published Horizon Worlds. This is currently only open to people 18 and older, living in the United States or Canada.

The pandemic gave momentum to Horizon Workrooms since the lockdowns lasting weeks or months and the continuous online meetings from the home office, which became the norm for years even, turned the lives of many companies and employees upside down.

Facebook, apologies, Meta states that Workrooms is their flagship, which makes it possible for people to work together in the same virtual room regardless of their physical distance from each other.

The Horizon Workrooms system works both in virtual reality and on the internet, and it was designed to improve team cooperation, communication, and joint remote work with the power of VR.

The Hungarian economics magazine, HVG, also dedicated a shorter article to the topic and the only thing sure for now is that this market, which is already significant, will grow to a gigantic size in the coming decade. Especially since Microsoft has also entered and immediately began to dominate it.

The metaverse and cryptocurrencies

It is self-evident that a thing as big as the metaverse cannot come into being without monetisation. Since it is the 2020s, it is understandable that cryptocurrencies are used primarily during transactions.

In this regard, MANA is especially prominent, which is Decentraland‘s own token. Decentraland is a fully decentralised virtual world, where users irrevocably own their virtual items.

The MANA token is also used as a currency inside the virtual world for purchasing goods, services, and land.

SAND should also be mentioned, which is the token used by the Sandbox metaverse. Sandbox is a community-driven (gaming) platform, where users can also be creators and monetise their content.

Will Microsoft win it all with the HoloLens 2?

Microsoft HoloLens 2 is the answer from the Redmond company to the smart glasses and VR headsets of their rivals. Microsoft claims no less than that HoloLens 2 makes precise and effective work possible without the use of the hands.

However, it is immediately clear from the demonstration animations that hands will definitely be required when working with the HoloLens 2, not only for using the device but for completely new types of XR tasks.

HoloLens 2 practically makes the holographic activities well known from sci-fi possible: the system displays virtual 3D bodies and images and tracks eye and hand movements, in order to ensure that the holograms react to ‘being touched’ the same way as real objects do.

It is primarily targeted at three key sectors: manufacturing, healthcare, and education. The official demos by these corporations are naturally always exaggerating but the various user tests (e.g. UploadVR, Paul Easker, or a comparison of Facebook Meta and the Microsoft metaverses on CNET Highlights) show that what Microsoft promises overlaps with the actual possibilities to a significant extent.

Some of these as a teaser:

• Virtually disassembling or re-assembling any item or object,
• Typing without a keyboard or monitor in front of the eyes without projectors,
• Virtual presence (all participants appear to other participants as a digital avatar),
• Drawing, creation, performing administrative and office tasks,
• Playing AR/MR games.

It can be controlled with gestures and voice at the same time and its biggest advantage is that it provides the advantages of a VR headset or AR smart device while we are also able to see our environment.

Is Apple staying out of the metaverse? They are preparing for a bigger hit

The great rival, Apple, appears to be a step behind this time (although true that a Windows version was made for smartphones before the iOS too) since they are only now planning to launch their own ‘smart headset’ (metaheadset? iVR? iHeadset?); however, they do not intend to enter the fray between metaverses at all.

The metaverse-reaction by Apple raises some questions since the latest articles claim that they truly do not wish to deal with Meta. Among others, they firmly excluded the possibility that they would provide it access to their own VR headset.

Since the 2017 WWDC developer conference, where the company presented the AR Kit, Apple and its partners have been focusing on AR and not VR. Gurman further confirmed this by stating in his Power On newsletter: Apple will not participate in the ‘metaverse zeitgeist’.

It appears that Apple chose a completely different path and the challenge it wishes to tackle is not smaller by any measure: its main goal is to switch from mobile phones to AR. Apple must have big plans for its headsets. The aim of the company is to ‘replace the iPhone with AR within 10 years’ explained Kuo in a note to investors.

It’s true that they talk about an Apple metaverse in this video summary; however, in reality, they are talking about a pair of AR glasses, since one of the most important requirements for the metaverse is not met, that is the interaction with virtual reality through an avatar.

Instead, Apple is more or less aiming for the same thing that Google wanted to achieve with Google Glass.

What is the future of augmented reality?

It is worth being careful with big promises since the ups and downs of VR show that the majority of the market prefers to stick to traditional solutions.

However, the appearance of HoloLens and the metaverses may bring forth a future that is not as VR-focused as the sci-fi near-future of Ready Player One but allows additional realities to become integral parts of our everyday lives.

Let’s imagine that people on the street, in restaurants, on public transport, or in workplaces are no longer tapping on their phones but waving their arms in the air. It would be a funny sight and also quite realistic; however, due to the price tag, it is not likely in the next 10-15 years.

Because of this, we can say that we are almost certain that the UI solutions that have become traditional by now will stay with us and be complemented by new technologies, maybe appear in virtual realities, but in the form of keyboards or identification/certification solutions for example.

Or if Apple’s plans come true, in ten years, we will not even need a phone to ‘tap on our phone’. An AR headset will be enough.